Implementation method of heavy-haul locomotive synchronous operation safety auxiliary system

Abstract

The application discloses an implementation method of a heavy haul locomotive synchronous control safety auxiliary system. The system comprises a host computer, an intelligent camera and a microphone array. The host computer comprises a safety auxiliary system processing unit containing a control program, a CIR access unit and a voice recognition unit. The system is connected with a CIR device and an MMI operation display unit outside the system through the host computer. The MMI operation display unit is connected with an external loudspeaker. The system is a beneficial supplement to the original synchronous control system of the heavy haul locomotive. The machine vision is used to identify the main control locomotive control action, and the CIR device is used to remind the slave control locomotive control content through voice, so that the slave control locomotive driver can review the locomotive state. Meanwhile, the system can analyze and judge the synchronous state of the master and slave locomotives in real time, automatically connect the CIR voice communication of the master and slave locomotives in an emergency, help the master and slave locomotive drivers to complete the complex calling operation, facilitate the master and slave locomotive drivers to focus on the locomotive control, and further improve the heavy haul locomotive driving safety guarantee.

Description

Technical Field

[0001] This invention relates to the field of railway locomotive safety control technology, and in particular to a method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives. Background Technology

[0002] In recent years, heavy-haul transportation has become a key focus of railway freight transport. Due to the increased traction weight of heavy-haul trains, multiple locomotives are often required to coordinate traction to ensure stable train operation. Currently, wireless data transmission is mainly used to provide data transmission channels for the main control locomotive at the front and the slave locomotives in the middle, enabling synchronized traction and braking. However, in complex and ever-changing operating environments, the wireless data transmission system used for synchronized control can experience delays or even failures due to terrain obstruction or electromagnetic interference, potentially leading to traffic accidents. Therefore, utilizing advanced machine vision and voice recognition technologies to assist the drivers of both the main and slave locomotives in understanding the current synchronized control status more easily, allowing them to focus more on controlling the locomotives, reducing operational difficulty, and improving operational safety has become an urgent requirement for heavy-haul railway transportation. Summary of the Invention

[0003] In view of the current state of technology and existing problems, the purpose of this invention is to provide a method for implementing a safety auxiliary system for synchronized control of heavy-haul locomotives. This method, based on the original integrated wireless communication system for locomotives, utilizes machine vision to judge the control actions of the master and slave locomotives, and controls the CIR device to enable the slave locomotive to automatically broadcast the control content of the master locomotive. It utilizes voice recognition to achieve voice control and communication between the master and slave locomotives according to the driver's voice commands. Even in special circumstances such as when the driver's movement is restricted, voice communication between the master and slave locomotives can still be achieved. When an abnormality in the synchronization of the master and slave locomotives is detected, a voice call between the master and slave locomotives can be automatically initiated. This assists the drivers of both locomotives in focusing on locomotive control in abnormal situations, reducing operational difficulty and thus improving the safety of heavy-haul locomotive operation. The system also has a fault mitigation function to further enhance safety.

[0004] The technical solution adopted by this invention is: a method for implementing a synchronous control safety assistance system for heavy-duty locomotives. The system includes a safety assistance system host, an intelligent camera, and a microphone array. The safety assistance system host includes a safety assistance system processing unit, a CIR access unit, a voice recognition unit, and a control program. The system is connected to an external CIR device and an MMI operation display unit via the safety assistance system host. The MMI operation display unit is connected to an external speaker. The CIR access unit is connected in series between the CIR device and the MMI operation display unit via a CIR device control cable, used to listen to the RS422 bus communication interface in the control cable to determine whether the CIR device is in a call state. The CIR access unit also connects to a safety... The auxiliary system processing unit and the voice recognition unit are connected and controlled by the safety auxiliary system processing unit to send the same locomotive group call command and voice signal to the CIR device; the microphone array is connected to the voice recognition unit and installed in the driver's cab of the heavy-haul locomotive, used to receive voice commands, send the recognition results to the safety auxiliary system processing unit, and generate the voice signal under control; the smart camera is connected to the safety auxiliary system processing unit, installed in the driver's cab of the heavy-haul locomotive, and points to the TCMS main display screen on the control panel, used to identify the content of the TCMS main display screen to determine the locomotive's operating status, and send the status information to the safety auxiliary system processing unit; the system is installed in the driver's cab of the main control locomotive and the driver's cab of the slave control locomotive respectively.

[0005] The main control program of the safety auxiliary system installed in the driver's cab of the main locomotive synchronous control safety auxiliary system for heavy-haul locomotives performs the following operations:

[0006] A1. The intelligent camera of the safety assistance system host determines whether there is a vehicle braking or emergency braking action. If so, proceed to step A2 to determine whether the CIR device voice channel is occupied and then send the control action voice to the slave vehicle. If not, proceed to step A4.

[0007] A2. If it is determined that the CIR device's voice channel is occupied, proceed to step A3 to further determine whether the CIR device is still in the voice channel occupancy after 5 seconds. Otherwise, proceed to step A5 to send the control action voice to the slave locomotive.

[0008] A3. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A5.

[0009] A4. Determine if the driver has made a voice command to call the slave locomotive. If so, proceed to step A6 to determine if the CIR device's voice channel is occupied and then initiate a voice call to the slave locomotive. Otherwise, return to step A1.

[0010] A5. Send the control command voice to the slave locomotive and then return to step A1.

[0011] A6. Determine again whether the CIR device's voice channel is occupied. If the CIR device's voice channel is occupied, proceed to step A7 to determine whether the CIR device is still in the voice channel occupancy stage after 5 seconds. Otherwise, proceed to step A8 to initiate a voice call to the slave locomotive.

[0012] A7. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A8.

[0013] A8. Initiate a voice call to the slave locomotive and then return to step A1.

[0014] The main control program of the safety auxiliary system installed in the synchronous control safety auxiliary system of heavy-duty locomotives in the driver's cab of the slave locomotive performs the following operations:

[0015] B1. The voice recognition unit of the safety auxiliary system host determines whether it has received the main control locomotive control command. If it has, it proceeds to step B2 to further determine whether the slave control locomotive control action has been recognized within 5 seconds. Otherwise, it proceeds to step B3.

[0016] B2. The safety auxiliary system processing unit continuously identifies the control actions of the slave locomotive through the smart camera. If the control action of the slave locomotive is identified within 5 seconds and is the same as the control command of the master locomotive, it is determined that the locomotive synchronization control is normal and returns to step B1; otherwise, the safety auxiliary system processing unit automatically determines that the master and slave locomotive synchronization is abnormal and enters step B4 to determine whether the voice channel of the CIR device is occupied and then initiates a voice call to the master locomotive.

[0017] B3. Determine if the driver has made a voice command to call the main control locomotive. If so, proceed to step B4 to further determine if the CIR is occupied by the voice channel. Otherwise, return to step B1.

[0018] B4. If it is determined that the CIR device's voice channel is occupied, proceed to step B5 to further determine whether the CIR device is still in the voice channel occupancy after 5 seconds; otherwise, proceed to step B6 to initiate a voice call to the main control locomotive.

[0019] B5. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step B1; otherwise, proceed to step B6.

[0020] B6. Initiate a voice call to the main control locomotive and then return to step B1.

[0021] The beneficial effects of this invention are as follows:

[0022] 1. The heavy-duty locomotive synchronous control safety auxiliary system utilizes the locomotive's existing CIR communication system, eliminating the need for extensive modifications to the CIR system. This reduces the difficulty of construction, installation, and operation, making it a low-cost safety solution.

[0023] 2. The advantage of the heavy-haul locomotive synchronous control safety assistance system is that it is independent of the original heavy-haul locomotive synchronous control system. Even if the original heavy-haul locomotive synchronous control system malfunctions, the driver can still use the heavy-haul locomotive synchronous control safety assistance system to smoothly realize the function of voice command and manual control, thereby avoiding safety accidents.

[0024] 3. As an auxiliary system to the existing heavy-duty locomotive synchronous control system, it can promptly play the control actions of the master locomotive to the slave locomotive via voice, reminding the slave locomotive driver to pay attention to and verify the control actions.

[0025] 4. The heavy-duty locomotive synchronous control safety assistance system integrates the command signals received by the voice recognition unit and the visual motion signals recognized by the intelligent camera. It automatically diagnoses the synchronization status of the master and slave locomotives in the time dimension and automatically initiates an emergency call when an abnormality occurs. This constructs a redundant safety barrier independent of the original synchronous control system, greatly improving the active safety of driving.

[0026] 5. The heavy-duty locomotive synchronous control safety auxiliary system can initiate a master-slave locomotive communication via voice control or automatically initiating a master-slave control communication in emergency situations. This avoids the driver making mistakes due to operating multiple systems simultaneously in emergency situations, allowing the driver to focus on locomotive control and reducing the occurrence of safety accidents.

[0027] 6. The heavy-haul locomotive synchronous control safety auxiliary system is connected in series with the CIR equipment control cable via an RS422 bus and a relay. The normally closed contacts of the relays are set in the MIC audio signal path between the CIR equipment and the MMI operation display unit. Therefore, even if the heavy-haul locomotive synchronous control safety auxiliary system fails or loses power, it will not affect the CIR equipment from realizing normal wireless train dispatching voice and data communication services. The system has fault mitigation function. Attached Figure Description

[0028] Figure 1 A connection principle block diagram of a heavy-duty locomotive synchronous control safety auxiliary system provided in an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram illustrating an application scenario of a heavy-duty locomotive synchronous control safety auxiliary system provided in an embodiment of the present invention;

[0030] Figure 3 A control flowchart of the main control locomotive safety auxiliary system processing unit of a heavy-haul locomotive synchronous control safety auxiliary system is provided in an embodiment of the present invention;

[0031] Figure 4 A control flowchart of the processing unit of the slave locomotive safety auxiliary system of the synchronous control safety auxiliary system for heavy-duty locomotives is provided in an embodiment of the present invention.

[0032] Figure 5 for Figure 1 Block diagram of CIR access unit in China;

[0033] Figure 6 for Figure 1 Block diagram of the processing unit of the safety auxiliary system. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings.

[0035] The heavy-haul locomotive synchronous control safety assistance system serves as a valuable supplement to the existing synchronous control system. It uses machine vision to recognize the master locomotive's control actions and provides voice prompts to the slave locomotive via CIR equipment, facilitating the slave locomotive driver's review of the locomotive's status. Simultaneously, the system analyzes and judges the synchronization status of the master and slave locomotives in real time. In emergencies, it automatically connects the master and slave locomotives via CIR voice communication, assisting both drivers in completing complex call operations. This allows both drivers to focus on locomotive control, thereby enhancing the safety of heavy-haul locomotive operation.

[0036] like Figure 1 , Figure 2 As shown, a heavy-haul locomotive synchronous control safety assistance system is used to connect with the locomotive's existing CIR equipment, MMI operation display unit, and external speakers. The heavy-haul locomotive synchronous control safety assistance system includes a safety assistance system host, a smart camera, and a microphone array. The safety assistance system host includes a safety assistance system processing unit containing control programs, a CIR access unit, and a voice recognition unit. The heavy-haul locomotive synchronous control safety assistance system connects to the external CIR equipment and MMI operation display unit via the safety assistance system host; the MMI operation display unit is connected to the external speakers.

[0037] The CIR access unit is connected in series between the CIR device and the MMI operation display unit via the CIR device control cable. It is used to listen to the RS422 bus communication interface in the control cable to determine whether the CIR device is in a call state. The CIR access unit is also connected to the safety assistance system processing unit and the voice recognition unit, and is controlled by the safety assistance system processing unit to send the same locomotive group call command and voice signal to the CIR device. The microphone array is connected to the voice recognition unit, installed in the driver's cab of the heavy-haul locomotive, and facing the driver's face. It is used to receive voice commands, send the recognition results to the safety assistance system processing unit, and generate voice signals under control. The smart camera is connected to the safety assistance system processing unit, installed in the driver's cab of the heavy-haul locomotive, and pointed at the TCMS main display screen on the control panel. It is used to identify the content of the TCMS main display screen to determine the locomotive's operating status and send the status information to the safety assistance system processing unit. The heavy-haul locomotive synchronous control safety assistance system is installed in the driver's cab of the main control locomotive and the driver's cab of the slave control locomotive.

[0038] The CIR equipment and MMI operation display unit utilize the existing equipment on the locomotive. Aside from integrating a safety auxiliary system between them, no hardware or software modifications are required. Communication between the CIR equipment and the MMI operation display unit is relayed through the CIR access unit, enabling normal wireless train dispatching services. The CIR access unit contains relays controlled by the safety auxiliary system processing unit. It can connect the CIR equipment's microphone audio interface to the voice output interface of the voice recognition unit, thereby automatically controlling the CIR equipment to transmit main locomotive control commands.

[0039] The CIR access unit is responsible for determining whether the CIR device is in a call state by listening to the RS422 bus communication interface in the control cable. Simultaneously, it sends group call commands to the CIR device through the RS422 bus interface of the control cable.

[0040] like Figure 5 As shown, the CIR access unit includes a relay; the normally closed contact of the relay is connected in series in the MIC audio signal path between the CIR device and the MMI operation display unit; the controlled end of the relay is connected to the safety auxiliary system processing unit; the safety auxiliary system processing unit switches the MIC audio signal path by controlling the opening and closing of the relay: when a voice signal needs to be sent, the safety auxiliary system processing unit controls the relay to be energized, its normally closed contact disconnects from the MMI operation display unit, and switches to connect to the voice output end of the voice recognition unit; under normal conditions or when the system loses power, the normally closed contact of the relay remains closed, ensuring the normal connection of the audio path between the CIR device and the MMI operation display unit to achieve fault mitigation function.

[0041] The CIR access unit directly connects the serial port of the safety auxiliary system processing unit to the RS422 bus of the CIR device control cable. In this way, the safety auxiliary system processing unit can listen to communication messages on the CIR device control cable and send group call commands to the CIR device. Simultaneously, the internal relays of the CIR access unit are controlled by the GPIO ports of the safety auxiliary system processing unit, allowing the MIC audio interface of the CIR device control cable to switch between the MMI operation display unit and the voice recognition unit. This enables the safety auxiliary system processing unit to control the voice recognition unit to send locomotive control commands to the CIR device through the CIR access unit.

[0042] like Figure 6 As shown, the safety assistance system processing unit includes a microprocessor (MCU), which is connected to a power supply circuit, a clock reset circuit, and an external memory. The MCU's multiple UART interfaces are connected to a voice recognition unit, a smart camera, and a CIR access unit via level conversion circuits. The MCU's GPIO interface is directly connected to the CIR access unit to control the opening and closing of internal relays.

[0043] The UART0, UART1, and UART2 interfaces of the safety auxiliary system processing unit microprocessor MCU are connected to the voice recognition unit, smart camera, and CIR access unit through the first level conversion circuit, the second level conversion circuit, and the third level conversion circuit, respectively.

[0044] The safety assistance system processing unit is responsible for obtaining the current locomotive control status from the smart camera and the driver's voice commands from the voice recognition unit. Simultaneously, it obtains the CIR device's busy / idle status from the CIR access unit to determine whether the CIR device's voice channel can be used to communicate with other locomotives in the formation.

[0045] Based on the information sent by each unit, the safety auxiliary system processing unit can generate a call command for the same locomotive group according to the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" (10). This command is then sent to the CIR device via the CIR access unit. Simultaneously, the voice recognition unit controls the voice recognition unit to send audio signals indicating locomotive braking or emergency braking to the CIR device via the CIR access unit. After the audio signal is sent, the safety auxiliary system processing unit sends an hang-up message to the CIR device and controls the relay in the CIR access unit to promptly restore the communication connection between the CIR device and the MMI operation display unit, thus realizing automatic voice call and hang-up functions.

[0046] The safety assistance system processing unit can employ a high-performance ARM core microprocessor, such as NXP's MIMXRT1062CVL5B. This chip, based on the Arm Cortex-M7 core, features real-time performance and high integration, operating at frequencies up to 528MHz and has 1MB of on-chip RAM. It offers a wide range of connectivity interfaces, including UART, SPI, I²C, USB, 10 / 100M Ethernet, and CAN. Characterized by ease of use, low cost, and strong compatibility, it can fulfill all the business requirements of the safety assistance system processing unit.

[0047] The voice recognition unit uses an offline voice module with a microphone array; the voice recognition unit is pre-set with wake-up words for calling the master vehicle, calling the slave vehicle, locomotive braking, and emergency braking, which are used to receive and recognize the corresponding voice commands; the voice recognition unit pre-stores the voice broadcast content of locomotive braking and emergency braking, and is controlled by the safety assistance system processing unit to generate the corresponding audio voice signals.

[0048] The voice recognition unit can utilize, for example, the HLK-V25 dual-microphone offline voice module from Hailink Technology. The HLK-V25 employs acoustic front-end solutions such as voice enhancement, sound source localization, and echo cancellation to achieve functions like voice wake-up and offline recognition. The front-end uses the latest generation of microphone array algorithms, integrating traditional signal processing and neural network processing capabilities to significantly improve the front-end acoustic processing effect. It can accurately recognize voice commands even in noisy in-vehicle environments. Before using the HLK-V25 dual-microphone offline voice module, wake-up words for calling the master vehicle, calling the slave vehicle, locomotive braking, and emergency braking are created using Hailink Technology's backend software. Upon receiving the corresponding voice command, the module can immediately notify the safety assistance system processing unit via serial port to initiate master and slave vehicle calls. Simultaneously, utilizing the HLK-V25 dual-microphone offline voice module's voice broadcast function, voice broadcast content for locomotive braking and emergency braking is created using Hailink Technology's backend software before use. Connect the voice output interface of the voice recognition unit to the relay of the CIR access unit. In this way, the voice recognition unit can generate locomotive braking or emergency braking voice signals under the control of the safety assistance system processing unit and send them to the CIR device.

[0049] The intelligent camera is installed on the control panel in the driver's cab of the heavy-duty locomotive, pointing towards the TCMS main display screen on the control panel. It is responsible for recognizing the content displayed on the TCMS main display screen and judging the control status of the locomotive, such as braking and emergency braking.

[0050] The intelligent camera is an intelligent camera that integrates offline AI deep learning algorithms; the intelligent camera is trained using images from the locomotive's TCMS main display screen under traction, braking, and emergency braking conditions, and is configured to recognize changes in the control status of the traction or braking main screen of the TCMS main display screen; the intelligent camera sends the recognized locomotive operation status information to the safety assistance system processing unit through a serial interface.

[0051] The smart camera can be, for example, the Hikvision SC6016M smart camera. The SC6016M smart camera embeds AI deep learning algorithms, enabling functions such as character recognition, object recognition, and defect detection. It also integrates the VM algorithm platform software, allowing free use of over 140 algorithm functions. Before use, the smart camera is trained using Hikvision's VM algorithm platform software with images from the TCMS main display during traction, braking, and emergency braking. This allows the SC6016M smart camera to easily detect the control status of the traction or braking main screen on the TCMS main display. During installation, the smart camera's pointing and focus are adjusted to ensure the TCMS main display is completely and clearly centered in the camera's field of view. The smart camera constantly monitors the content displayed on the TCMS main display. When a change is detected in the traction or braking main screen of the TCMS main display, the smart camera immediately sends the change information to the safety assistance system processing unit via a serial interface.

[0052] The microphone array is mounted on the control panel in the driver's cab of the heavy-haul locomotive, facing the driver's face, and is responsible for receiving audio commands from the driver's cab. The microphone array consists of two high-sensitivity electret microphones, with a horizontal distance of 35mm between them to facilitate acoustic processing by the voice recognition unit and improve recognition accuracy.

[0053] As one embodiment of the present invention, a heavy-duty locomotive synchronous control safety auxiliary system is installed in the driver's cab of the main control locomotive and the driver's cab of the slave control locomotive (one main control locomotive device may be equipped with one or more slave control locomotive devices depending on the train formation).

[0054] like Figure 3 As shown, the main control program of the safety auxiliary system installed in the driver's cab of the heavy-haul locomotive performs the following operations:

[0055] A1. The intelligent camera of the safety assistance system host determines whether there is a vehicle braking or emergency braking action. If so, proceed to step A2 to determine whether the CIR device voice channel is occupied and then send the control action voice to the slave vehicle. If not, proceed to step A4.

[0056] A2. If it is determined that the CIR device's voice channel is occupied, proceed to step A3 to further determine whether the CIR device is still in the voice channel occupancy after 5 seconds. Otherwise, proceed to step A5 to send the control action voice to the slave locomotive.

[0057] A3. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A5.

[0058] A4. Determine whether the driver has made a voice command to call the slave locomotive. If so, proceed to step A6 to determine whether the CIR device's voice channel is occupied and then initiate a voice call to the slave locomotive. Otherwise, return to step A1.

[0059] A5. Send the control command voice to the slave locomotive and then return to step A1.

[0060] A6. Determine again whether the CIR device's voice channel is occupied. If the CIR device's voice channel is occupied, proceed to step A7 to determine whether the CIR device is still in the voice channel occupancy stage after 5 seconds. Otherwise, proceed to step A8 to initiate a voice call to the slave locomotive.

[0061] A7. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A8.

[0062] A8. Initiate a voice call to the slave locomotive and then return to step A1.

[0063] The safety auxiliary system processing unit, installed on the main control locomotive, receives information from intelligent cameras in real time to determine whether there is locomotive braking or emergency braking information, and receives information from the voice recognition unit to determine whether there is a driver's voice command calling the slave locomotive. Two independent sets of judgment program codes—one for determining whether there is locomotive braking or emergency braking information and the other for determining whether there is a driver's voice command calling the slave locomotive—run continuously in a loop within the safety auxiliary system's main control program, ensuring that braking or emergency braking information and driver voice commands can be responded to in real time.

[0064] When the main control locomotive performs braking or emergency braking maneuvers, the intelligent camera installed on the main control locomotive captures the changes in the information displayed on the TCMS main display screen in real time. After identifying the maneuver as braking or emergency braking, it immediately sends the information to the safety auxiliary system processing unit. The safety auxiliary system processing unit listens to the communication content of the RS422 bus between the CIR device and the MMI operation display unit through the CIR access unit. Based on the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" 10, it determines whether the CIR device's voice channel is occupied. If it is occupied, meaning the CIR device is in a call state, it waits 5 seconds and then checks again to see if the CIR device is in a call state. The 5-second duration is comprehensively set based on the call characteristics of the wireless train dispatching system and the real-time response requirements of synchronous control to ensure the necessity of the operation. If the CIR device is still in a call state, the operation is abandoned. Otherwise, the safety auxiliary system processing unit, according to the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" 10, generates a group call message for the controlled locomotives in the same group and sends it to the CIR access unit, which then forwards it to the CIR device. After the safety auxiliary system processing unit detects a call connection by listening to the RS422 bus communication interface through the CIR access unit, it controls the voice recognition unit to synthesize an audio signal for "locomotive braking" or "emergency braking" and applies it to the MIC audio line of the control cable through the CIR access unit, enabling the CIR device to send audio information to the locomotives in the same group via the CIR network. After the audio information is sent, the safety auxiliary system processing unit sends an hang-up message to the CIR device through the CIR access unit. It also controls the CIR access unit relay to disconnect the MIC audio line of the control cable from the voice output terminal of the voice recognition unit and reconnect it to the MMI operation display unit, thus enabling the transmission of master locomotive control content to the controlled locomotives.

[0065] When the driver of the main locomotive issues the voice command "Call slave locomotive" from the driver's cab, the microphone array installed on the main locomotive receives the voice command, converts the audio signal into an electrical signal, and transmits it to the voice recognition unit. The voice recognition unit, after recognizing the audio as the "Call slave locomotive" command, immediately sends the information to the safety auxiliary system processing unit. The safety auxiliary system processing unit checks whether the CIR device's voice channel is occupied through the CIR access unit. If it is occupied, meaning the CIR device is in a call state, it waits 5 seconds and checks again to see if the CIR device is in a call state. The 5-second duration is based on the characteristics of the wireless train dispatch system's call and the real-time response requirements of synchronous control, ensuring the necessity of the operation. If the CIR device is still in a call state, the operation is abandoned. Otherwise, the safety auxiliary system processing unit generates a communication message for calling the slave locomotive in the same group according to the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" (10). This message is sent to the CIR access unit, which then forwards it to the CIR device, realizing the voice control initiation of the call to the slave locomotive.

[0066] like Figure 4 As shown, the main control program of the safety auxiliary system installed in the driver's cab of the heavy-haul locomotive performs the following operations:

[0067] B1. The voice recognition unit of the safety auxiliary system host determines whether it has received the main control locomotive control command. If it has, it proceeds to step B2 to further determine whether the slave control locomotive control action has been recognized within 5 seconds. Otherwise, it proceeds to step B3.

[0068] B2. The safety auxiliary system processing unit continuously identifies the control actions of the slave locomotive through the smart camera. If the control action of the slave locomotive is identified within 5 seconds and is the same as the control command of the master locomotive, it is determined that the locomotive synchronization control is normal and returns to step B1; otherwise, the safety auxiliary system processing unit automatically determines that the master and slave locomotive synchronization is abnormal and enters step B4 to determine whether the voice channel of the CIR device is occupied and then initiates a voice call to the master locomotive.

[0069] B3. Determine if the driver has made a voice command to call the main control locomotive. If so, proceed to step B4 to further determine if the CIR is occupied by the voice channel. Otherwise, return to step B1.

[0070] B4. If it is determined that the CIR device's voice channel is occupied, proceed to step B5 to further determine whether the CIR device is still in the voice channel occupancy after 5 seconds; otherwise, proceed to step B6 to initiate a voice call to the main control locomotive.

[0071] B5. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step B1; otherwise, proceed to step B6.

[0072] B6. Initiate a voice call to the main control locomotive and then return to step B1.

[0073] The safety auxiliary system processing unit installed on the slave locomotive receives information from the intelligent camera in real time to determine whether there is locomotive braking or emergency braking information. It also receives information from the voice recognition unit to determine whether there are driver voice commands calling the master locomotive and audio messages from the MMI external speaker indicating locomotive braking, emergency braking, or other master locomotive control actions. Two independent sets of judgment program codes—one for master locomotive control commands and the other for driver voice commands calling the master locomotive—run continuously in a loop within the safety auxiliary system's host control program, ensuring that master locomotive control commands and driver voice commands can be responded to in real time.

[0074] When the master locomotive performs braking or emergency braking maneuvers, the heavy-haul locomotive synchronous control safety assistance system controls the master locomotive's CIR equipment to call the slave locomotive and send voice control information. Upon receiving the call, the slave locomotive's MMI external speaker broadcasts the voice prompts "Locomotive Braking" or "Emergency Braking." The microphone array installed on the slave locomotive receives the voice information, converts it into an electrical signal, and transmits it to the voice recognition unit. Once the voice recognition unit identifies the voice as a master locomotive control action, it immediately sends the information to the safety assistance system processing unit. The safety assistance system processing unit starts a 5-second timer. This 5-second timer is based on the typical response delay time of the locomotive synchronous control system and the driver's reaction time to ensure accurate judgment. If no identical control action information is received from the slave locomotive via the intelligent camera within 5 seconds, a malfunction in the heavy-haul locomotive synchronous control is considered, and the heavy-haul locomotive synchronous control safety assistance system will automatically initiate a voice call to the master locomotive. The safety assistance system processing unit then uses the CIR access unit to determine if the CIR equipment's voice channel is occupied. If the CIR device is occupied (i.e., in a call state), wait 5 seconds to reassess its call status. This 5-second interval is based on the call characteristics of the wireless train dispatch system and the real-time response requirements of synchronous control, ensuring the necessity of the operation. If the CIR device is still in a call state, the operation is abandoned. Otherwise, the safety auxiliary system processing unit generates a communication message for calling the main locomotive in the same group according to the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" (10). This message is sent to the CIR access unit, which then forwards it to the CIR device, enabling automatic initiation of a call to the main locomotive in an emergency. This allows the driver to focus on locomotive control in abnormal situations.

[0075] When the driver of the slave locomotive issues the voice command "Call Master Locomotive" from the driver's cab, the microphone array installed on the slave locomotive receives the voice command, converts the audio signal into an electrical signal, and transmits it to the voice recognition unit. The voice recognition unit, recognizing the audio as the "Call Master Locomotive" command, immediately sends the information to the safety auxiliary system processing unit. The safety auxiliary system processing unit checks whether the CIR device's voice channel is occupied via the CIR access unit. If occupied, meaning the CIR device is in a call state, it waits 5 seconds and checks again. The 5-second duration is based on the characteristics of the wireless train dispatching system's call function and the real-time response requirements of synchronous control, ensuring the necessity of the operation. If the CIR device is still in a call state, the operation is abandoned. Otherwise, the safety auxiliary system processing unit generates a communication message for calling the master locomotive in the same group according to the data communication protocol in Q / CR651.1-2018 "Locomotive Integrated Wireless Communication Equipment Part 1: Technical Conditions" 10, and sends it to the CIR access unit, which then forwards it to the CIR device. This realizes the function of initiating a call to the master locomotive via voice control.

[0076] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Contents not described in detail in the embodiments of the present invention are prior art known to those skilled in the art and will not be elaborated upon here.

Claims

1. A method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives, characterized in that, The system includes a safety assistance system host, a smart camera, and a microphone array; the safety assistance system host includes a safety assistance system processing unit containing control programs, a CIR access unit, and a voice recognition unit. The system is connected to an external CIR device and an MMI operation display unit via a safety auxiliary system host; the MMI operation display unit is connected to an external speaker. The CIR access unit is connected in series between the CIR device and the MMI operation display unit via the CIR device control cable. It is used to listen to the RS422 bus communication interface in the control cable to determine whether the CIR device is in a call state. The CIR access unit is also connected to the safety auxiliary system processing unit and the voice recognition unit, and is controlled by the safety auxiliary system processing unit to send the same locomotive group call command and voice signal to the CIR equipment; The microphone array is connected to the voice recognition unit, installed in the driver's cab of the heavy-duty locomotive and facing the driver's face, for receiving voice commands, sending the recognition results to the safety assistance system processing unit, and generating voice signals in a controlled manner; The intelligent camera is connected to the safety assistance system processing unit, installed in the driver's cab of the heavy-duty locomotive, and points to the TCMS main display screen on the control panel. It is used to identify the content of the TCMS main display screen to determine the locomotive's operating status and send the status information to the safety assistance system processing unit. The systems are installed in the driver's cab of the main control locomotive and the driver's cab of the slave control locomotive, respectively. The main control program of the safety auxiliary system installed in the driver's cab of the main locomotive synchronous control safety auxiliary system for heavy-haul locomotives performs the following operations: A1. The intelligent camera of the safety assistance system host determines whether there is a vehicle braking or emergency braking action. If so, proceed to step A2 to determine whether the CIR device voice channel is occupied and then send the control action voice to the slave vehicle. If not, proceed to step A4. A2. If it is determined that the CIR device's voice channel is occupied, proceed to step A3 to further determine whether the CIR device is still in the voice channel occupancy after 5 seconds; otherwise, proceed to step A5 to send the control action voice to the slave locomotive. A3. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A5. A4. Determine whether the driver has made a voice command to call the slave locomotive. If so, proceed to step A6 to determine whether the CIR device's voice channel is occupied and then initiate a voice call to the slave locomotive. Otherwise, return to step A1. A5. Send the control command voice to the slave locomotive and then return to step A1; A6. Check again whether the CIR device voice channel is occupied. If the CIR device voice channel is occupied, proceed to step A7 and check again whether the CIR device is still occupied after 5 seconds. Otherwise, proceed to step A8 to initiate a voice call to the slave locomotive. A7. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step A1; otherwise, proceed to step A8. A8. Initiate a voice call to the slave locomotive and then return to step A1.

2. The method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives according to claim 1, characterized in that, The CIR access unit includes a relay inside; The normally closed contact of the relay is connected in series in the MIC audio signal path between the CIR device and the MMI operation display unit; The controlled terminal of the relay is connected to the safety auxiliary system processing unit; The safety auxiliary system processing unit switches the path of the MIC audio signal by controlling the opening and closing of the relay: when a voice signal needs to be sent, the safety auxiliary system processing unit controls the relay to be energized, its normally closed contact disconnects from the MMI operation display unit, and switches to connect to the voice output terminal of the voice recognition unit; under normal conditions or when the system loses power, the normally closed contact of the relay remains closed, ensuring the normal connection of the audio path between the CIR device and the MMI operation display unit to achieve fault mitigation function.

3. The method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives according to claim 1, characterized in that, The safety auxiliary system processing unit includes a microprocessor (MCU), which is connected to a power supply circuit, a clock reset circuit, and an external memory. The multiple UART interfaces of the MCU are respectively connected to the voice recognition unit, the smart camera, and the CIR access unit through a level conversion circuit; The GPIO interface of the MCU is directly connected to the CIR access unit and is used to control the opening and closing of the internal relay.

4. The method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives according to claim 1, characterized in that, The main control program of the safety auxiliary system installed in the synchronous control safety auxiliary system of heavy-duty locomotives in the driver's cab of the slave locomotive performs the following operations: B1. The voice recognition unit of the safety auxiliary system host determines whether it has received the main control locomotive control command. If it has, it proceeds to step B2 to further determine whether the slave control locomotive control action has been recognized within 5 seconds. Otherwise, it proceeds to step B3. B2. The safety auxiliary system processing unit continuously identifies the control actions of the slave locomotive through the smart camera. If the control action of the slave locomotive is identified within 5 seconds and is the same as the control command of the master locomotive, it is determined that the locomotive synchronization control is normal and returns to step B1; otherwise, the safety auxiliary system processing unit automatically determines that the master and slave locomotive synchronization is abnormal and enters step B4 to determine whether the voice channel of the CIR device is occupied and then initiates a voice call to the master locomotive. B3. Determine whether the driver has made a voice command to call the main control locomotive. If so, proceed to step B4 to further determine whether the CIR is occupied by the voice channel. Otherwise, return to step B1. B4. If it is determined that the CIR device's voice channel is occupied, proceed to step B5 to further determine whether the CIR device is still occupying the voice channel after 5 seconds; otherwise, proceed to step B6 to initiate a voice call to the main control locomotive. B5. If it is determined that the CIR device is still in the voice channel after 5 seconds, return to step B1; otherwise, proceed to step B6. B6. Initiate a voice call to the main control locomotive and then return to step B1.

5. The method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives according to claim 1, characterized in that, The speech recognition unit uses an offline speech module with a microphone array. The voice recognition unit is pre-set with wake-up words for calling the master vehicle, calling the slave vehicle, locomotive braking, and emergency braking, which are used to receive and recognize the corresponding voice commands; The voice recognition unit pre-stores the voice broadcast content of vehicle braking and emergency braking, and is controlled by the safety assistance system processing unit to generate corresponding audio voice signals.

6. The method for implementing a synchronous control safety auxiliary system for heavy-haul locomotives according to claim 1, characterized in that, The smart camera is a smart camera that integrates offline AI deep learning algorithms; The intelligent camera is trained using images from the locomotive's TCMS main display screen during traction, braking, and emergency braking, and is configured to recognize changes in the control status of the TCMS main display screen during traction or braking. The smart camera sends the identified locomotive operation status information to the safety assistance system processing unit via a serial interface.

Images