Train driver anti-fatigue device and method based on multi-mode perception and stimulation

Abstract

The invention discloses a train driver anti-fatigue device and method based on multi-mode perception and stimulation, and relates to the technical field of anti-fatigue devices.The train driver anti-fatigue device comprises a vehicle-mounted end machine, and the vehicle-mounted end machine is arranged on a driving platform and electrically isolated from a train TCMS bus; the vehicle-mounted terminal machine comprises a multi-mode sensor array which is used for synchronously collecting a face visible light image, an infrared thermal image, a millimeter wave radar micro-motion signal, a voice acoustic signal and a steering wheel holding pressure distribution signal of a driver; and the edge calculation unit is internally provided with a neural network SoC. According to the train driver anti-fatigue device based on multi-modal sensing and stimulation, through the full-closed-loop design of multi-modal sensing, edge real-time analysis, multi-channel collaborative intervention and cloud dynamic optimization, a multi-modal sensor array synchronously collects driver state signals from multiple dimensions such as vision, physiology and behaviors; in combination with real-time fusion analysis of an edge calculation unit, fatigue features can be captured more accurately, and misjudgment of a single signal is avoided.

Description

Technical Field

[0001] This invention relates to the field of anti-fatigue devices, and more specifically, to a train driver anti-fatigue device and method based on multimodal perception and stimulation. Background Technology

[0002] Drivers are prone to fatigue when driving for extended periods. When fatigued, drivers experience decreased judgment, slower reaction times, and increased operational errors. Mild fatigue can lead to delayed or inaccurate gear shifting; moderate fatigue can cause sluggish movements and even forgetting to perform actions; severe fatigue can result in subconscious actions or short periods of sleep, and in severe cases, loss of control over the vehicle.

[0003] To address driver fatigue, some fatigue-related devices for train drivers have emerged in existing technologies, but they still have many shortcomings. In terms of fatigue detection, most devices rely on a single sensor to collect signals, capturing only a single dimension of the driver's state characteristics. This makes them susceptible to environmental interference, resulting in low accuracy and a high false positive rate, making it difficult to comprehensively and accurately reflect the driver's true fatigue level. Regarding intervention methods, traditional devices often employ a single stimulus, resulting in limited intervention effects and a lack of personalized adaptation capabilities. They cannot adjust intervention strategies according to the different physiological characteristics and tolerance levels of different drivers, and long-term use can easily lead to stimulus tolerance in drivers, further weakening the intervention effect.

[0004] Meanwhile, the core control unit (vehicle-mounted terminal) of the fatigue-resistant device needs to operate continuously for a long time, carrying out tasks such as signal acquisition, data processing, fatigue identification, and intervention control, which will generate a lot of heat during operation. Summary of the Invention

[0005] The present invention provides a train driver fatigue-resistant device and method based on multimodal perception and stimulation, which aims to solve the following problems: In terms of fatigue recognition, most devices rely on a single sensor to collect signals, which can only capture the driver's state characteristics in a single dimension. This is easily affected by environmental interference, resulting in low recognition accuracy and a high misjudgment rate, making it difficult to comprehensively and accurately reflect the driver's true fatigue level. In terms of intervention methods, traditional devices mostly use a single stimulus form, which has limited intervention effect and lacks personalized adaptation capabilities. It is impossible to adjust the intervention strategy according to the different physiological characteristics and tolerance of different drivers. Long-term use can also easily lead to stimulation tolerance in drivers, further weakening the intervention effect.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a train driver fatigue-resistant device based on multimodal perception and stimulation, including an on-board terminal unit, which is arranged on the driver's cab and electrically isolated from the train TCMS bus; The vehicle-mounted terminal is fitted with a ventilated outer shell. An air inlet chamber and an air outlet chamber are formed between the inner wall of the ventilated outer shell and the outer wall of the vehicle-mounted terminal. The air inlet chamber and the air outlet chamber are connected through a ventilation gap. A side groove communicating with the air inlet chamber is opened on the side of the ventilated outer shell. A cylinder is rotatably installed inside the side groove. An inner side groove and an outer side groove are opened on both sides of the cylinder. A filter screen is slidably installed inside the outer side groove. A fixing plate is fixed on the inner wall of the ventilated outer shell. The fixing plate is attached to the outer wall of the cylinder. A collection chamber is formed between the fixing plate and the inner wall of the ventilated outer shell. The vehicle-mounted terminal includes: A multimodal sensor array is used to simultaneously acquire visible light images of the driver's face, infrared thermal imaging, millimeter-wave radar micro-motion signals, voice acoustic signals, and steering wheel grip pressure distribution signals. The edge computing unit, with a built-in neural network SoC, is used to fuse signals in real time and output fatigue confidence scores. The multi-channel stimulation execution unit includes: a programmable cold light LED array, a directional ultrasonic beam emitter, a low-frequency vibration pad, and an aroma microcapsule electronically controlled release module; The cloud-based fatigue big data platform communicates bidirectionally with the vehicle-mounted terminal via a 5G-R private network, enabling online updates of the fatigue recognition model and the distribution of personalized stimulation strategies. The privacy protection module uses the national cryptographic algorithm SM4 to homomorphically encrypt uploaded data; When the fatigue confidence level exceeds the first-level threshold, the edge computing unit immediately triggers at least two heterogeneous stimulation channels in the multi-channel stimulation execution unit to perform coordinated intervention, and reassesses the fatigue confidence level within 5 seconds after the intervention. If the fatigue confidence level does not decrease, the stimulation intensity is automatically upgraded and a help message is sent to the cloud-based fatigue big data platform at the same time.

[0007] Preferably, the thickness of the filter screen is less than the thickness of the outer groove, forming an outer edge area between the filter screen and the inner wall of the outer groove. A fixed block is fixedly connected to the inner wall of the ventilation shell. The fixed block has a groove at one end near the collection chamber. A fixed shaft is rotatably mounted on the fixed block, and the fixed shaft is distributed correspondingly to the groove. A moving strip is rotatably mounted on the fixed shaft. A torsion spring is fitted on the fixed shaft. One end of the torsion spring is fixedly connected to the fixed shaft, and the other end of the torsion spring is fixedly connected to the moving strip. The moving strip has a first arc portion and a first right-angle portion at one end near the fixed block, wherein the first right-angle portion is close to the filter screen. A middle rod is fixedly connected to the filter screen. An arc strip is fixedly connected to the end of the middle rod away from the filter screen. The arc strip abuts against the moving strip. Both ends of the arc strip have oblique cuts. A first motor is fixedly installed on the outer wall of the ventilation shell, and the cylinder is fixedly connected to the drive shaft of the first motor.

[0008] Preferably, a vertical plate is fixedly connected to the inner wall of the cylinder, and an elastic telescopic rod is fixedly connected to the vertical plate. The end of the elastic telescopic rod away from the vertical plate is fixedly connected to the filter screen.

[0009] Preferably, a frame is provided at the air outlet chamber, the frame is integrally formed with the ventilation shell, a second motor is installed inside the frame, a fan is fixedly installed on the drive shaft of the second motor, a plate for closing the frame is provided on the side of the fan facing away from the vehicle terminal, a fixed rod is rotatably provided on the top of the plate, the fixed rod is fixedly connected to the inner wall of the frame, and a second right angle portion and a second arc portion are provided at the top of the plate, wherein the second right angle portion is close to the vehicle terminal.

[0010] Preferably, a heat-absorbing strip is provided inside the ventilation gap. The heat-absorbing strip is fixedly connected to the inner wall of the ventilation shell. Multiple heat-absorbing strips are provided and are evenly distributed.

[0011] Preferably, the multi-channel stimulation execution unit synchronizes the output phase, duration interval and repetition count of the programmable cold light LED array, directional ultrasonic beam emitter, low-frequency vibration pad and aromatic microcapsule electronically controlled release module at the microsecond level through the "heterogeneous stimulation timing arrangement register", so that any two heterogeneous stimulation channels can be programmed to offset within the phase difference range of 0–180°. The low-frequency vibration pad utilizes a dual modulation circuit of "low-frequency-ultrasonic carrier" to superimpose an ultrasonic carrier onto a 0.3–8Hz baseband signal. The baseband controls the vibration amplitude, and the carrier phase is locked to the driving phase of the directional ultrasonic beam transmitter through the same clock tree.

[0012] Preferably, the aromatic microcapsule electronically controlled release module includes a first microcapsule array for releasing refreshing odors, and an odor rapid neutralization subunit is built into the slot on the wall of the original airflow channel, downstream of the air outlet side of the first microcapsule array. The odor rapid neutralization subunit includes a second set of microcapsule patches, an independent micro-heating electrode, a one-way airflow valve, and a closed-loop concentration locking circuit. The core of the second group of microcapsule patches is a compound neutralizing agent of β-cyclic citral and tea polyphenols; the independent micro-heating electrode shares the same ITO glass substrate and FPC cable as the original array; the one-way airflow valve is used to guide the neutralizing agent to diffuse unidirectionally with the airflow; the closed-loop concentration locking circuit uses a vehicle-mounted micro gas chromatograph and PID controller to sample and feedback the concentration of odor molecules in real time, and control the amount of neutralized release.

[0013] Preferably, the privacy protection module is equipped with a two-level encryption link of "differential-homomorphic". The first-level encryption unit uses a pixel-level differential circuit to perform differential analysis on adjacent frame images and outputs a differential bitstream. The second-level encryption unit uses the national cryptographic SM4 homomorphic encryption core to encrypt the differential bitstream. The encrypted data packets are transmitted through the 5G-R private network and, with the help of the loop buffer built into the vehicle terminal, the data is stored locally in the event of a signal interruption. After the network is restored, the missing frames are resent through the breakpoint resume transmission protocol.

[0014] Preferably, the edge computing unit uses a "help message hierarchical packaging mechanism" to handle high-risk fatigue events: when the fatigue confidence level is continuously higher than the secondary threshold, while sending a standard help message to the cloud fatigue big data platform, a lightweight vehicle-to-ground linkage broadcast frame is generated through the 5G-R protocol stack. The vehicle-to-ground linkage broadcast frame uses ASN.1 compression encoding and utilizes 5G-RSystemInformation messages to directly broadcast to the nearest ground base station when the RRC connection is not established, enabling the dispatcher to obtain the driver ID, train position, speed, and fatigue level.

[0015] This invention also discloses a train driver fatigue prevention method based on multimodal perception and stimulation, applied to the aforementioned train driver fatigue prevention device based on multimodal perception and stimulation, and further includes the following operating steps: S1. Multimodal acquisition: The driver's face visible light image, infrared thermal image, millimeter-wave radar micro-motion signal, voice acoustic signal and steering wheel grip pressure distribution signal are continuously acquired through a multimodal sensor array at a synchronous frame rate of 30Hz, and each frame of signal is stamped with a UTC timestamp. S2, Edge Fusion: Utilizing the neural network SoC built into the edge computing unit, the S1 stamped signal is aligned frame by frame and cascaded feature extraction is performed. The fatigue confidence in the 0–1 interval is output through fully connected regression, with a refresh rate of not less than 10Hz. S3, Threshold Judgment: Compare the latest fatigue confidence level with the preset first-level threshold; if the confidence level is ≥ the first-level threshold, proceed to S4; if the confidence level is < the first-level threshold, return to S1. S4. Cooperative Intervention: The edge computing unit immediately activates at least two heterogeneous stimulus channels through the "heterogeneous stimulus timing arrangement register" and emits cooperative stimulation with a programmable phase difference of 0–180°. The intervention lasts for 2–4 seconds. S5. Short-term review: After the intervention, wait 1 second for a buffer period and recalculate the fatigue confidence level; if the confidence level decreases, return to S1; if the confidence level does not decrease, increase the stimulus intensity by one level and proceed to S6. S6, Vehicle-to-Ground Reporting: The edge computing unit sends a standard help message to the cloud-based fatigue big data platform through the 5G-R private network, and at the same time generates a 256-byte vehicle-to-ground linkage broadcast frame, which is broadcast to the nearest ground base station within 1 second via the SystemInformation message, completing one closed-loop anti-fatigue cycle.

[0016] The beneficial effects of this invention are as follows: 1. This invention employs a fully closed-loop design of "multimodal perception - real-time edge analysis - multi-channel collaborative intervention - dynamic optimization in the cloud". The multimodal sensor array synchronously collects driver status signals from multiple dimensions such as vision, physiology, and behavior. Combined with the real-time fusion analysis of the edge computing unit, it can more accurately capture fatigue characteristics and avoid misjudgment of single signals. The multi-channel stimulation execution unit achieves the synergistic effect of heterogeneous stimuli through microsecond-level synchronous control, and can dynamically adjust parameters according to personalized strategies issued by the cloud, ensuring the effectiveness of intervention while adapting to the tolerance and habits of different drivers.

[0017] 2. This invention combines differential-homogeneous two-level encryption with a train-to-ground linkage broadcast mechanism to ensure driver privacy and security while enabling rapid response to high-risk fatigue events. The privacy protection module uses differential operations and national cryptographic algorithms for dual protection, reducing the risk of raw data leakage and supporting local storage and breakpoint resumption to ensure data integrity. The train-to-ground linkage broadcast mechanism can quickly synchronize critical information to the dispatching terminal with lightweight broadcast frames when the driver is severely fatigued. Compared to traditional reporting methods that rely on a complete network connection, this significantly shortens response time, provides a window for emergency dispatch, and improves the level of train operation safety.

[0018] 3. This invention filters impurities in the gas entering the ventilation housing by setting up structures such as cylinders and filters, and can collect the impurities to achieve efficient heat dissipation and ensure the efficiency of the vehicle-mounted terminal unit.

[0019] 4. This invention uses a cylindrical structure and an inner groove, and controls the angle of the inner groove to change, so as to avoid local heat dissipation dead zones caused by the airflow always flowing along a fixed path. This allows the cold air to pass through the ventilation gap more evenly and make full contact with multiple heat absorption strips, quickly removing the heat from the surface of the vehicle terminal, significantly improving the uniformity and efficiency of heat dissipation, and preventing local overheating of the equipment.

[0020] 5. By setting up structures such as a composite plate and a fixed rod, the thrust generated by the airflow can push the composite plate to rotate and open around the fixed rod, ensuring that the airflow is discharged smoothly. When the fan stops working, the composite plate automatically closes under its own gravity, preventing external dust from entering the equipment and achieving dust protection. Attached Figure Description

[0021] Figure 1This is a schematic diagram of the overall structure of the train driver anti-fatigue device based on multimodal perception and stimulation according to the present invention. Figure 2 This is a schematic diagram of the vehicle-mounted terminal structure of the present invention; Figure 3 This is a schematic diagram of the vehicle-mounted terminal and ventilation housing structure of the present invention; Figure 4 This is a schematic diagram of the ventilation housing and heat-absorbing strip structure of the present invention; Figure 5 This is a schematic diagram of the ventilation housing and ventilation gap structure of the present invention; Figure 6 For the present invention Figure 5 Enlarged schematic diagram of the structure at point A in the middle; Figure 7 For the present invention Figure 5 Enlarged schematic diagram of the structure at point B; Figure 8 This is a schematic diagram of the ventilation housing structure of the present invention; Figure 9 For the present invention Figure 8 Enlarged structural diagram of the structure at point C; Figure 10 This is a schematic diagram of the upright plate and fixed block structure of the present invention; Figure 11 This is a schematic diagram of the arc strip and intermediate rod structure of the present invention.

[0022] The attached figures are labeled as follows: 1. Vehicle-mounted terminal unit; 2. Ventilation housing; 201. Ventilation gap; 202. Air inlet chamber; 203. Air outlet chamber; 204. Side groove; 3. Heat absorption strip; 4. Cylinder; 5. Inner side groove; 6. Outer side groove; 601. Outer side area; 7. Filter screen; 8. Vertical plate; 9. Elastic telescopic rod; 10. Fixed plate; 11. Collection bin; 12. Arc strip; 13. Intermediate rod; 14. Fixed block; 15. Fixed shaft; 16. Torsion spring; 17. Moving strip; 1701. First arc portion; 1702. First right angle portion; 18. Slanted cut; 19. First motor; 20. Fan; 21. Second motor; 22. Composite plate; 23. Fixed rod; 24. Second right angle portion; 25. Second arc portion; 26. Frame. Detailed Implementation

[0023] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0024] This invention provides, for example Figures 1 to 11The train driver fatigue prevention device based on multimodal perception and stimulation shown includes an on-board terminal 1. The on-board terminal 1 adopts a compact design and its size is adapted to the reserved installation position of the mainstream train cab. It is fixed to the central area of ​​the cab by an insulating bracket to ensure electrical isolation from the train TCMS bus and avoid electromagnetic interference affecting the stability of the train control system.

[0025] Specifically, it includes: (a) Multimodal sensor array The multimodal sensor array is integrated into the front panel of the vehicle terminal unit 1, and the layout of each sensor has been optimized: The visible light camera and the infrared thermal imager are installed side by side, with the lens facing the driver's face area. A wide-angle lens is used to ensure coverage of the face in different sitting positions. The resolution of the visible light camera is no less than 1920×1080, and the temperature measurement range of the infrared thermal imager is 32-42℃, with a temperature measurement accuracy of ±0.2℃.

[0026] The millimeter-wave radar sensor is installed below the camera, with a transmission frequency of 77GHz and a detection range of 0.3-5m. It can accurately capture subtle movement signals such as the driver's head movements and blinking frequency.

[0027] The voice acoustic sensor uses an omnidirectional microphone with a sensitivity of ≥-38dBV / Pa and a sampling rate of 48kHz. It is installed on the dashboard near the driver's mouth to effectively collect voice signals and characteristic audio such as breathing sounds and yawning sounds.

[0028] The steering wheel grip pressure distribution sensor uses a flexible pressure sensing membrane that is attached to the inside of the train steering wheel. The sensing membrane is divided into 20 uniform pressure acquisition areas, with each area having a pressure detection range of 0-50N and a sampling frequency of 30Hz, capturing changes in grip force in real time.

[0029] All sensors are connected to the edge computing unit via shielded cables, which are concealed to avoid interfering with driving operations.

[0030] (ii) Edge computing unit The neural network SoC built into the edge computing unit uses a high-performance, low-power chip with a computing power of ≥1 TOPS, supports multi-threaded parallel processing, and can process signals collected by multi-modal sensors in real time.

[0031] In the signal processing flow, the raw signals collected by each sensor are first preprocessed, including noise reduction of visible light images, temperature calibration of infrared thermal imaging, filtering of millimeter-wave radar signals, noise reduction of speech signals, and smoothing of grip pressure signals.

[0032] The different modal signals are synchronized and matched according to UTC timestamps by a frame-by-frame alignment algorithm to ensure that the time error is ≤1ms. Then, fatigue-related features are extracted by a cascaded feature extraction network, such as eyelid closure degree and pupil diameter changes in facial features, respiratory rate and heart rate changes in physiological features (calculated by fusing infrared thermal imaging and millimeter-wave radar signals), and grip stability and head shaking frequency in behavioral features.

[0033] The fully connected regression model outputs fatigue confidence scores in the range of 0-1, where 0 represents no fatigue and 1 represents severe fatigue. The preset first-level threshold is 0.6 and the second-level threshold is 0.8. The refresh rate is kept stable at 10Hz to ensure real-time performance.

[0034] The edge computing unit has a built-in 16GB circular cache, which can store 24 hours of raw data and processing results. When the 5G-R network is interrupted, it automatically switches to local storage mode. After the network is restored, it immediately resends the missing data through the breakpoint resume protocol to ensure data integrity.

[0035] (III) Multi-channel stimulus execution unit Module layout and connection: A programmable cold-light LED array is installed on the top of the vehicle-mounted terminal unit 1. It uses soft-light LED beads and emits a cool white light (color temperature 6000K). The brightness can be adjusted within the range of 100-1000 lux to avoid glare. A directional ultrasonic beam emitter is installed on both sides of the driver's cab with a emission angle of ≤30° and an effective range of 1-2m, ensuring that the ultrasonic beam is accurately directed at the driver's ear area. A low-frequency vibration pad is embedded in the backrest and seat cushion of the driver's seat, covering the waist and buttocks, and is designed to be waterproof and dustproof. An aromatic microcapsule electronically controlled release module is installed at the ventilation opening of the driver's cab and connected to the air conditioning airflow channel of the train to ensure uniform diffusion of the odor.

[0036] Each stimulation module is connected to the "heterogeneous stimulation timing arrangement register" via a dedicated control line. The register uses a high-precision clock chip with timing accuracy down to the microsecond level, enabling precise synchronization of the outputs of each module.

[0037] Stimulus parameters and control logic: The heterogeneous stimulus timing orchestration register can be configured with personalized strategies issued by a cloud-based fatigue big data platform, allowing for the setting of the phase difference, duration interval, and repetition count for any two heterogeneous stimulus channels. For example, a programmable cold light LED array and a directional ultrasonic beam emitter can be set with a 90° phase difference, with the LED array continuously lit for 0.5 seconds, at 1-second intervals, and repeated 3 times, while the ultrasonic beam is emitted synchronously, enhancing the stimulation effect.

[0038] The low-frequency vibration pad features a dual modulation circuit of "low frequency-ultrasonic carrier". The baseband signal frequency is dynamically adjusted according to the fatigue level. A low frequency of 0.3-2Hz is used for mild fatigue, 2-5Hz for moderate fatigue, and 5-8Hz for severe fatigue. The ultrasonic carrier is strictly controlled within the range of 40kHz±0.4kHz. The carrier phase and the driving phase of the directional ultrasonic beam transmitter are locked through the same clock tree, with a phase difference of ≤1°, to ensure the synergistic effect of vibration and ultrasonic stimulation.

[0039] The first microcapsule array of the aromatic microcapsule electro-controlled release module contains microcapsules with natural refreshing scents such as menthol and limonene. The release amount is controlled by heating electrodes. The release amount is 0.1-0.3 mg / m³ for mild fatigue, 0.3-0.5 mg / m³ for moderate fatigue, and 0.5-0.8 mg / m³ for severe fatigue.

[0040] Odor rapid neutralization subunit: The second set of microcapsule patches is 5 cm apart from the first microcapsule array. The β-cyclic citral and tea polyphenol compound neutralizing agent in the core is in a 3:7 ratio, which can quickly neutralize the refreshing odor and prevent the odor from accumulating and causing driver discomfort. The independent micro-heating electrode uses an ITO transparent conductive film and shares the ITO glass substrate and FPC cable with the original array, simplifying the structural design. The one-way airflow valve is made of silicone and has an opening pressure of 0.01 MPa, ensuring that the neutralizing agent diffuses only in one direction with the airflow. The on-board micro gas chromatograph in the closed-loop concentration locking circuit has a sampling frequency of 1 Hz. The PID controller adjusts the heating electrode power according to the sampling results to control the concentration of odor molecules in the cockpit within the set range with an error of ≤0.05 mg / m³.

[0041] (iv) Cloud-based fatigue big data platform The cloud-based fatigue big data platform is deployed in the railway operation and dispatch center, adopting a distributed server architecture to support massive data storage and processing. The platform establishes bidirectional communication with the on-board terminal 1 via a 5G-R private network, with a communication rate of ≥1Gbps and a latency of ≤20ms, ensuring real-time data transmission.

[0042] The platform's built-in fatigue recognition model uses a deep learning algorithm. It is trained by continuously collecting multimodal data of train drivers across the country, and the model parameters are updated quarterly and distributed to each onboard terminal to improve the accuracy of fatigue recognition.

[0043] Personalized stimulation strategies are generated based on the driver's age, gender, driving habits, and historical fatigue data. For example, for drivers who are sensitive to light, the brightness and duration of the LED array are reduced, while the intensity of vibration and aromatherapy stimulation is increased; for drivers who drive for long periods of time, a cyclical stimulation strategy is developed to avoid the development of stimulation tolerance.

[0044] (v) Privacy Protection Module A two-level encryption link, using differential and homomorphic methods, ensures driver privacy and security. The first-stage pixel-level differential circuit performs differential operations on adjacent frame facial images, retaining only the changed parts of the image to generate a differential bitstream, reducing the risk of leakage of original image information.

[0045] The second-level national cryptographic SM4 homomorphic encryption core uses a 128-bit key to encrypt the differential code stream, preserving the data's computational characteristics during encryption. The platform can perform model training and analysis without decryption.

[0046] The encrypted data packets are transmitted in segments, with each segment having a checksum added to ensure they are not tampered with during transmission. At the same time, the vehicle-mounted terminal 1 uses a loop buffer to enable resume transmission after network disconnection and reconnection, thus avoiding data loss.

[0047] (vi) Vehicle-to-ground linkage broadcasting mechanism When the fatigue confidence level remains above the secondary threshold (0.8) for 3 consecutive seconds, the edge computing unit activates the "help message hierarchical packaging mechanism": The standard distress message includes information such as driver ID, train number, current location (latitude and longitude error ≤ 10m), train speed, fatigue level, and raw sensor data fragments, with a data size of 1KB.

[0048] The lightweight vehicle-to-ground linkage broadcast frame uses ASN.1 compression encoding to compress key information to 256 bytes. It is broadcast directly to the nearest ground base station using 5G-RSystemInformation messages when the RRC connection is not established, with a broadcast delay of ≤1s. This ensures that dispatchers can quickly obtain key information and take timely dispatch measures, such as arranging temporary stops or changing drivers.

[0049] Specific operation method: S1, Multimodal signal acquisition Once the train starts, the onboard terminal automatically powers on and starts, and all modules synchronously enter the working state. At this time, the multimodal sensor array continuously collects visible light images of the driver's face, infrared thermal images, millimeter-wave radar micro-motion signals, voice acoustic signals, and steering wheel grip pressure distribution signals at a synchronous frame rate of 30Hz. Each frame of the acquired signal is stamped with a UTC timestamp to ensure that different types of signals are accurately aligned in the time dimension, providing an accurate and synchronous data foundation for subsequent signal processing and analysis.

[0050] S2, Edge Signal Fusion and Fatigue Confidence Output After the multimodal sensor array transmits the timestamped signals to the edge computing unit, the edge computing unit uses its built-in neural network SoC to perform frame-by-frame alignment processing on these tamped signals to eliminate time deviations between different modal signals. Subsequently, through a cascaded feature extraction network, fatigue-related features such as eyelid closure degree, pupil diameter change, respiratory rate, heart rate change, grip pressure stability, and head shaking frequency are extracted from the signals. Finally, through a fully connected regression model, a fatigue confidence score in the range of 0–1 is output (0 represents no fatigue state, 1 represents severe fatigue state), and the confidence score refresh frequency is not less than 10Hz to ensure real-time monitoring and judgment of the driver's fatigue state.

[0051] S3, Fatigue Threshold Determination After the edge computing unit outputs the latest fatigue confidence score, it immediately compares it with the preset first-level threshold (0.6). If the fatigue confidence score is less than 0.6, it indicates that the driver is not currently in a state of obvious fatigue, and the device returns to S1 to continue to collect signals through the multimodal sensor array to maintain real-time monitoring; if the fatigue confidence score is greater than or equal to 0.6, it indicates that the driver has shown signs of fatigue, and the device enters S4 to initiate the collaborative intervention process.

[0052] S4, Multi-channel Collaborative Intervention When the driver is found to be showing signs of fatigue, the edge computing unit immediately activates at least two heterogeneous stimulation channels in the multi-channel stimulation execution unit through the "heterogeneous stimulus timing arrangement register" and emits synergistic stimulation according to a programmable phase difference of 0–180°. For example, the programmable cold light LED array and the directional ultrasonic beam emitter are activated simultaneously, and the intervention duration is controlled within 2–4 seconds. Through multi-dimensional stimulation, the driver's attention is quickly awakened and fatigue is relieved.

[0053] S5. Short-term follow-up assessment after intervention After the collaborative intervention ends, the device waits for a 1-second buffer period (to avoid misjudgment due to incomplete intervention effect). Then, the edge computing unit recalculates the driver's fatigue confidence level. If the re-evaluated fatigue confidence level is significantly lower than before the intervention (below the first-level threshold of 0.6), it indicates that the intervention is effective, and the device returns to S1 to resume continuous monitoring. If the re-evaluated fatigue confidence level does not decrease (still greater than or equal to 0.6), the stimulation intensity is automatically increased by one level (e.g., increasing the brightness of the programmable cold light LED array, increasing the vibration frequency of the low-frequency vibration pad, increasing the odor release of the aromatic microcapsule electronically controlled release module, etc.), and the device enters S6 to initiate the vehicle-to-ground reporting process.

[0054] S6, Vehicle-to-Ground Linkage Reporting The edge computing unit sends a standard distress message containing driver ID, train number, current location, speed, fatigue level, and raw sensor data fragments to the cloud-based fatigue big data platform via the 5G-R private network. Simultaneously, it generates a 256-byte lightweight vehicle-to-ground linkage broadcast frame, which is directly broadcast to the nearest ground base station within 1 second via a 5G-R System Information message. This ensures that dispatchers quickly obtain critical information on driver fatigue and train operation, enabling timely emergency dispatching measures (such as arranging temporary stops or coordinating driver replacements), thus completing a full closed-loop anti-fatigue cycle. If continued monitoring is required, the device can restart the process from S1.

[0055] Through a fully closed-loop design of "multimodal perception - real-time edge analysis - multi-channel collaborative intervention - dynamic optimization in the cloud", the problem of low fatigue recognition accuracy and limited intervention effect of single stimulation by traditional single sensors is solved. The multimodal sensor array synchronously collects driver status signals from multiple dimensions such as vision, physiology, and behavior. Combined with real-time fusion analysis by the edge computing unit, it can more accurately capture fatigue characteristics and avoid misjudgment of single signals. The multi-channel stimulation execution unit realizes the synergistic effect of heterogeneous stimuli through microsecond-level synchronous control, and can dynamically adjust parameters according to personalized strategies issued by the cloud. This ensures the effectiveness of intervention and adapts to the tolerance and habits of different drivers, solving the problem of tolerance and poor adaptability of traditional fixed-mode stimulation.

[0056] Furthermore, by combining differential-homogeneous two-level encryption with a train-to-ground linkage broadcast mechanism, the system ensures driver privacy and security while enabling rapid response to high-risk fatigue events. The privacy protection module utilizes differential operations and national cryptographic algorithms for dual protection, reducing the risk of raw data leakage and supporting local storage and resume transmission even when the network is down, ensuring data integrity. The train-to-ground linkage broadcast mechanism can quickly synchronize critical information to the dispatch center with lightweight broadcast frames when the driver is severely fatigued. Compared to traditional reporting methods that rely on a complete network connection, this significantly shortens response time, provides a window for emergency dispatch, and improves the level of train operation safety.

[0057] Considering the need for continuous fatigue monitoring and management during train operation to ensure driving safety, the on-board unit 1 generates a significant amount of heat. To achieve efficient heat dissipation for the on-board unit 1, a specially designed ventilated outer shell 2 is installed on its exterior. Its core function is to provide efficient heat dissipation and dust protection for the on-board unit 1, ensuring stable operation of the equipment over extended periods in the complex environment of the train driver's cab. The ventilated outer shell 2 is constructed from a high-strength, lightweight alloy material, possessing both good structural stability and reduced overall installation weight. An air inlet chamber 202 and an air outlet chamber 203 are precisely pre-drilled between its inner wall and the outer wall of the on-board unit 1. The two chambers are interconnected through a surrounding ventilation gap 201, forming a complete airflow circulation channel.

[0058] To further improve heat dissipation efficiency, multiple heat-absorbing strips 3 are evenly distributed inside the ventilation gap 201. These heat-absorbing strips 3 are made of aluminum alloy with high thermal conductivity and are firmly fixed to the inner wall of the ventilation shell 2 by buckles. They can quickly absorb the heat generated when the vehicle terminal 1 is working, and then efficiently dissipate the heat with the airflow, avoiding performance degradation or failure of the equipment due to high temperature.

[0059] The side of the ventilation housing 2 is provided with a side groove 204 that communicates with the air inlet chamber 202.

[0060] A frame 26 integrally formed with the ventilation shell 2 is provided at the air outlet chamber 203. A second motor 21 is fixedly installed inside the frame 26 by a bracket. A fan 20 is fixedly installed on the drive shaft of the second motor 21. When the fan 20 is started, it can accelerate the airflow circulation between the air inlet chamber 202 and the air outlet chamber 203 and improve the heat dissipation efficiency.

[0061] A panel 22 for closing and dustproofing the frame 26 is provided on the side of the fan 20 facing away from the vehicle-mounted terminal unit 1. Its top is rotatably connected to the inner wall of the frame 26 via a fixed rod 23. The fixed rod 23 and the frame 26 are integrally formed to ensure rotational stability. The top of the panel 22 has an integrally formed second right-angle portion 24 and a second arc portion 25, wherein the second right-angle portion 24 is located close to the vehicle-mounted terminal unit 1, as shown in the reference section. Figure 6 When the plywood 22 is in a vertical position, it can only rotate clockwise, and not clockwise.

[0062] When the fan 20 starts, the thrust generated by the airflow can push the plate 22 to rotate around the fixed rod 23 and open, ensuring that the airflow is discharged smoothly; when the fan 20 stops working, the plate 22 automatically closes under its own gravity, preventing external dust from entering the equipment and achieving dust protection.

[0063] The side groove 204 has a rotating cylinder 4 inside. A first motor 19 is fixedly installed on the outer wall of the ventilation housing 2. The cylinder 4 is fixedly connected to the drive shaft of the first motor 19. The first motor 19 is connected to the factory's power supply and drives the cylinder 4 to rotate.

[0064] The cylinder 4 has an inner groove 5 and an outer groove 6 on its two sides respectively. A filter screen 7 is slidably installed inside the outer groove 6. A fixing plate 10 is fixed on the inner wall of the ventilation shell 2. The fixing plate 10 is attached to the outer wall of the cylinder 4. A collection chamber 11 is formed between the fixing plate 10 and the inner wall of the ventilation shell 2. The ventilation shell 2 is provided with a material receiving trough, a cover plate, etc. (not shown in the figure) that cooperate with the collection chamber 11, which are used to remove the impurities collected in the collection chamber 11 by the material receiving trough.

[0065] Meanwhile, the thickness of the filter screen 7 is less than the thickness of the outer groove 6, forming an outer edge area 601 between the filter screen 7 and the inner wall of the outer groove 6, where impurities accumulate. A vertical plate 8 is fixedly connected to the inner wall of the cylinder 4, and an elastic telescopic rod 9 is fixedly connected to the vertical plate 8. The end of the elastic telescopic rod 9 away from the vertical plate 8 is fixedly connected to the filter screen 7. When the elastic telescopic rod 9 is fully retracted, the inner side of the filter screen 7 corresponds to the inner wall of the cylinder 4, meaning that the filter screen 7 will not move into the interior of the cylinder 4.

[0066] A fixed block 14 is fixedly connected to the inner wall of the ventilation housing 2. The fixed block 14 has a groove at one end near the collection chamber 11. A fixed shaft 15 is rotatably mounted on the fixed block 14, corresponding to the groove. A movable rod 17 is rotatably mounted on the fixed shaft 15. A torsion spring 16 is fitted onto the fixed shaft 15. One end of the torsion spring 16 is fixedly connected to the fixed shaft 15, and the other end is fixedly connected to the movable rod 17. The movable rod 17 has a first arc portion 1701 and a first right-angle portion 1702 at one end near the fixed block 14, with the first right-angle portion 1702 near the filter screen 7. The first arc portion 1701 and the first right-angle portion 1702 are provided (see reference). Figure 9 As shown, the movable bar 17 can only rotate counterclockwise starting from the state of being collinear with the fixed block 14, and the torsion spring 16 can assist the fixed block 14 to reset after rotation; while the movable bar 17 cannot rotate clockwise starting from the state of being collinear with the fixed block 14.

[0067] A middle rod 13 is fixedly connected to the filter screen 7. An arc strip 12 is fixedly connected to the end of the middle rod 13 away from the filter screen 7. The arc strip 12 abuts against the moving strip 17. Both ends of the arc strip 12 are provided with oblique cuts 18 to facilitate contact and cooperation with the moving strip 17.

[0068] In actual use, when the fan 20 is started, the thrust generated by the airflow can push the plate 22 to rotate around the fixed rod 23 and open. At the same time, the external air enters the interior of the cylinder 4 under the filtration of the filter screen 7, and then enters the ventilation housing 2 through the inner side groove 5. Under the action of the fan 20, it is discharged from the frame 26, thereby achieving efficient heat dissipation and ensuring the efficiency of the vehicle terminal 1. Meanwhile, impurities accumulate inside the outer side area 601.

[0069] During the heat dissipation process, the first motor 19 drives the cylinder 4 to rotate back and forth in a small amplitude, causing the angle of the inner groove 5 to change. This avoids the local heat dissipation dead zone caused by the airflow always flowing along a fixed path, allowing the cold air to pass through the ventilation gap 201 more evenly and fully contact the multiple heat absorption strips 3, quickly removing the heat from the surface of the vehicle terminal 1, significantly improving the uniformity and efficiency of heat dissipation, and preventing local overheating of the equipment; and during the above process, the arc strip 12 does not contact the moving strip 17.

[0070] If there are many impurities accumulated inside the outer region 601, refer to Figure 9 The cylinder 4 is controlled to rotate counterclockwise. Then, the arc strip 12 squeezes the moving strip 17, so that the moving strip 17 can rotate counterclockwise with the collinear distribution with the fixed block 14 as the starting state. At this time, the filter screen 7 remains stationary inside the outer groove 6 until the arc strip 12 and the moving strip 17 are misaligned, and the torsion spring 16 assists the fixed block 14 to reset. Then, the cylinder 4 rotates clockwise. Then, the arc strip 12 squeezes the moving strip 17, but the moving strip 17 cannot rotate clockwise with the collinear distribution with the fixed block 14 as the starting state. With the cooperation of the arc strip 12 and the middle rod 13, the filter screen 7 moves away from the axis of the cylinder 4. During the rotation of the cylinder 4, the edge of the side groove 204 pushes and scrapes the impurities accumulated on the filter screen 7, so that the impurities are collected inside the collection chamber 11 until the filter screen 7 is reset.

[0071] In summary, by setting up structures such as the cylinder 4 and the filter screen 7, the present invention filters impurities in the gas entering the ventilation housing 2 and can collect the impurities in a concentrated manner, thereby achieving efficient heat dissipation and ensuring the utilization efficiency of the vehicle-mounted terminal unit 1.

Claims

1. A train driver fatigue-resistant device based on multimodal perception and stimulation, including an on-board terminal (1), wherein the on-board terminal (1) is arranged on the driver's cab and electrically isolated from the train TCMS bus; The vehicle-mounted terminal (1) is fitted with a ventilation shell (2). An air inlet chamber (202) and an air outlet chamber (203) are formed between the inner wall of the ventilation shell (2) and the outer wall of the vehicle-mounted terminal (1). The air inlet chamber (202) and the air outlet chamber (203) are connected through a ventilation gap (201). A side groove (204) communicating with the air inlet chamber (202) is provided on the side of the ventilation shell (2). A cylinder (4) is rotatably arranged inside the side groove (204). An inner side groove (5) and an outer side groove (6) are respectively provided on both sides of the cylinder (4). A filter screen (7) is slidably arranged inside the outer side groove (6). A fixing plate (10) is fixed on the inner wall of the ventilation shell (2). The fixing plate (10) is attached to the outer wall of the cylinder (4). A collection chamber (11) is formed between the fixing plate (10) and the inner wall of the ventilation shell (2). The vehicle-mounted terminal (1) includes: A multimodal sensor array is used to simultaneously acquire visible light images of the driver's face, infrared thermal imaging, millimeter-wave radar micro-motion signals, voice acoustic signals, and steering wheel grip pressure distribution signals. The edge computing unit, with a built-in neural network SoC, is used to fuse signals in real time and output fatigue confidence scores. The multi-channel stimulation execution unit includes: a programmable cold light LED array, a directional ultrasonic beam emitter, a low-frequency vibration pad, and an aroma microcapsule electronically controlled release module; The cloud-based fatigue big data platform communicates bidirectionally with the vehicle terminal (1) via a 5G-R private network to update the fatigue recognition model online and issue personalized stimulation strategies. The privacy protection module uses the national cryptographic algorithm SM4 to homomorphically encrypt uploaded data; When the fatigue confidence level exceeds the first-level threshold, the edge computing unit immediately triggers at least two heterogeneous stimulation channels in the multi-channel stimulation execution unit to perform coordinated intervention, and reassesses the fatigue confidence level within 5 seconds after the intervention. If the fatigue confidence level does not decrease, the stimulation intensity is automatically upgraded and a help message is sent to the cloud-based fatigue big data platform at the same time.

2. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The thickness of the filter screen (7) is less than the thickness of the outer groove (6). An outer edge area (601) is formed between the filter screen (7) and the inner wall of the outer groove (6). A fixed block (14) is fixedly connected to the inner wall of the ventilation shell (2). The fixed block (14) has a groove at one end near the collection chamber (11). A fixed shaft (15) is rotatably mounted on the fixed block (14). The fixed shaft (15) is distributed correspondingly to the groove. A moving bar (17) is rotatably mounted on the fixed shaft (15). A torsion spring (16) is fitted on the fixed shaft (15). One end of the torsion spring (16) is fixedly connected to the fixed shaft (15), and the other end of the torsion spring (16) is fixedly connected to the moving bar (17). 7) The moving bar (17) is provided with a first arc portion (1701) and a first right angle portion (1702) at one end near the fixed block (14), wherein the first right angle portion (1702) is close to the filter screen (7), a middle rod (13) is fixedly connected to the filter screen (7), and an arc strip (12) is fixedly connected to one end of the middle rod (13) away from the filter screen (7). The arc strip (12) abuts against the moving bar (17), and oblique cuts (18) are provided at both ends of the arc strip (12). A first motor (19) is fixedly installed on the outer wall of the ventilation shell (2), and the cylinder (4) is fixedly connected to the drive shaft of the first motor (19).

3. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: A vertical plate (8) is fixedly connected to the inner wall of the cylinder (4), and an elastic telescopic rod (9) is fixedly connected to the vertical plate (8). The end of the elastic telescopic rod (9) away from the vertical plate (8) is fixedly connected to the filter screen (7).

4. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: A frame (26) is provided at the air outlet chamber (203). The frame (26) is integrally formed with the ventilation shell (2). A second motor (21) is installed inside the frame (26). A fan (20) is fixedly installed on the drive shaft of the second motor (21). A plate (22) for closing the frame (26) is provided on the side of the fan (20) facing away from the vehicle terminal (1). A fixed rod (23) is rotatably provided on the top of the plate (22). The fixed rod (23) is fixedly connected to the inner wall of the frame (26). A second right angle portion (24) and a second arc portion (25) are provided at the top of the plate (22). The second right angle portion (24) is close to the vehicle terminal (1).

5. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The ventilation gap (201) is provided with heat-absorbing strips (3), which are fixedly connected to the inner wall of the ventilation shell (2). There are multiple heat-absorbing strips (3), which are evenly distributed.

6. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The multi-channel stimulation execution unit synchronizes the output phase, duration interval and repetition count of the programmable cold light LED array, directional ultrasonic beam emitter, low-frequency vibration pad and aromatic microcapsule electronically controlled release module at the microsecond level through the "heterogeneous stimulation timing arrangement register", so that any two heterogeneous stimulation channels can be programmed to offset within the phase difference range of 0–180°. The low-frequency vibration pad utilizes a "low-frequency-ultrasonic carrier" dual modulation circuit, superimposing an ultrasonic carrier onto a 0.3–8Hz baseband signal. The baseband controls the vibration amplitude, and the carrier phase is locked to the driving phase of the directional ultrasonic beam transmitter through the same clock tree.

7. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The aromatic microcapsule electronically controlled release module includes a first microcapsule array for releasing a refreshing scent, and an odor rapid neutralization subunit is built into a slot on the wall of the original airflow channel, located downstream of the air outlet side of the first microcapsule array. The odor rapid neutralization subunit includes a second set of microcapsule patches, an independent micro-heating electrode, a one-way airflow valve, and a closed-loop concentration locking circuit. The core of the second group of microcapsule patches is a compound neutralizing agent of β-cyclic citral and tea polyphenols; the independent micro-heating electrode shares the same ITO glass substrate and FPC cable as the original array; the one-way airflow valve is used to guide the neutralizing agent to diffuse unidirectionally with the airflow; the closed-loop concentration locking circuit uses a vehicle-mounted micro gas chromatograph and PID controller to sample and feedback the concentration of odor molecules in real time, and control the amount of neutralized release.

8. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The privacy protection module is equipped with a two-level encryption link of "differential-homomorphic". The first-level encryption unit uses a pixel-level differential circuit to perform differential analysis on adjacent frame images and outputs a differential code stream. The second-level encryption unit uses the national cryptographic SM4 homomorphic encryption core to encrypt the differential code stream. The encrypted data packets are transmitted through the 5G-R private network and, with the help of the built-in loop buffer of the vehicle terminal (1), the data is stored locally in the case of signal interruption. After the network is restored, the missing frames are resent through the breakpoint resume transmission protocol.

9. The train driver anti-fatigue device based on multimodal perception and stimulation according to claim 1, characterized in that: The edge computing unit uses a "hierarchical packaging mechanism for help messages" to handle high-risk fatigue events: when the fatigue confidence level is continuously higher than the secondary threshold, while sending a standard help message to the cloud fatigue big data platform, a lightweight vehicle-to-ground linkage broadcast frame is generated through the 5G-R protocol stack. The vehicle-to-ground linkage broadcast frame uses ASN.1 compression encoding and utilizes 5G-RSystemInformation messages to directly broadcast to the nearest ground base station when the RRC connection is not established, enabling the dispatcher to obtain the driver ID, train position, speed, and fatigue level.

10. A method for combating fatigue in train drivers based on multimodal perception and stimulation, characterized in that: The train driver fatigue resistance device based on multimodal perception and stimulation as described in any one of claims 1 to 9 further includes the following operational steps: S1. Multimodal acquisition: The driver's face visible light image, infrared thermal imaging, millimeter-wave radar micro-motion signal, voice acoustic signal and steering wheel grip pressure distribution signal are continuously acquired through a multimodal sensor array at a synchronous frame rate of 30Hz, and each frame of signal is stamped with a UTC timestamp. S2, Edge Fusion: Utilizing the neural network SoC built into the edge computing unit, the S1 stamped signal is aligned frame by frame and cascaded feature extraction is performed. The fatigue confidence in the 0–1 interval is output through fully connected regression, with a refresh rate of not less than 10Hz. S3, Threshold Judgment: Compare the latest fatigue confidence level with the preset first-level threshold; if the confidence level is ≥ the first-level threshold, proceed to S4; if the confidence level is < the first-level threshold, return to S1. S4. Cooperative Intervention: The edge computing unit immediately activates at least two heterogeneous stimulus channels through the "heterogeneous stimulus timing arrangement register" and emits cooperative stimulation with a programmable phase difference of 0–180°. The intervention lasts for 2–4 seconds. S5. Short-term review: After the intervention, wait 1 second for a buffer period and recalculate the fatigue confidence level; if the confidence level decreases, return to S1; if the confidence level does not decrease, increase the stimulus intensity by one level and proceed to S6. S6, Vehicle-to-Ground Reporting: The edge computing unit sends a standard help message to the cloud-based fatigue big data platform through the 5G-R private network, and at the same time generates a 256-byte vehicle-to-ground linkage broadcast frame, which is broadcast to the nearest ground base station within 1 second via the SystemInformation message, completing one closed-loop anti-fatigue cycle.

Images