Fatigue driving early warning processing system and method applied to commercial vehicle

Abstract

The invention provides a fatigue driving early warning processing system and method applied to a commercial vehicle, and relates to the technical field of vehicle fatigue driving early warning. The system comprises a data acquisition device, a data processing device, a fatigue detection device and an early warning processing device, the devices are in data connection and are connected with a vehicle control unit of the commercial vehicle; the data acquisition device is used for acquiring vehicle driving behavior data and user data; the data processing device is used for preprocessing the behavior data and self data to obtain to-be-matched data; the fatigue detection device is used for performing fatigue detection processing according to the to-be-matched data, pre-stored reference data, a preset matching condition and a pre-training deep learning model, and determining a driving state; the early warning processing device is used for executing early warning processing and generating an early warning signal when the driving state is a fatigue state. The vehicle driving behavior data and the user data are fused to judge the driving state, the defect of a single data source is overcome, and the detection accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicle fatigue driving warning technology, and in particular to a fatigue driving warning processing system and method for commercial vehicles. Background Technology

[0002] With rapid economic development and increasing logistics demands, drivers are spending more time on long-distance driving, leading to a growing problem of driver fatigue. Fatigue driving not only threatens the safety of the driver themselves but also poses potential risks to other road users. Therefore, developing effective fatigue driving prevention technologies is crucial for improving road safety.

[0003] Currently, existing fatigue driving prevention systems primarily analyze driver fatigue levels by analyzing driving behavior and vehicle status parameters. However, this monitoring method, which relies on only a single data source, has a low accuracy rate in determining fatigue driving and cannot comprehensively and accurately reflect the driver's true fatigue state. Summary of the Invention

[0004] This application provides a fatigue driving early warning and processing system and method for commercial vehicles, in order to solve the technical problem that the accuracy of fatigue driving judgment using a single data source is low in the prior art.

[0005] In a first aspect, this application provides a fatigue driving warning and processing system for commercial vehicles, comprising: a data acquisition device, a data processing device, a fatigue detection device, and a warning and processing device;

[0006] The data acquisition device, the data processing device, the fatigue detection device, and the early warning processing device are all connected to each other, and the data acquisition device, the data processing device, the fatigue detection device, and the early warning processing device are connected to the vehicle control unit of the commercial vehicle via the vehicle bus.

[0007] The data acquisition device is used to acquire vehicle driving behavior data and user data.

[0008] The data processing device is used to preprocess the vehicle driving behavior data and the user's own data to obtain data to be matched.

[0009] The fatigue detection device is used to perform fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions and pre-trained deep learning model to determine the user's driving state.

[0010] The warning processing device is used to perform warning processing and generate a warning signal when it detects that the user's driving state is fatigued.

[0011] Secondly, this application provides a fatigue driving warning and processing method for commercial vehicles, applied to a vehicle control unit, wherein the vehicle control unit is data-connected to the fatigue driving warning and processing system for commercial vehicles provided in the first aspect of this application, and the method includes:

[0012] Acquire vehicle driving behavior data and user personal data;

[0013] The vehicle driving behavior data and the user's own data are preprocessed to obtain the data to be matched.

[0014] Fatigue detection is performed based on the data to be matched, pre-stored benchmark data, preset matching conditions, and pre-trained deep learning model to determine the user's driving status.

[0015] When the user's driving state is detected to be fatigued, an early warning process is executed, and an early warning signal is generated.

[0016] In one possible design, the data to be matched includes physiological signal feature data, visual feature data, and driving behavior feature data, and the pre-stored benchmark data includes physiological feature thresholds, visual feature thresholds, and driving behavior feature thresholds.

[0017] The step of performing fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions, and a pre-trained deep learning model to determine the user's driving state includes:

[0018] The physiological signal feature data and the physiological feature threshold are input into a pre-trained deep learning model for physiological feature matching processing to obtain physiological fatigue detection results.

[0019] The visual feature data and the visual feature threshold are input into a pre-trained deep learning model for visual feature matching processing to obtain visual fatigue detection results.

[0020] The driving behavior feature data and the driving behavior feature threshold are input into a pre-trained deep learning model for driving behavior feature matching processing to obtain driving behavior fatigue detection results.

[0021] The user's driving status is determined based on the physiological fatigue detection results, the visual fatigue detection results, the driving behavior fatigue detection results, and preset matching conditions.

[0022] In one possible design, the physiological fatigue detection result includes physiological fatigue similarity, the visual fatigue detection result includes visual fatigue similarity, the driving behavior fatigue detection result includes driving behavior fatigue similarity, and the preset matching conditions include physiological matching threshold, visual matching threshold, and behavioral matching threshold.

[0023] The step of determining the user's driving state based on the physiological fatigue detection results, the visual fatigue detection results, the driving behavior fatigue detection results, and preset matching conditions includes:

[0024] The physiological fatigue similarity and the physiological matching threshold are compared to determine the physiological matching result.

[0025] The visual fatigue similarity and the visual matching threshold are compared to determine the visual matching result.

[0026] The driving behavior fatigue similarity and the behavior matching threshold are compared to determine the behavior matching result.

[0027] When the physiological matching result, the visual matching result, and the behavioral matching result all meet the matching success conditions, the user's driving state is determined to be fatigued.

[0028] One possible design also includes:

[0029] A comprehensive fatigue probability value is obtained by performing a comprehensive probability calculation based on the physiological matching result, the visual matching result, and the behavioral matching result.

[0030] When the overall fatigue probability value is detected to be greater than a preset probability threshold, and at least two of the physiological matching results, the visual matching results, and the behavioral matching results meet the matching success conditions, the user's driving state is determined to be fatigued.

[0031] One possible design also includes:

[0032] When the visual feature data is detected to meet the dynamic adjustment conditions, the dynamic weight coefficient ratio is obtained;

[0033] The dynamic fatigue probability value is determined based on the dynamic weighting coefficient ratio, the physiological matching result, the visual matching result, and the behavioral matching result.

[0034] When the dynamic fatigue probability value is detected to be greater than the preset probability threshold, and both the physiological matching result and the behavioral matching result meet the matching success condition, the user's driving state is determined to be fatigued.

[0035] One possible design also includes:

[0036] When the user's driving state is detected to be fatigued, the fatigue driving level is determined based on the comprehensive fatigue probability value, the physiological matching result, the visual matching result, and the behavioral matching result.

[0037] Based on the fatigue driving level, a corresponding warning strategy is selected from the preset warning strategy library to perform warning processing and generate a warning signal.

[0038] One possible design also includes:

[0039] Obtain the duration of the warning signal after it is generated;

[0040] When the warning duration is detected to be greater than the intelligent driving takeover duration threshold, data on the vehicle's surrounding environment is acquired.

[0041] Based on the environmental data surrounding the vehicle, the vehicle is controlled to decelerate to the right until it stops in a safe area.

[0042] In one possible design, the execution of the early warning process, generating an early warning signal, includes:

[0043] Perform warning processing, generate warning data and send it to the vehicle's instrument panel to cause the instrument panel to generate a visual warning signal; and / or generate warning data and send it to a sound generator to cause the sound generator to generate an audible warning signal; and / or generate warning data and send it to the seat to cause the seat to generate a behavioral perception signal.

[0044] In one possible design, the data preprocessing of the vehicle driving behavior data and the user's own data to obtain the data to be matched includes:

[0045] The vehicle driving behavior data and the user's own data are subjected to noise reduction processing to obtain denoised data to be processed.

[0046] The denoised data to be processed is normalized to obtain normalized data to be processed.

[0047] Feature extraction is performed on the normalized data to be processed to obtain the data to be matched.

[0048] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0049] The memory stores computer-executed instructions;

[0050] The processor executes computer execution instructions stored in the memory to implement the fatigue driving warning processing method for commercial vehicles provided in the second aspect of this application.

[0051] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the fatigue driving warning processing method for commercial vehicles provided in the second aspect of this application.

[0052] Fifthly, this application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the fatigue driving warning processing method for commercial vehicles provided in the second aspect of this application.

[0053] This application provides a fatigue driving warning and processing system and method for commercial vehicles. The system includes: a data acquisition device, a data processing device, a fatigue detection device, and a warning processing device. All three devices are interconnected and connected to the vehicle control unit of the commercial vehicle via a vehicle bus. The data acquisition device acquires vehicle driving behavior data and user data. The data processing device preprocesses the vehicle driving behavior data and user data to obtain matching data. The fatigue detection device performs fatigue detection processing based on the matching data, pre-stored benchmark data, preset matching conditions, and a pre-trained deep learning model to determine the user's driving state. The warning processing device executes warning processing and generates a warning signal when the user's driving state is detected to be fatigued. Through this structural design, the following technical effects are achieved: by fusing vehicle driving behavior data and user data to determine the user's driving state, the shortcomings of a single data source are overcome, and the accuracy of detection is improved. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 An interactive schematic diagram of a fatigue driving warning and processing system for commercial vehicles provided in an embodiment of this application;

[0056] Figure 2 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 1 ;

[0057] Figure 3A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 2 ;

[0058] Figure 4 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 3 ;

[0059] Figure 5 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 4 ;

[0060] Figure 6 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 5 ;

[0061] Figure 7 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 6 ;

[0062] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0063] Explanation of reference numerals in the attached figures:

[0064] 810 - Processor; 820 - Memory; 830 - Communication components; 840 - Bus. Detailed Implementation

[0065] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0066] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply difference. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more.

[0067] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0068] Head-Up Display (HUD): A display technology. HUDs typically project information onto a transparent display screen or windshield, allowing operators to view important data without shifting their gaze. This allows users to obtain crucial real-time information while focusing on their current task, making it suitable for scenarios requiring high concentration and rapid response.

[0069] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0070] Current technologies primarily analyze driver fatigue levels by analyzing driving behavior and vehicle status parameters. This monitoring method, which relies on a single data source, has a low accuracy rate in determining driver fatigue and cannot comprehensively and accurately reflect the driver's true fatigue state.

[0071] In summary, how to design a technical solution to the problem of low accuracy in fatigue driving judgment using a single data source is the problem that this application urgently needs to solve.

[0072] Therefore, in view of the above-mentioned technical problems existing in the prior art, the embodiments of this application provide a fatigue driving warning processing system and method for commercial vehicles, which can be used in the field of vehicle fatigue driving warning technology, and aims to solve the technical problem of low accuracy of fatigue driving judgment using a single data source.

[0073] The following describes the application scenarios of the fatigue driving warning and processing system for commercial vehicles provided in the embodiments of this application. The following application scenarios are merely examples, intended to help those skilled in the art understand the technical content of this application, but do not mean that the embodiments of this application cannot be used in other devices, systems, environments, or scenarios.

[0074] 1) Long-distance freight transportation: During long-distance freight transportation, drivers typically need to drive for extended periods, resulting in a high risk of driver fatigue. The fatigue driving warning and processing system and method for commercial vehicles provided in this application can monitor the driver's fatigue state in real time and issue warnings when the driver is driving while fatigued, reminding the driver to rest in time and thus avoiding traffic accidents caused by fatigue driving.

[0075] 2) Public Transportation Vehicles: In long-distance passenger transport services, drivers often need to traverse long distances, making fatigue driving a more prominent issue. The fatigue driving warning and processing system and method for commercial vehicles provided in this application can effectively detect the driver's fatigue level and ensure passenger safety. Simultaneously, it can automatically activate warning measures when fatigue driving occurs, prompting the driver to rest or hand over driving duties.

[0076] 3) Logistics and delivery vehicles: In urban delivery and rural logistics, commercial vehicles often need to drive in narrow streets and complex traffic environments, and drivers' prolonged concentration may lead to fatigue. The fatigue driving warning and processing system and method for commercial vehicles provided in this application embodiment can automatically issue a warning when the driver is fatigued, reminding the driver to stop and rest or to change drivers, thereby improving road safety and reducing the occurrence of traffic accidents.

[0077] 4) Special Operation Vehicles: In some special operations, such as engineering vehicles, fire trucks, and rescue vehicles, drivers often need to perform emergency tasks and are under high pressure for extended periods. Fatigue driving can significantly affect the successful completion of tasks and create safety hazards. The fatigue driving warning and processing system and method for commercial vehicles provided in this application can ensure that drivers remain alert during operation, improving the efficiency and safety of task execution.

[0078] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0079] Figure 1This is an interactive schematic diagram of a fatigue driving warning and processing system for commercial vehicles provided in an embodiment of this application. Figure 1 As shown in the figure, this application provides a fatigue driving warning and processing system for commercial vehicles. The system includes: a data acquisition device, a data processing device, a fatigue detection device, and a warning and processing device.

[0080] The data acquisition device, data processing device, fatigue detection device, and early warning processing device are all connected to each other, and the data acquisition device, data processing device, fatigue detection device, and early warning processing device are connected to the vehicle control unit of the commercial vehicle via the vehicle bus.

[0081] Data acquisition devices are used to acquire vehicle driving behavior data and user personal data.

[0082] In this embodiment, vehicle driving behavior data, including steering wheel angle, vehicle speed, and lateral acceleration, can be acquired through onboard sensors for monitoring driving behavior. User data, including the driver's facial expressions, eye movements, head posture, and physiological signals, is monitored via a facial recognition camera to track changes in the driver's driving state and alertness. Wearable devices such as smart bracelets can collect physiological signals such as the driver's heart rate and respiratory rate. This process involves physical transformation, converting the driver's physiological state and vehicle driving behavior characteristics into quantifiable data. Sensors in the smart seat can also be used to acquire changes in the driver's heart rate, increasing the flexibility of data sources.

[0083] The data processing device is used to preprocess vehicle driving behavior data and user data to obtain data to be matched.

[0084] In this embodiment, vehicle driving behavior data and user data undergo data preprocessing, including noise reduction, normalization, and feature extraction, to ensure the data is clear and suitable for further analysis. Key steps include: using filtering algorithms to remove noise from physiological signal data; and extracting key features such as heart rate fluctuations and facial expression changes through feature selection for subsequent analysis.

[0085] The fatigue detection device is used to perform fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions, and pre-trained deep learning models to determine the user's driving state.

[0086] In this embodiment, fatigue detection is performed using a pre-trained deep learning model based on the data to be matched, pre-stored benchmark data, and preset matching conditions. The pre-trained deep learning model can be CNN-Transformer, or other machine learning models or algorithms; there are no specific limitations on this.

[0087] By integrating vehicle driving behavior data and user data to determine the user's driving status, the shortcomings of a single data source are overcome, and the accuracy of detection can be improved.

[0088] The warning processing device is used to perform warning processing and generate a warning signal when it detects that the user is in a state of fatigue while driving.

[0089] In this embodiment, when the system detects that the user is fatigued while driving, it provides a visual warning through the head-up display (HUD) system, and at the same time plays a voice reminder or haptic feedback to improve the driver's alertness.

[0090] This application provides a fatigue driving warning and processing system for commercial vehicles. The system includes: a data acquisition device, a data processing device, a fatigue detection device, and a warning processing device. All three devices are interconnected and connected to the vehicle control unit of the commercial vehicle via a vehicle bus. The data acquisition device acquires vehicle driving behavior data and user data. The data processing device preprocesses the vehicle driving behavior data and user data to obtain matching data. The fatigue detection device performs fatigue detection processing based on the matching data, pre-stored benchmark data, preset matching conditions, and a pre-trained deep learning model to determine the user's driving state. The warning processing device executes warning processing and generates a warning signal when the user's driving state is detected to be fatigued. Through this structural design, the following technical effects are achieved: by fusing vehicle driving behavior data and user data to determine the user's driving state, the shortcomings of a single data source are overcome, and the accuracy of detection is improved.

[0091] Figure 2 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 1 This embodiment provides a fatigue driving warning and processing method for commercial vehicles. The method is applied to a vehicle control unit, which is data-connected to the fatigue driving warning and processing system for commercial vehicles provided in the above embodiment. The method includes the following steps:

[0092] S101. Obtain vehicle driving behavior data and user's own data.

[0093] In this embodiment, the methods for obtaining vehicle driving behavior data and user personal data have been mentioned in the above embodiments and will not be repeated here.

[0094] S102. Perform data preprocessing on vehicle driving behavior data and user's own data to obtain data to be matched.

[0095] In this embodiment, the method of preprocessing vehicle driving behavior data and user data has been mentioned in the above embodiments and will not be repeated here.

[0096] S103. Perform fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions, and pre-trained deep learning model to determine the user's driving status.

[0097] In this embodiment, the method for fatigue detection has been mentioned in the above embodiments and will not be repeated here.

[0098] S104. When the user's driving state is detected to be fatigued, a warning process is executed and a warning signal is generated.

[0099] In this embodiment, when the user's driving state is detected to be fatigued, a warning process is executed. The method of generating the warning signal has been mentioned in the above embodiments and will not be repeated here.

[0100] The technical effects of the embodiments in this application are similar to those of the embodiments described above, and will not be repeated here.

[0101] Figure 3 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 2 Based on the above embodiments, this embodiment further explains the fatigue driving warning processing method applied to commercial vehicles. In this embodiment, the data to be matched includes physiological signal feature data, visual feature data, and driving behavior feature data, and the pre-stored benchmark data includes physiological feature thresholds, visual feature thresholds, and driving behavior feature thresholds. S103 includes:

[0102] S201. Input the physiological signal feature data and physiological feature threshold into the pre-trained deep learning model for physiological feature matching processing to obtain the physiological fatigue detection result.

[0103] In this embodiment, the pre-stored benchmark data is a pre-established feature library of normal driving states, including physiological feature thresholds, visual feature thresholds, driving behavior feature thresholds, and dynamic thresholds. The dynamic thresholds are dynamically adjusted based on driving scenarios such as nighttime and rainy days, as well as environmental factors such as wearing masks. For example, when wearing a mask, the matching weight of the mouth closure rate (MCR) is reduced, while the weight of the pitch angle is increased. The purpose of this design is to adapt to complex scenarios and avoid misjudgments caused by sensor obstruction or environmental interference.

[0104] Physiological signal characteristics include heart rate variability (HRV) and respiratory rate. Physiological fatigue detection results include indicators of abnormalities in HRV and respiratory rate.

[0105] The physiological characteristic threshold corresponding to HRV is 50ms, and the physiological characteristic thresholds corresponding to respiratory rate are 10 breaths / minute and 25 breaths / minute. Physiological signal characteristic data and their corresponding physiological characteristic thresholds are input into a pre-trained deep learning model for physiological characteristic matching. If HRV is less than 50ms, it is considered abnormal. Similarly, respiratory rate and its corresponding physiological characteristic thresholds are input into the pre-trained deep learning model for physiological characteristic matching. If the respiratory rate is less than 10 breaths / minute or greater than 25 breaths / minute, it is considered abnormal.

[0106] S202. Input the visual feature data and visual feature threshold into the pre-trained deep learning model for visual feature matching processing to obtain the visual fatigue detection results.

[0107] In this embodiment, visual feature data includes eye movements, facial expressions, and head posture. Eye movements include the percentage of eyelid closure time over the pupillary area (PERCLOS), blink frequency, and eye-closing time. Facial expressions include yawning frequency and MCR (motor circle rate). Head posture includes non-frontal face rate (NFR), pitch angle, and nodding frequency. Visual fatigue detection results include indications of abnormalities in PERCLOS, blink frequency, yawning frequency, MCR, NFR, pitch angle, and nodding frequency. Pitch angle can be obtained through an inertial measurement unit (IMU).

[0108] The visual feature thresholds corresponding to PERCLOS are 20%, those corresponding to blink frequency are 8 times / minute and 12 times / minute, those corresponding to eye-closing time are 5 seconds, those corresponding to yawn frequency are 3 times / minute, those corresponding to MCR are 50%, those corresponding to NFR are 30%, those corresponding to pitch angle are 15°, and those corresponding to nodding frequency are 5 times / minute. Visual feature data and their corresponding visual feature thresholds are input into a pre-trained deep learning model for visual feature matching. If PERCLOS is greater than or equal to 20%, PERCLOS is considered abnormal; if blinking frequency is less than 8 times / minute or greater than 12 times / minute, blinking frequency is considered abnormal; if eye closure time is greater than 5 seconds, eye closure time is considered abnormal; if yawning frequency is greater than 3 times / minute, yawning frequency is considered abnormal; if MCR is greater than 50%, MCR is considered abnormal; if NFR is greater than 30%, NFR is considered abnormal; if pitch angle is greater than 15°, pitch angle is considered abnormal; if head nodding frequency is greater than 5 times / minute, head nodding frequency is considered abnormal.

[0109] In special circumstances where it is impossible to accurately capture eye movements, facial expressions, and head posture, such as when the driver is wearing a mask or in insufficient lighting, infrared cameras and thermal imaging technology, along with head posture compensation and physiological signal enhancement, can be used. Specifically, infrared sensors capture facial heat distribution, identify pupil thermal radiation characteristics in the eye area and mouth movements under the mask, use an IMU to monitor head acceleration and angular velocity, and combine this with steering wheel angle data to infer the direction of gaze (e.g., the head naturally tilts to the right when frequently turning right), and indirectly estimate heart rate using steering wheel grip force sensors and seat pressure sensors.

[0110] S203. Input the driving behavior feature data and driving behavior feature threshold into the pre-trained deep learning model to perform driving behavior feature matching processing and obtain the driving behavior fatigue detection result.

[0111] In this embodiment, driving behavior characteristic data includes steering wheel angular velocity and lane departure frequency. Driving behavior fatigue detection results include indications of whether there are abnormalities in the steering wheel angular velocity and lane departure frequency.

[0112] The driving behavior feature threshold corresponding to steering wheel angular velocity is 30° / s, and the driving behavior feature threshold corresponding to lane departure frequency is 3 times / minute. Driving behavior feature data and their corresponding thresholds are input into a pre-trained deep learning model for driving behavior feature matching. If the steering wheel angular velocity is greater than 30° / s to the left or to the right, it is considered abnormal; if the lane departure frequency is greater than 3 times / minute, it is considered abnormal.

[0113] S204. Determine the user's driving status based on the results of physiological fatigue detection, visual fatigue detection, driving behavior fatigue detection, and preset matching conditions.

[0114] In this embodiment, if the physiological fatigue detection results, visual fatigue detection results, and driving behavior fatigue detection results all show abnormalities in physiological signal characteristic data, visual characteristic data, or driving behavior characteristic data, then the user's driving state is determined to be fatigued. For example, if PERCLOS is greater than or equal to 20% or the yawning frequency is greater than 3 times / minute, then the user's driving state is determined to be fatigued.

[0115] By fusing physiological signal characteristic data, visual characteristic data, and driving behavior characteristic data, the accuracy of the judgment can be improved in determining whether the user is in a state of fatigue.

[0116] Figure 4 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 3 Based on the above embodiments, this embodiment further explains the fatigue driving warning processing method applied to commercial vehicles. In this embodiment, the physiological fatigue detection result includes physiological fatigue similarity, the visual fatigue detection result includes visual fatigue similarity, the driving behavior fatigue detection result includes driving behavior fatigue similarity, and the preset matching conditions include physiological matching threshold, visual matching threshold, and behavioral matching threshold. S204 includes:

[0117] S301. Perform threshold comparison processing on physiological fatigue similarity and physiological matching threshold to determine the physiological matching result.

[0118] In this embodiment, the physiological matching threshold is set to 85%. The physiological fatigue detection result also includes physiological fatigue similarity, which includes HRV fatigue similarity and respiratory rate fatigue similarity. If the feature similarity between HRV and its corresponding physiological feature threshold, i.e., the HRV fatigue similarity, is greater than or equal to 85%, the physiological matching result is successful. Similarly, if the feature similarity between respiratory rate and its corresponding physiological feature threshold, i.e., the respiratory rate fatigue similarity, is greater than or equal to 85%, the physiological matching result is successful. Optionally, the feature similarity includes Euclidean distance and cosine similarity.

[0119] S302. Perform threshold comparison processing on visual fatigue similarity and visual matching threshold to determine the visual matching result.

[0120] In this embodiment, the visual matching threshold is set to 85%. The visual fatigue detection result also includes visual fatigue similarity, which includes PERCLOS fatigue similarity, blink frequency fatigue similarity, yawn frequency fatigue similarity, MCR fatigue similarity, NFR fatigue similarity, pitch angle fatigue similarity, and nodding frequency fatigue similarity. The definitions of PERCLOS fatigue similarity, blink frequency fatigue similarity, yawn frequency fatigue similarity, MCR fatigue similarity, NFR fatigue similarity, pitch angle fatigue similarity, and nodding frequency fatigue similarity are similar to the definitions of HRV fatigue similarity, and will not be repeated here.

[0121] For PERCLOS fatigue similarity, blink frequency fatigue similarity, yawn frequency fatigue similarity, MCR fatigue similarity, NFR fatigue similarity, pitch angle fatigue similarity, and nodding frequency fatigue similarity, if any of the above data is greater than or equal to 85%, the visual matching result is considered successful.

[0122] S303. Perform threshold comparison processing on the similarity of driving behavior fatigue and the behavior matching threshold to determine the behavior matching result.

[0123] In this embodiment, the behavior matching threshold is set to 85%. The driving behavior fatigue detection result also includes driving behavior fatigue similarity, which includes steering wheel angular velocity similarity and lane departure frequency similarity. The definitions of steering wheel angular velocity similarity and lane departure frequency similarity are similar to those of HRV fatigue similarity, and will not be repeated here.

[0124] For steering wheel angular velocity similarity and lane departure frequency similarity, if any of the above data is greater than or equal to 85%, the behavior matching result is successful.

[0125] S304. When the physiological matching result, visual matching result and behavioral matching result all meet the matching success conditions, the user's driving state is determined to be fatigued.

[0126] In this embodiment, when the physiological matching result, visual matching result, and behavioral matching result all meet the matching success condition, the user's driving state is determined to be fatigued. By fusing physiological fatigue similarity, visual fatigue similarity, and driving behavior fatigue similarity to determine whether the user's driving state is fatigued, the accuracy of the judgment can be improved.

[0127] Figure 5 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 4 Based on the above embodiments, this embodiment further explains the fatigue driving warning processing method applied to commercial vehicles. In this embodiment, after determining the physiological matching result in S301, the visual matching result in S302, and the behavioral matching result in S303, the method further includes:

[0128] S401. Based on the physiological matching results, visual matching results, and behavioral matching results, a comprehensive probability calculation is performed to obtain the comprehensive fatigue probability value.

[0129] In this embodiment, a comprehensive fatigue probability value is calculated using a random forest or SVM model based on physiological matching results, visual matching results, and behavioral matching results. This involves a weighted fusion of physiological fatigue similarity, visual fatigue similarity, and driving behavior fatigue similarity. Different data fusion algorithms, such as neural networks or support vector machines, can be used to adapt to different driving environments and individual differences.

[0130] S402. When the overall fatigue probability value is detected to be greater than the preset probability threshold, and at least two of the physiological matching results, visual matching results and behavioral matching results meet the matching success conditions, the user's driving state is determined to be fatigued.

[0131] In this embodiment, the preset probability threshold is 0.7. When the overall fatigue probability value is detected to be greater than 0.7, and at least two of the physiological matching result, visual matching result and behavioral matching result meet the matching success condition, the user's driving state is determined to be fatigued.

[0132] When the physiological matching result, visual matching result, and behavioral matching result all fail to meet the matching success conditions, and the overall fatigue probability value is less than 0.3, the user's driving state is a non-fatigue state.

[0133] The overall fatigue probability value supplements the similarity threshold. If a match fails but other matches succeed, a conditional compensation mechanism is triggered, which improves the accuracy of the judgment. For example, a visual fatigue similarity of 60% means that the visual match fails. However, if the overall fatigue probability value is 0.75, and both the visual and behavioral match results meet the matching success conditions, the user's driving state is determined to be fatigued.

[0134] Here is a real-world example:

[0135] During nighttime driving, the driver's wearing of sunglasses rendered visual feature data ineffective. At this time, the steering wheel angular velocity abruptly changed 6 times per minute, the heart rate variability (HRV) was 45 ms, and the pitch angle was 18°. In this scenario, the visual matching result failed, the overall fatigue probability value was 0.75, and both the visual and behavioral matching results met the matching success conditions. Therefore, the user's driving state was determined to be fatigued.

[0136] Figure 6 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 5 Based on the above embodiments, this embodiment further explains the fatigue driving warning processing method applied to commercial vehicles. In this embodiment, after determining the physiological matching result in S301, the visual matching result in S302, and the behavioral matching result in S303, the method further includes:

[0137] S501. When visual feature data is detected to meet the dynamic adjustment conditions, the dynamic weight coefficient ratio is obtained.

[0138] In this embodiment, if visual feature data indicates that the user is wearing a mask or the lighting conditions are insufficient, the weighting coefficient for visual fatigue similarity is reduced, while the weighting coefficient for physiological fatigue similarity is increased. Initial values ​​are set for the weighting coefficients of these three data types. The weighting coefficient for visual fatigue similarity is 0.5 under normal lighting conditions, decreases to 0.4 under slight occlusion or low light conditions, and decreases to 0.2 in scenarios where visual feature data is ineffective, such as when wearing a mask. After implementation, the weighting coefficients will be further fine-tuned and corrected based on the collection and feedback of actual measurement data.

[0139] As an optional implementation method, the weighting coefficient for visual fatigue similarity is 0.2, the weighting coefficient for physiological fatigue similarity is 0.5, and the weighting coefficient for driving behavior fatigue similarity is 0.3.

[0140] S502. Determine the dynamic fatigue probability value based on the dynamic weighting coefficient ratio, physiological matching results, visual matching results, and behavioral matching results.

[0141] In this embodiment, with the weighting coefficients of visual fatigue similarity being 0.2, physiological fatigue similarity being 0.5, and driving behavior fatigue similarity being 0.3, the similarity of physiological fatigue, visual fatigue, and driving behavior fatigue is weighted and fused to obtain a dynamic fatigue probability value.

[0142] S503. When the detected dynamic fatigue probability value is greater than the preset probability threshold, and both the physiological matching result and the behavioral matching result meet the matching success condition, the user's driving state is determined to be fatigued.

[0143] In this embodiment, if the dynamic fatigue probability value is greater than the preset probability threshold, and both the physiological matching result and the behavioral matching result meet the matching success condition, then the user's driving state is determined to be fatigued.

[0144] When it is determined that a user is in a special situation such as wearing a mask or insufficient lighting, the accuracy of the judgment can be effectively improved by reducing the weight coefficient of visual fatigue similarity and increasing the weight coefficient of physiological fatigue similarity.

[0145] Figure 7 A flowchart illustrating the fatigue driving warning and processing method for commercial vehicles provided in this application embodiment. Figure 6 Based on the above embodiments, this embodiment further explains the fatigue driving warning and processing method applied to commercial vehicles. In this embodiment, after S402, it further includes:

[0146] S601. When a user's driving state is detected to be fatigued, the fatigue driving level is determined based on the comprehensive fatigue probability value, physiological matching result, visual matching result, and behavioral matching result.

[0147] In this embodiment, when the overall fatigue probability value is between 0.3 and 0.5, PERCLOS is less than 30%, or the blinking frequency is less than 68 times / minute, the fatigue driving level is determined to be Level 1, mild fatigue; when the overall fatigue probability value is between 0.5 and 0.7, PERCLOS is greater than 30%, HRV is less than 50ms, or the yawning frequency is greater than or equal to 3 times / minute, the fatigue driving level is determined to be Level 2, moderate fatigue; when the overall fatigue probability value is greater than or equal to 0.7, the eye-closing time is greater than 5s, and the steering wheel angular velocity change frequency is greater than 5 times / minute, the fatigue driving level is determined to be Level 3, severe fatigue.

[0148] Once the driver fatigue level is determined, the intensity and frequency of the warning signal can be adjusted based on the driver's fatigue level and the driving environment to avoid alarm fatigue.

[0149] S602. Select the corresponding warning strategy from the preset warning strategy library according to the fatigue driving level, and perform warning processing according to the corresponding warning strategy to generate a warning signal.

[0150] In this embodiment, when the driver fatigue level is determined to be Level 1 (mild fatigue), the warning strategy is to provide a text prompt through the instrument panel; when the driver fatigue level is determined to be Level 2 (moderate fatigue), the warning strategy is to provide a short sound prompt through a buzzer and to provide a vibration through the seat; when the driver fatigue level is determined to be Level 3 (severe fatigue), the warning strategy is to force deceleration, activate the turn signal, and pull over to the right side of the road when it is safe to do so.

[0151] Determining the level of driver fatigue and implementing corresponding early warning strategies can improve driver response speed and reduce information redundancy.

[0152] Here is a real-world example:

[0153] After driving continuously for 3 hours at night, with insufficient lighting inside the vehicle and the driver wearing a mask, visual characteristic data becomes invalid. The IMU detects a pitch angle greater than 20°, PERCLOS greater than 30%, HRV less than 40ms, and respiratory rate greater than 28 breaths / minute; the steering wheel angular velocity abrupt change frequency is greater than 4 times / minute. At this point, the system will reduce the weighting coefficient of visual fatigue similarity to 0.2, increase the weighting coefficient of physiological fatigue similarity to 0.5, and set the weighting coefficient of driving behavior fatigue similarity to 0.3. The overall fatigue probability value is calculated as follows: Where P is the overall fatigue probability value, m is the visual fatigue similarity, n is the physiological fatigue similarity, and t is the driving behavior fatigue similarity. If the calculated overall fatigue probability value P is 0.65, the fatigue driving level is determined to be Level 2 moderate fatigue, and a short audible warning is given via a buzzer and the seat is vibrated.

[0154] This embodiment provides a fatigue driving warning and processing method applied to commercial vehicles, and further explains the above embodiment. In this embodiment, after S602, it further includes:

[0155] S701, Obtain the warning duration after the warning signal is generated.

[0156] In this embodiment, the duration of the early warning signal after it is generated can be obtained through a real-time monitoring platform.

[0157] S702. When the warning duration is detected to be greater than the intelligent driving takeover duration threshold, acquire the environmental data around the vehicle.

[0158] In this embodiment, if the detected warning duration exceeds the intelligent driving takeover duration threshold, it is determined that the driver has not made a significant response after the initial warning. At this time, the system will enter the intelligent driving takeover mode and automatically execute intervention measures. Data on the vehicle's surrounding environment is acquired through the collaborative work of sensors such as cameras, millimeter-wave radar, and lidar.

[0159] S703: Based on the surrounding environmental data, control the vehicle to slow down to the right until the vehicle stops in a safe area.

[0160] In this embodiment, when the vehicle is determined to be in a safe state based on the surrounding environmental data, the vehicle is controlled to decelerate to the right until it stops in a safe area. This process involves a physical process conversion, from a warning signal to actual driver assistance operation, with the computer program executing intervention measures through the vehicle control interface.

[0161] Optionally, adjusting the cabin lighting, such as increasing the brightness or changing the color of the lights, can help keep the driver alert. Appropriate lighting adjustments can effectively reduce driver fatigue; for example, green light can improve driving focus in tunnels.

[0162] Optionally, driver fatigue can be alleviated by automatically playing music.

[0163] Optionally, the system can also notify the fleet manager via the vehicle communication system to request remote assistance or dispatch a substitute driver.

[0164] When the driver does not respond significantly after an initial warning, the system can actively intervene, which can effectively improve driving safety.

[0165] This embodiment provides a fatigue driving warning processing method applied to commercial vehicles, further explaining the above embodiment. In this embodiment, step S104: performing warning processing and generating a warning signal includes:

[0166] S801, Perform warning processing, generate warning data and send it to the vehicle's instrument panel to cause the instrument panel to generate a visual warning signal; and / or generate warning data and send it to a sound generator to cause the sound generator to generate an audible warning signal; and / or generate warning data and send it to the seat to cause the seat to generate a behavioral perception signal.

[0167] In this embodiment, visual warning signals, such as "Keep your attention" with a warning color, can be generated through the vehicle's infotainment display or head-up display; audible warning signals can be generated through a buzzer, car audio system, or car horn; behavioral perception signals can be generated by vibrating or swaying the driver's seat; and behavioral perception signals can also be generated through irregular vibrations of the steering wheel and the vibration of the buzzer attached to the seat belt.

[0168] Optionally, the audible warning signal can be changed to the driver's preferred voice prompt or other sound effects to reduce driver annoyance.

[0169] Optionally, visual and audible warning signals can be combined with augmented reality (AR) technology to overlay navigation and fatigue warning information on the HUD screen, thereby optimizing the user experience.

[0170] When a driver is detected to be fatigued, a warning signal is sent to the driver through multiple sensory channels such as vision, hearing, and touch. Compared with a single warning method, the multi-sensory interaction design can effectively improve the driver's response speed to the warning signal.

[0171] This embodiment provides a fatigue driving warning and processing method applied to commercial vehicles, and further explains the above embodiment. In this embodiment, S102 includes:

[0172] S901. Perform noise reduction processing on vehicle driving behavior data and user's own data to obtain noise-reduced data to be processed.

[0173] In this embodiment, for vehicle driving behavior data and user data, noise reduction processing refers to removing frames or images with high noise levels, and the remaining data is the data to be processed after noise reduction.

[0174] S902. Normalize the data to be processed to obtain normalized data to be processed.

[0175] In this embodiment, normalization processing refers to unifying the format of the data to be processed for denoising, obtaining data with the same size and pixel format, and using it as the normalized data to be processed.

[0176] S903. Perform feature extraction processing on the normalized data to be processed to obtain the data to be matched.

[0177] In this embodiment, feature extraction processing refers to extracting the feature data needed for subsequent fatigue detection from the normalized data to be processed, so as to perform feature matching similarity in the future.

[0178] The collected raw data undergoes noise reduction, normalization, and feature extraction. In this step, the computer program performs data cleaning and feature selection, removing invalid or redundant information and retaining key features through algorithms, which facilitates subsequent data analysis and processing.

[0179] When a user's driving state is detected as fatigued, a fatigue warning event record is generated and sent to the fleet management backend computer. This fatigue warning event record includes data to be matched, physiological fatigue detection results, visual fatigue detection results, driving behavior fatigue detection results, physiological matching results, visual matching results, behavioral matching results, and vehicle and user data after warning processing. The fleet management backend computer can be a host computer, server, laptop, or other hardware device with a fleet management system deployed on it. This process provides fleet managers with data analysis to facilitate driver training and system optimization.

[0180] The above process involves the physical transformation of data storage and post-processing. The computer program stores the collected data and early warning results in a database and provides data analysis tools. The system can integrate more data analysis tools, such as machine learning models, to predict fatigue trends and provide personalized fatigue management recommendations.

[0181] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device is intended for use with various electronic devices capable of executing fatigue driving warning processing methods applied to commercial vehicles, such as microcomputers, single-chip microcomputers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0182] like Figure 8 As shown, the electronic device includes at least one processor 810 and a memory 820. The electronic device also includes a communication component 830. The processor 810, memory 820, and communication component 830 are connected via a bus 840.

[0183] In the specific implementation process, at least one processor 810 executes computer execution instructions stored in memory 820, causing at least one processor 810 to execute the fatigue driving warning processing method applied to commercial vehicles as executed on the electronic device side as described above.

[0184] The specific implementation process of processor 810 can be found in the above embodiment of the fatigue driving warning processing method applied to commercial vehicles. Its implementation principle and technical effect are similar, and will not be repeated here.

[0185] In the above embodiments, it should be understood that the processor 810 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor 810 can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0186] The memory 820 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage.

[0187] Bus 840 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Bus 840 can be divided into an address bus, a data bus, and a control bus. For ease of illustration, the bus 840 in the accompanying drawings of this application is not limited to only one bus or one type of bus.

[0188] The above description addresses the functions implemented by electronic devices and main control devices, and introduces the solutions provided in the embodiments of this application. It is understood that, in order to achieve the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments disclosed in this application, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of this application.

[0189] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-mentioned fatigue driving warning processing method for commercial vehicles.

[0190] The aforementioned computer-readable storage media can be implemented by any type of volatile, non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0191] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. The readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0192] The memory 820 is the non-transitory computer-readable storage medium provided by this invention. The non-transitory computer-readable storage medium of this invention stores computer information, enabling the computer to execute the fatigue driving warning processing method for commercial vehicles provided by this invention.

[0193] The memory 820, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 810 executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 820, thereby implementing the fatigue driving warning processing method for commercial vehicles in the above method embodiments.

[0194] In addition, this embodiment also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the fatigue driving warning processing method for commercial vehicles described above.

[0195] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0196] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0197] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0198] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0199] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0200] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0201] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0202] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A fatigue driving warning and processing system for commercial vehicles, characterized in that, include: Data acquisition device, data processing device, fatigue detection device, and early warning processing device; The data acquisition device, the data processing device, the fatigue detection device, and the early warning processing device are all connected to each other, and the data acquisition device, the data processing device, the fatigue detection device, and the early warning processing device are connected to the vehicle control unit of the commercial vehicle via the vehicle bus. The data acquisition device is used to acquire vehicle driving behavior data and user data. The data processing device is used to preprocess the vehicle driving behavior data and the user's own data to obtain data to be matched. The fatigue detection device is used to perform fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions and pre-trained deep learning model to determine the user's driving state. The warning processing device is used to perform warning processing and generate a warning signal when it detects that the user's driving state is fatigued.

2. A fatigue driving warning and processing method applied to commercial vehicles, characterized in that, The method is applied to a vehicle control unit, wherein the vehicle control unit is data-connected to the fatigue driving warning and processing system for commercial vehicles as described in claim 1, and the method includes: Acquire vehicle driving behavior data and user personal data; The vehicle driving behavior data and the user's own data are preprocessed to obtain the data to be matched. Fatigue detection is performed based on the data to be matched, pre-stored benchmark data, preset matching conditions, and pre-trained deep learning model to determine the user's driving status. When the user's driving state is detected to be fatigued, an early warning process is executed, and an early warning signal is generated.

3. The fatigue driving warning and processing method for commercial vehicles according to claim 2, characterized in that, The data to be matched includes physiological signal feature data, visual feature data, and driving behavior feature data, and the pre-stored benchmark data includes physiological feature thresholds, visual feature thresholds, and driving behavior feature thresholds; The step of performing fatigue detection processing based on the data to be matched, pre-stored benchmark data, preset matching conditions, and a pre-trained deep learning model to determine the user's driving state includes: The physiological signal feature data and the physiological feature threshold are input into a pre-trained deep learning model for physiological feature matching processing to obtain physiological fatigue detection results. The visual feature data and the visual feature threshold are input into a pre-trained deep learning model for visual feature matching processing to obtain visual fatigue detection results. The driving behavior feature data and the driving behavior feature threshold are input into a pre-trained deep learning model for driving behavior feature matching processing to obtain driving behavior fatigue detection results. The user's driving status is determined based on the physiological fatigue detection results, the visual fatigue detection results, the driving behavior fatigue detection results, and preset matching conditions.

4. The fatigue driving warning and processing method for commercial vehicles according to claim 3, characterized in that, The physiological fatigue detection results include physiological fatigue similarity, the visual fatigue detection results include visual fatigue similarity, the driving behavior fatigue detection results include driving behavior fatigue similarity, and the preset matching conditions include physiological matching threshold, visual matching threshold, and behavioral matching threshold. The step of determining the user's driving state based on the physiological fatigue detection results, the visual fatigue detection results, the driving behavior fatigue detection results, and preset matching conditions includes: The physiological fatigue similarity and the physiological matching threshold are compared to determine the physiological matching result. The visual fatigue similarity and the visual matching threshold are compared to determine the visual matching result. The driving behavior fatigue similarity and the behavior matching threshold are compared to determine the behavior matching result. When the physiological matching result, the visual matching result, and the behavioral matching result all meet the matching success conditions, the user's driving state is determined to be fatigued.

5. The fatigue driving warning and processing method for commercial vehicles according to claim 4, characterized in that, Also includes: A comprehensive fatigue probability value is obtained by performing a comprehensive probability calculation based on the physiological matching result, the visual matching result, and the behavioral matching result. When the overall fatigue probability value is detected to be greater than a preset probability threshold, and at least two of the physiological matching results, the visual matching results, and the behavioral matching results meet the matching success conditions, the user's driving state is determined to be fatigued.

6. The fatigue driving warning and processing method for commercial vehicles according to claim 4, characterized in that, Also includes: When the visual feature data is detected to meet the dynamic adjustment conditions, the dynamic weight coefficient ratio is obtained; The dynamic fatigue probability value is determined based on the dynamic weighting coefficient ratio, the physiological matching result, the visual matching result, and the behavioral matching result. When the dynamic fatigue probability value is detected to be greater than the preset probability threshold, and both the physiological matching result and the behavioral matching result meet the matching success condition, the user's driving state is determined to be fatigued.

7. The fatigue driving warning and processing method for commercial vehicles according to claim 5, characterized in that, Also includes: When the user's driving state is detected to be fatigued, the fatigue driving level is determined based on the comprehensive fatigue probability value, the physiological matching result, the visual matching result, and the behavioral matching result. Based on the fatigue driving level, a corresponding warning strategy is selected from the preset warning strategy library to perform warning processing and generate a warning signal.

8. The fatigue driving warning and processing method for commercial vehicles according to claim 7, characterized in that, Also includes: Obtain the duration of the warning signal after it is generated; When the warning duration is detected to be greater than the intelligent driving takeover duration threshold, data on the vehicle's surrounding environment is acquired. Based on the environmental data surrounding the vehicle, the vehicle is controlled to decelerate to the right until it stops in a safe area.

9. The fatigue driving warning and processing method for commercial vehicles according to claim 2, characterized in that, The execution of the early warning process, generating an early warning signal, includes: Perform warning processing, generate warning data and send it to the vehicle's instrument panel to cause the instrument panel to generate a visual warning signal; and / or generate warning data and send it to a sound generator to cause the sound generator to generate an audible warning signal; and / or generate warning data and send it to the seat to cause the seat to generate a behavioral perception signal.

10. The fatigue driving warning and processing method for commercial vehicles according to claim 2, characterized in that, The process of preprocessing the vehicle driving behavior data and the user's own data to obtain the data to be matched includes: The vehicle driving behavior data and the user's own data are subjected to noise reduction processing to obtain denoised data to be processed. The denoised data to be processed is normalized to obtain normalized data to be processed. Feature extraction is performed on the normalized data to be processed to obtain the data to be matched.

Images