Method and system for automatically judging driving state based on multi-target recognition

Abstract

The invention provides a driving state automatic discrimination method and system based on multi-target identification. The method belongs to the cross technical field of intelligent driving assistance, man-machine interaction and vehicle-mounted health monitoring. The method comprises the following steps: synchronously acquiring millimeter wave radar physiological signals, multi-target visual behavior characteristic data and vehicle operation data; according to the collected millimeter wave radar physiological signals, the multi-target visual behavior characteristic data and the vehicle operation data, constructing a physiological instability, behavior abnormity and control degradation three-level cascade discrimination model; by synchronously collecting millimeter wave radar physiological signals, multi-target visual behavior characteristics and vehicle operation data and constructing a physiological instability, behavior abnormity and control degradation three-level cascade discrimination model, the accuracy and timeliness of driver state discrimination are significantly improved.

Description

Automatic Driving State Judgment Method and System Based on Multi-Target Recognition Technical Field

[0001] This invention proposes a method and system for automatic driving state determination based on multi-target recognition, belonging to the interdisciplinary fields of intelligent driving assistance, human-computer interaction and vehicle health monitoring. Background Technology

[0002] In the field of existing driver condition monitoring technology, mainstream driver monitoring systems (DMS) generally have significant shortcomings.

[0003] Current technologies often rely on single-modal data, such as detecting behaviors like closing the driver's eyes or yawning using only visible light or infrared cameras, but they cannot effectively detect internal physiological crises in the driver, such as sudden changes in heart rate caused by myocardial infarction or hypoglycemia.

[0004] Meanwhile, existing systems often monitor the driver's face, hands, and body posture independently, lacking multi-target collaborative criteria, making it difficult to accurately identify complex abnormal states such as "head drooping + hands off the rudder + breathing pause".

[0005] In addition, most systems ignore vehicle operation data when determining the driver's state, which leads to biases in the risk assessment of the driver's closed-eye behavior at different vehicle speeds.

[0006] More importantly, existing technologies mostly remain at the level of audible and visual alarms, lacking a graded linkage mechanism with the vehicle's braking system, and thus cannot control the vehicle in a timely and effective manner when the driver is incapacitated.

[0007] Therefore, developing an automatic driving state discrimination method based on multi-target recognition to achieve coupling of physiological, behavioral and vehicle domains, dynamic risk stratification and hierarchical control closed loop has become an urgent need to improve driving safety. Summary of the Invention

[0008] This invention provides a method and system for automatic driving state determination based on multi-target recognition, in order to solve the problems mentioned in the background art above:

[0009] The present invention proposes an automatic driving state discrimination method based on multi-target recognition, the method comprising:

[0010] S1. Simultaneously acquire millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data;

[0011] S2. Based on the collected millimeter-wave radar physiological signals, multi-target visual behavior feature data, and vehicle operation data, a three-level cascaded discrimination model for physiological instability, behavioral abnormality, and control degradation is constructed.

[0012] S3. Based on a three-level cascaded discrimination model, the driver's status is dynamically assessed within 4 to 7 seconds to generate dynamic risk level data.

[0013] S4. Drive the tiered response strategy based on dynamic risk level data; and synchronously upload alarm information to the management backend;

[0014] S5. Based on the execution results of the hierarchical response strategy, continuously optimize the parameters of the three-level cascaded discrimination model to form a closed-loop control system for perception, discrimination, decision-making and execution; the closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism.

[0015] The present invention proposes a system for implementing the above-described method for automatic driving state determination based on multi-target recognition, the system comprising:

[0016] Data acquisition module: synchronously acquires millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data;

[0017] Model building module: Based on the collected millimeter-wave radar physiological signals, multi-target visual behavior feature data and vehicle operation data, a three-level cascaded discrimination model for physiological instability, behavioral abnormality and control degradation is constructed;

[0018] Risk assessment module: Based on a three-level cascaded discriminant model, it dynamically assesses the driver's status risk level within 4 to 7 seconds and generates dynamic risk level data;

[0019] Strategy execution module: Drives tiered response strategies based on dynamic risk level data; and synchronously uploads alarm information to the management backend;

[0020] Real-time feedback module: Based on the execution results of the hierarchical response strategy, continuously optimize the parameters of the three-level cascaded discrimination model to form a closed-loop control system for perception, discrimination, decision-making and execution; the closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism.

[0021] The beneficial effects of this invention are as follows: By simultaneously collecting millimeter-wave radar physiological signals, multi-target visual behavioral characteristics, and vehicle operation data, and constructing a three-level cascaded discrimination model for physiological instability, behavioral abnormalities, and control degradation, the accuracy and timeliness of driver status judgment are significantly improved. In complex and ever-changing driving environments, this method can reduce the risk of misjudgment and missed judgment caused by the limitations of single-modal data, enhancing the reliability of driving safety warnings. At the same time, the implementation of dynamic risk stratification and graded control strategies reduces unnecessary audio-visual interference, avoiding frequent disturbances to the driver during normal driving. More importantly, this method can quickly initiate smooth braking when the driver is incapacitated, effectively preventing vehicle loss of control accidents and preventing potential serious personal injury and property damage. It can accurately perceive the driver's physiological and behavioral abnormalities and control the vehicle in a timely and effective manner, providing a comprehensive and multi-level driver incapacity protection solution for commercial vehicles and passenger vehicles. Attached Figure Description

[0022] Figure 1 is a flowchart of the method steps described in this invention;

[0023] Figure 2 is a system module diagram of the present invention. Detailed Implementation

[0024] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0025] An embodiment of the present invention, as shown in FIG1, provides an automatic driving state determination method based on multi-target recognition, the method comprising:

[0026] S1. Simultaneously collect millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; among which, millimeter-wave radar physiological signals include the driver's respiratory rate and heart rate data, multi-target visual behavioral feature data covers the driver's facial, hand, and body posture features, and vehicle operation data includes vehicle speed, throttle opening, braking status, steering angle, and steering wheel angle data.

[0027] S2. Based on the collected millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data, a three-level cascaded discrimination model for physiological instability, behavioral abnormality, and control degradation is constructed. Among them, the physiological instability criterion is based on abnormal changes in respiratory rate and heart rate, the behavioral abnormality criterion integrates facial drooping, hand off the rudder, and abnormal body posture, and the control degradation criterion combines abnormal control patterns in vehicle operation data.

[0028] S3. Based on the three-level cascaded discrimination model, the driver's status is dynamically assessed within 4 to 7 seconds, and dynamic risk level data is generated. The dynamic risk level is divided into three levels: low risk, medium risk and high risk, which correspond to the driver's normal status, potential abnormality and severe disability, respectively.

[0029] S4. Based on dynamic risk level data, drive a graded response strategy; in low-risk conditions, only trigger an audible and visual warning to alert the driver; in medium-risk conditions, initiate speed limit control to reduce vehicle speed; in high-risk conditions, perform smooth braking to safely stop the vehicle and simultaneously upload alarm information to the management backend.

[0030] S5. Based on the execution results of the hierarchical response strategy, continuously optimize the parameters of the three-level cascaded discrimination model to form a closed-loop control system for perception, discrimination, decision-making and execution. The closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism to improve the accuracy and response efficiency of automatic driving state discrimination.

[0031] The working principle and effects of the above technical solution are as follows: By using the above-mentioned automatic driving state discrimination method, the accuracy and response efficiency of driving state recognition are greatly improved, and the dynamically optimized discrimination model makes anomaly detection more accurate; it effectively reduces the risk of vehicle loss of control due to driver fatigue, illness or distraction, especially reducing the major hazards that may be caused by loss of control of commercial vehicles; it significantly enhances the safety of drivers and passengers, filling the gap of traditional braking technology that only focuses on external obstacles; it avoids the dangerous situation of no intervention after driver incapacitation, and avoids the chain of damage caused by vehicle loss of control; it can accurately identify abnormalities in multiple dimensions such as physiology, behavior and operation, and can flexibly adapt to different risk scenarios through graded response, without interfering with the normal driving rhythm, and can deal with emergencies in a timely manner, making travel safer and more reliable.

[0032] In one embodiment of the present invention, S1 includes:

[0033] S11. Activate the in-vehicle preset camera and millimeter-wave radar to simultaneously collect multi-target visual behavioral characteristic data of the driver and physiological signals of the millimeter-wave radar.

[0034] S12. Collect data on vehicle speed, throttle opening, braking status, steering angle and steering wheel angle during vehicle operation, and integrate them with the above-mentioned collected data to generate a multi-dimensional raw dataset.

[0035] S13. Denoise the multi-dimensional raw dataset to remove invalid data caused by environmental interference and equipment fluctuations.

[0036] S14. Standardize and organize the denoised dataset to unify the format and dimensions of different types of data.

[0037] S15. Perform time-series alignment processing on the standardized dataset to ensure that the timelines of visual behavioral feature data, physiological signals and vehicle operation data are consistent, and generate a time-series synchronized dataset.

[0038] The working principle and effects of the above technical solution are as follows: Through the above data acquisition and processing scheme, the completeness and accuracy of driving-related data are greatly improved. The synchronous acquisition of multi-source data allows for a more comprehensive capture of driver status and vehicle operation. It effectively reduces the impact of invalid data caused by environmental interference and equipment fluctuations, reducing the probability of misjudgment by subsequent discrimination models. It enhances the consistency and usability of different types of data, and time sequence alignment allows for the effective correlation of multi-dimensional information. It avoids analysis bias caused by chaotic data formats and asynchronous time, and avoids invalid data interfering with the extraction of core features. It can comprehensively cover key data on driver physiology, behavior, and vehicle operation, and optimize data quality through layer-by-layer processing, providing a solid and reliable foundation for subsequent driving status discrimination, making the perception link of the entire protection system more stable and efficient.

[0039] In one embodiment of the present invention, step S12 includes:

[0040] Start the vehicle operation data acquisition module to collect basic operating parameters such as vehicle speed and throttle opening, and generate a basic operation dataset.

[0041] Continuously collect data related to braking status, steering angle, and steering wheel angle to generate a braking and steering dataset; supplement the collection of vehicle gear information to improve the dimensions of the operating data and generate a gear dataset.

[0042] The basic operation dataset, braking and steering dataset, and gear dataset are summarized and integrated to generate a complete vehicle operation dataset;

[0043] The complete vehicle operation dataset is merged with the initial visual and physiological datasets generated by S11 to generate a multi-dimensional original dataset.

[0044] The working principle and effects of the above technical solution are as follows: Through the aforementioned vehicle operation data collection and integration scheme, the completeness and comprehensiveness of vehicle operation data are significantly improved, covering everything from basic parameters to operational details and gear information without omission; effectively reducing analytical bias caused by missing data dimensions and minimizing information blind spots in subsequent driving status judgment; enhancing the correlation between vehicle operation data and driver's visual and physiological data, allowing multi-source data to form complementary support; avoiding the risk of misjudgment caused by insufficient single data dimensions and preventing the omission of key operational information from affecting abnormal state identification; accurately capturing the core operating state of the vehicle while driving, and smoothly connecting with previously collected driver-related data, laying a solid foundation for multi-dimensional raw datasets, making subsequent driving status analysis more aligned with actual scenarios, and improving the perception accuracy of the entire protection system.

[0045] In one embodiment of the present invention, S15 includes:

[0046] Extract the original timestamps from visual behavior feature data, physiological signals, and vehicle operation data in the standardized dataset to generate a multi-source independent timestamp sequence;

[0047] By comparing the temporal resolution differences of multi-source independent timestamp sequences, the highest precision timestamp is selected as the benchmark reference to generate a unified time benchmark framework.

[0048] Based on a unified time reference framework, interpolation calibration is performed on the time nodes of visual behavioral feature data to fill in missing time segment data and generate a calibrated visual feature dataset.

[0049] Based on a unified time reference framework, physiological signals are corrected for time offset to eliminate timing deviations caused by equipment acquisition delays and generate corrected physiological signal datasets.

[0050] By referring to a unified time reference framework, the vehicle operation data is aligned with the time scale, the time record nodes of each parameter are synchronized, and an aligned vehicle operation dataset is generated.

[0051] Verify the time synchronization error of the three calibrated datasets, remove data with deviations exceeding the threshold, and generate a time consistency verification dataset; integrate the time consistency verification datasets to form a time-series synchronization dataset in which the time axes of data in each dimension are fully synchronized.

[0052] The working principle and effects of the above technical solution are as follows: By using the above time-series alignment processing scheme, the time synchronization accuracy of multi-source data is significantly improved, making the time axis of visual features, physiological signals and vehicle operation data highly consistent; effectively reducing the time-series deviation caused by equipment acquisition delay and resolution differences, reducing misjudgments caused by data asynchrony in subsequent discrimination models; enhancing the correlation and reliability of data in various dimensions, and completing missing segments to make the data more complete; avoiding feature mismatch caused by time node misalignment, and avoiding interference from delayed or missing data with accurate judgment of driving status; it can standardize the time standard of all data through a unified benchmark framework, and eliminate abnormal data through error verification, making the time-series synchronized dataset more in line with actual driving scenarios, providing accurate data support for subsequent risk assessment and response, and making the discrimination link of the entire protection system more reliable.

[0053] In one embodiment of the present invention, S2 includes:

[0054] S21. Based on the time-series synchronous dataset, it is divided into three independent criteria dimensions: physiological instability, behavioral abnormality, and manipulation degradation.

[0055] S22. Establish the basic framework of the cascaded model and clarify the judgment process and inter-connection logic of the three dimensions;

[0056] S23. Abnormal changes in respiratory rate and heart rate are included in the criteria for physiological instability; drooping face, hands off the rudder, and abnormal body posture are included in the criteria for behavioral abnormality; and abnormal control modes in vehicle operation data are included in the criteria for control degradation.

[0057] S24. Train the initially constructed model using multi-scenario driving samples, and calibrate the logical thresholds and dimensional correlation weights of each criterion.

[0058] S25. After completing the training and verification, a mature three-level cascaded discrimination model that can accurately determine the driving state is generated.

[0059] The working principle and effects of the above technical solution are as follows: By constructing the three-level cascaded discrimination model, the accuracy and comprehensiveness of driving state discrimination are greatly improved. The three independent criterion dimensions cover the core levels of physiology, behavior, and control, ensuring comprehensive anomaly detection. It effectively reduces the probability of misjudgment or missed judgment caused by a single criterion and reduces the recognition error caused by missing dimensions or parameter deviations. It enhances the model's adaptability to different driving scenarios, and multi-scenario sample training makes the criterion thresholds and weights more in line with actual needs. It avoids missing key abnormal signals due to one-sided discrimination logic and avoids inaccurate protection response caused by unreasonable parameters. It can accurately identify various situations such as driver physiological instability, abnormal behavior, and control degradation, and can make the model more reliable through layer-by-layer optimization, laying a solid foundation for subsequent dynamic risk assessment. This makes the discrimination process of the entire protection system more in line with real driving scenarios and improves the effectiveness of anomaly identification.

[0060] In one embodiment of the present invention, S24 includes:

[0061] Collect driving samples from multiple scenarios under different road conditions and driving states. These driving samples cover scenarios such as normal driving, physiological abnormalities, behavioral abnormalities, and control abnormalities, and generate a multi-scenario training sample set. Classify and label the multi-scenario training sample set according to three criteria: physiological instability, behavioral abnormalities, and control degradation, and generate a dimension-classified labeled sample set.

[0062] Input the dimension classification labeled sample set into the initially constructed three-level cascaded discriminant model, start multiple rounds of iterative training, and generate the intermediate result set of model training;

[0063] Analyze the intermediate training result set, compare the discrimination accuracy of each criterion dimension, adjust the logical threshold of physiological instability behavior abnormal control degradation, and generate threshold calibration parameters;

[0064] Based on the threshold calibration parameters, the correlation weights among the three criteria dimensions are optimized, the discrimination priorities of each dimension are balanced, and weight optimization parameters are generated.

[0065] Import the threshold calibration parameters and weight optimization parameters into the initial model, update the internal parameter configuration of the model, and generate an intermediate model with calibrated parameters.

[0066] The working principle and effects of the above technical solution are as follows: Through the above model training optimization scheme, the discrimination accuracy and scenario adaptability of the three-level cascaded discrimination model are significantly improved. Multi-scenario samples comprehensively cover various driving states, making the training more in line with actual application scenarios; the bias of single-dimensional criteria is effectively reduced, and false positives and false negatives caused by unreasonable thresholds or unbalanced weights are reduced; the model's sensitivity to different abnormal types of physiological, behavioral, and control abnormalities is enhanced, making the discrimination logic of each dimension more coordinated; the model avoids insufficient recognition of special road conditions or abnormal scenarios, and avoids protection lag caused by rigid parameter configuration; it can accurately capture abnormal driving signals in different scenarios, and balance the priority of each criterion through threshold and weight optimization, making the model discrimination more reliable, providing accurate support for subsequent risk assessment, and improving the abnormal identification efficiency of the entire protection system.

[0067] In one embodiment of the present invention, S3 includes:

[0068] S31. Input the time-series synchronization dataset into the mature three-level cascaded discriminant model and start the dynamic risk assessment process;

[0069] S32. Within a set time window of 4 to 7 seconds, continuously receive real-time updated timing synchronization data and perform multiple rounds of continuous discrimination;

[0070] S33. Weighted fusion of the results from multiple rounds of discrimination, and comprehensive consideration of the confidence level of the discrimination results in each round;

[0071] S34. Based on the preset standards, match the fusion results with the three levels of low risk, medium risk, and high risk.

[0072] S35. Output the matching results and generate dynamic risk level data, which corresponds to the driver's normal status, potential abnormality, and severe disability.

[0073] The working principle and effects of the above technical solution are as follows: Through the dynamic risk assessment scheme, the timeliness and accuracy of driving state risk identification are significantly improved. The 4-7 second time window combined with multiple rounds of continuous judgment ensures timely risk capture; it effectively reduces the randomness of single judgment results, minimizing inadequate protection due to misjudgment or omission; it enhances the accuracy of risk level classification, with weighted fusion taking into account the confidence level of each round of judgment, making the correspondence between normal, potential abnormality, and severe disability more closely match the actual state; it avoids erroneous responses caused by single judgment deviations, and avoids over- or under-protection due to risk level mismatch; it can quickly follow changes in driver state, and ensure the reliability of assessment results through multi-round fusion and level matching, providing accurate basis for subsequent graded responses, making the decision-making process of the entire protection system more targeted, and enhancing the safety protection of drivers and passengers.

[0074] In one embodiment of the present invention, S34 includes:

[0075] Extract driving status criteria corresponding to low, medium and high risks from the preset standards, clarify the quantitative characteristics of normal, potential abnormal and severe disability, and generate a level matching benchmark set;

[0076] Analyze the discrimination results after multiple rounds of fusion, extract the comprehensive abnormality index of physiological signals, behavioral characteristics and manipulation data, and generate a set of key indicators for the fusion results;

[0077] The key indicator set is compared one by one with the level matching benchmark set, and the characteristic threshold range of each risk level is used to generate the indicator comparison results. Based on the comparison results, the risk level range to which the fusion result belongs is determined, and the preliminary level matching result is generated.

[0078] Verify the consistency between the preliminary matching results and the data in each dimension, eliminate false matches caused by anomalies in a single dimension, and generate the final level matching results.

[0079] The working principle and effects of the above technical solution are as follows: The risk level matching scheme significantly improves the accuracy of determining the risk level of driving conditions. Clear quantitative features make the matching standards more explicit, and the extraction of comprehensive abnormal indicators makes the judgment dimensions more comprehensive. It effectively reduces the probability of mismatches caused by single-dimensional anomalies, and reduces protection deviations caused by mismatches between low, medium, and high risk levels. It enhances the consistency between the matching results and data from physiological, behavioral, and operational dimensions, ensuring that the risk level highly matches the actual driving condition. It avoids over-response or insufficient protection due to one-sided judgments, and prevents incorrect level matching from affecting driver and passenger safety. It can accurately capture multi-dimensional comprehensive abnormal signals and solidify the reliability of the results through one-by-one comparison and consistency verification, making the risk level classification more in line with the actual situation, providing precise guidance for subsequent graded responses, and further improving the safety protection effectiveness of the entire protection system.

[0080] In one embodiment of the present invention, step S4 includes:

[0081] S41. Receive dynamic risk level data and retrieve the system's preset hierarchical response rule base;

[0082] S42. If the risk level is determined to be low, the sound, light, and vibration warning modules are triggered to issue a status reminder to the driver.

[0083] S43. If the risk level is determined to be medium, speed limit control will be activated while the warning is being continuously issued to limit the throttle output and slow down the vehicle.

[0084] S44. If the risk is deemed high, activate the braking mechanism and bring the vehicle to a smooth stop by using the original EBS, an added valve, or a lever motor to pull the brake pedal.

[0085] S45. In high-risk situations, the alarm upload function is triggered synchronously to push abnormal information to the management backend, while recording the entire response process data and generating a response execution dataset.

[0086] The working principle and effects of the above technical solution are as follows: Through the tiered response scheme, the pertinence and timeliness of risk response are significantly improved, with differentiated handling for low, medium, and high risks, ensuring an appropriate response for each state; effectively reducing over-intervention or under-response, minimizing chain reactions caused by vehicle loss of control during driver anomalies; enhancing the safety of drivers and passengers, ensuring smooth braking in high-risk situations to avoid secondary risks from sudden braking, and enabling rapid intervention by management personnel through alarm transmission; reducing the possibility of accident escalation, especially reducing significant losses caused by loss of control of large vehicles such as commercial vehicles; avoiding the rigidity and limitations of a single response mode, and preventing dangerous situations where no one is available to handle high-risk situations. It can both alert drivers to make timely adjustments through audible, visual, and vibration warnings, and proactively control the vehicle to protect safety in emergencies; it can both ensure the safety of occupants and simultaneously transmit abnormal information, making the execution of the protection system more efficient and reliable, comprehensively strengthening the safety defense line for travel.

[0087] In one embodiment of the present invention, step S5 includes:

[0088] S51. Collect response execution datasets and real-time driving status feedback data, and integrate them to form a closed-loop feedback data set;

[0089] S52. Perform difference analysis on the closed-loop feedback data set to locate the deviation nodes and the lagging links of the response strategy in the model;

[0090] S53. Based on the analysis results, dynamically adjust the discrimination threshold and the correlation weight of each dimension of the three-level cascaded discrimination model;

[0091] S54. Modify the triggering conditions and execution parameters of the graded response strategy, and optimize the early warning timing and braking force;

[0092] S55. Import the adjusted parameters and strategies into the system, complete the iterative update of the model and strategies, and form a continuously optimized closed-loop control system.

[0093] The working principle and effects of the above technical solution are as follows: Through the closed-loop optimization scheme, the long-term discrimination accuracy of the three-level cascaded discrimination model and the scenario adaptability of the hierarchical response strategy are significantly improved, allowing the system to dynamically adjust according to actual use. This effectively reduces the bias caused by model parameter aging and minimizes protection vulnerabilities caused by lagging or improperly adapted response strategies; enhances the system's flexibility in responding to different driving scenarios and driver state changes, ensuring that each optimization aligns with real-world needs; avoids model rigidity and failure after long-term use, and prevents misjudgments or insufficient protection due to outdated response parameters; accurately locates discrimination bias and response problems through feedback data, and allows for targeted adjustment of thresholds, weights, and execution parameters; makes warning timing and braking force more reasonable, and enables the entire protection system to continuously evolve, becoming more reliable with use, providing long-term and stable protection for driver and passenger safety.

[0094] In one embodiment of the present invention, S53 includes:

[0095] Extract dimensional bias data and weight imbalance information from the closed-loop feedback difference analysis results to generate a set of core criteria for model adjustment.

[0096] Based on the criteria for physiological instability, the corresponding discrimination thresholds are adjusted by referring to the discriminant data of abnormal respiratory and heart rate in the core criteria, and physiological dimension threshold adjustment results are generated.

[0097] Based on the behavioral feature identification deviation information of the core criteria, the thresholds for facial, hand, and body posture discrimination of behavioral anomaly judgment are optimized, and behavioral dimension threshold adjustment results are generated.

[0098] By combining the core data on vehicle handling anomaly discrimination deviation, the operating parameter discrimination threshold of the handling degradation criterion is corrected, and the control dimension threshold adjustment result is generated.

[0099] Based on the degree of deviation in each dimension and the actual priority requirements for judgment, the correlation weight allocation ratio of physiological instability, behavioral abnormality and control degradation is optimized to generate dimension weight optimization parameters.

[0100] Integrate the three-dimensional threshold adjustment results with the weight optimization parameters to form a core parameter adjustment package for the model, thus completing the basic configuration for dynamic updates of the three-level cascaded discrimination model.

[0101] The working principle and effects of the above technical solution are as follows: By dynamically adjusting the model parameters, the accuracy of the three-level cascaded discrimination model in each dimension is significantly improved, and the threshold deviations of physiological, behavioral, and operational aspects are specifically corrected, making the anomaly identification of each dimension more realistic; the probability of misjudgment caused by deviations in a single dimension is effectively reduced, and the disorder of discrimination priority caused by weight imbalance is reduced; the model's adaptability to different abnormal states of the driver is enhanced, making dynamic updates more targeted; key abnormal signals are avoided due to unreasonable thresholds, and protection response is delayed due to improper weight allocation; the system can accurately fill the discrimination gaps in each dimension and balance the priority of the three dimensions; the model can keep up with changes in actual driving scenarios and provide more reliable support for subsequent protection decisions, further improving the recognition reliability and adaptation flexibility of the disability protection system.

[0102] An embodiment of the present invention, as shown in FIG2, provides a system for implementing the automatic driving state determination method based on multi-target recognition as described above, the system comprising:

[0103] Data acquisition module: synchronously acquires millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; among which, millimeter-wave radar physiological signals include the driver's respiratory rate and heart rate data, multi-target visual behavioral feature data covers the driver's facial, hand, and body posture features, and vehicle operation data includes vehicle speed, throttle opening, braking status, steering angle, and steering wheel angle data.

[0104] Model building module: Based on the collected millimeter-wave radar physiological signals, multi-target visual behavior feature data and vehicle operation data, a three-level cascaded discrimination model for physiological instability, behavioral abnormality and control degradation is constructed; among them, the physiological instability criterion is based on abnormal changes in respiratory rate and heart rate, the behavioral abnormality criterion integrates facial drooping, hand off the rudder and abnormal body posture, and the control degradation criterion combines abnormal control patterns in vehicle operation data.

[0105] Risk assessment module: Based on a three-level cascaded discriminant model, it performs dynamic risk level assessment of the driver's condition within 4 to 7 seconds and generates dynamic risk level data; the dynamic risk level is divided into three levels: low risk, medium risk and high risk, which correspond to normal driver condition, potential abnormality and severe disability, respectively.

[0106] Strategy execution module: Based on dynamic risk level data, it drives a graded response strategy; in low-risk situations, it only triggers an audible and visual warning to alert the driver; in medium-risk situations, it initiates speed limit control to reduce the vehicle speed; and in high-risk situations, it performs smooth braking to bring the vehicle to a safe stop and simultaneously uploads alarm information to the management backend.

[0107] Real-time feedback module: Based on the execution results of the hierarchical response strategy, it continuously optimizes the parameters of the three-level cascaded discrimination model to form a closed-loop control system for perception, discrimination, decision-making and execution; the closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism to improve the accuracy and response efficiency of automatic driving state discrimination.

[0108] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatic driving state determination based on multi-target recognition, characterized in that, The method includes: S1, simultaneously acquiring millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; S2, constructing a three-level cascaded discrimination model for physiological instability, behavioral abnormalities, and control degradation based on the acquired millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; S3, dynamically assessing the driver's state within 4 to 7 seconds based on the three-level cascaded discrimination model, generating dynamic risk level data; S4, driving a graded response strategy based on the dynamic risk level data, and simultaneously uploading alarm information to the management backend; S5, continuously optimizing the parameters of the three-level cascaded discrimination model based on the execution results of the graded response strategy, forming a closed-loop control system of perception, discrimination, decision-making, and execution; the closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism.

2. The automatic driving state determination method based on multi-target recognition according to claim 1, characterized in that, S1 includes: S11, activating the in-vehicle preset camera and millimeter-wave radar to simultaneously collect multi-target visual behavioral feature data of the driver and physiological signals from the millimeter-wave radar; S12, collecting data on vehicle speed, throttle opening, braking status, steering angle, and steering wheel angle during vehicle operation, integrating them with the collected data to generate a multi-dimensional raw dataset; S13, denoising the multi-dimensional raw dataset to remove invalid data caused by environmental interference and equipment fluctuations; S14, standardizing and regularizing the denoised dataset to unify the format and dimensions of different types of data; S15, performing time-series alignment processing on the standardized dataset to ensure that the time axis of the visual behavioral feature data, physiological signals, and vehicle operation data is consistent, generating a time-series synchronized dataset.

3. The automatic driving state determination method based on multi-target recognition according to claim 2, characterized in that, S12 includes: starting the vehicle operation data acquisition module, collecting basic operating parameters such as vehicle speed and throttle opening, and generating a basic operation dataset; continuously collecting data related to braking status, steering angle, and steering wheel angle, and generating a braking and steering dataset; supplementing the collection of vehicle gear information, improving the dimensions of the operation data, and generating a gear dataset; summarizing and integrating the basic operation dataset, braking and steering dataset, and gear dataset to generate a complete vehicle operation dataset; merging the complete vehicle operation dataset with the initial visual and physiological dataset generated in S11 to generate a multi-dimensional original dataset.

4. The automatic driving state determination method based on multi-target recognition according to claim 2, characterized in that, S15 includes: extracting the original timestamps of visual behavior feature data, physiological signals, and vehicle operation data from the standardized dataset to generate a multi-source independent timestamp sequence; comparing the temporal resolution differences of the multi-source independent timestamp sequences, selecting the highest precision timestamp as a benchmark reference, and generating a unified time benchmark framework; based on the unified time benchmark framework, interpolating and calibrating the time nodes of the visual behavior feature data, completing the missing time segment data, and generating a calibrated visual feature dataset; according to the unified time benchmark framework, correcting the time offset of the physiological signals, eliminating the time sequence deviation caused by the equipment acquisition delay, and generating a corrected physiological signal dataset; referring to the unified time benchmark framework, aligning the time scale of the vehicle operation data, synchronizing the time recording nodes of each parameter, and generating an aligned vehicle operation dataset; verifying the time synchronization error of the three sets of calibrated datasets, removing data with deviations exceeding the threshold, and generating a time consistency verification dataset; and integrating the time consistency verification dataset to form a time-series synchronized dataset with fully synchronized time axes for each dimension of data.

5. The automatic driving state determination method based on multi-target recognition according to claim 1, characterized in that, S2 includes: S21, based on the time-series synchronized dataset, splitting it into three independent criterion dimensions: physiological instability, behavioral abnormality, and control degradation; S22, building the basic framework of the cascaded model, clarifying the discrimination process and inter-dimensional connection logic of the three dimensions; S23, incorporating abnormal changes in respiratory rate and heart rate into the physiological instability criterion, facial drooping, hand off the rudder, and abnormal body posture into the behavioral abnormality criterion, and abnormal control patterns in vehicle operation data into the control degradation criterion; S24, training the initially constructed model using multi-scenario driving samples, calibrating the logical thresholds and dimensional correlation weights of each criterion; S25, after completing training and verification, generating a mature three-level cascaded discrimination model that can accurately discriminate driving states.

6. The automatic driving state determination method based on multi-target recognition according to claim 1, characterized in that, S3 includes: S31, inputting the time-series synchronized dataset into a mature three-level cascaded discrimination model to initiate a dynamic risk assessment process; S32, continuously receiving real-time updated time-series synchronized data within a set time window of 4 to 7 seconds, and performing multiple rounds of continuous discrimination; S33, weighting and fusing the results of multiple rounds of discrimination, and comprehensively considering the confidence level of the discrimination results of each round; S34, matching the fusion results with three levels of low risk, medium risk, and high risk according to preset standards; S35, outputting the matching results to generate dynamic risk level data, which corresponds to the driver's normal state, potential abnormality, and severe disability.

7. The automatic driving state determination method based on multi-target recognition according to claim 6, characterized in that, S34 includes: extracting driving state criteria corresponding to low, medium, and high risks from preset standards, clarifying the quantitative characteristics of normal, potentially abnormal, and severely disabled individuals, and generating a level matching benchmark set; analyzing the discrimination results after multiple rounds of fusion, refining the comprehensive abnormality index of physiological signals, behavioral characteristics, and control data, and generating a key indicator set for the fusion result; comparing the key indicator set with the level matching benchmark set one by one, corresponding to the feature threshold range of each risk level, and generating indicator comparison results; determining the risk level interval to which the fusion result belongs based on the comparison results, and generating a preliminary level matching result; verifying the consistency between the preliminary matching result and the data of each dimension, eliminating mismatches caused by anomalies in a single dimension, and generating a final level matching result.

8. The automatic driving state determination method based on multi-target recognition according to claim 1, characterized in that, S4 includes: S41, receiving dynamic risk level data and retrieving the system's preset graded response rule base; S42, if determined to be low risk, triggering the sound, light, and vibration warning modules to issue a status reminder to the driver; S43, if determined to be medium risk, while continuously issuing warnings, activating speed limit control to limit throttle output and slow down the vehicle's speed; S44, if determined to be high risk, activating the braking mechanism to smoothly stop the vehicle by means of the original vehicle's EBS, an added valve, or a lever motor pulling the brake pedal; S45, in the high-risk state, simultaneously triggering the alarm upload function to push abnormal information to the management backend, while recording the entire response process data and generating a response execution dataset.

9. The automatic driving state determination method based on multi-target recognition according to claim 1, characterized in that, S5 includes: S51, collecting response execution datasets and real-time driving status feedback data, and integrating them to form a closed-loop feedback data set; S52, performing difference analysis on the closed-loop feedback data set to locate model discrimination deviation nodes and response strategy lag links; S53, dynamically adjusting the discrimination threshold and correlation weights of each dimension of the three-level cascaded discrimination model based on the analysis results; S54, correcting the triggering conditions and execution parameters of the graded response strategy, and optimizing the warning timing and braking force; S55, importing the adjusted parameters and strategies into the system to complete the iterative update of the model and strategies, forming a continuously optimized closed-loop control system.

10. A system for implementing the automatic driving state determination method based on multi-target recognition as described in claim 1, characterized in that, The system includes: a data acquisition module that simultaneously acquires millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; a model building module that constructs a three-level cascaded discrimination model for physiological instability, behavioral abnormalities, and control degradation based on the acquired millimeter-wave radar physiological signals, multi-target visual behavioral feature data, and vehicle operation data; a risk assessment module that dynamically assesses the driver's state risk level within 4 to 7 seconds based on the three-level cascaded discrimination model and generates dynamic risk level data; a strategy execution module that drives a graded response strategy based on the dynamic risk level data and simultaneously uploads alarm information to the management backend; and a real-time feedback module that continuously optimizes the parameters of the three-level cascaded discrimination model based on the execution results of the graded response strategy, forming a closed-loop control system for perception, discrimination, decision-making, and execution. The closed-loop control system dynamically adjusts the discrimination threshold and response strategy through a real-time feedback mechanism.

Images