Unsafe driving behavior identification method and system based on multi-dimensional feature fusion

Abstract

The invention relates to an auxiliary driving system, and particularly discloses an unsafe driving behavior identification method and system based on multi-dimensional feature fusion, and the method comprises the steps: obtaining original monitoring data of a driver, the original monitoring data comprising driver monitoring data, driving scene data and driving path data; obtaining a driver state feature vector according to the driver monitoring data, and obtaining a driver state evaluation value according to the driver state feature vector; and obtaining a driving task load index according to the driving scene data and the driving path data. According to the method, the driving risk value is obtained by fusing the driver state evaluation value and the driving task load index, and dynamic early warning is performed, so that the problems of insufficient data robustness, rigid fusion and lack of state-task mismatch evaluation are solved, and the method has the advantages that the identification precision is improved, false alarm and missing alarm are reduced, and the robustness of the system in a complex environment is enhanced.

Description

Technical Field

[0001] This invention relates to driver assistance systems, and more particularly to a method and system for identifying unsafe driving behaviors based on multi-dimensional feature fusion. Background Technology

[0002] With the rapid development of intelligent transportation systems and advanced driver assistance systems (ADAS), real-time and accurate monitoring of driver status has become a key aspect of improving road safety. Unsafe driving behaviors, such as fatigued driving, distracted driving, and dangerous operations, are the main human factors causing traffic accidents.

[0003] In current driving technologies, the reliability and information value of different data sources are not static but fluctuate in real time with environmental factors such as lighting, road conditions, and traffic density. For example, the quality of information provided by visual sensors deteriorates under low-light conditions, and the interpretation of vehicle lateral dynamic data in continuous curves is even more complex. Most current fusion methods, when dealing with this dynamism, typically rely on pre-set, static fusion rules, lacking an adaptive fusion mechanism based on real-time scenarios and data quality. This may lead to over-reliance on the data source at that particular time in specific scenarios, or failure to fully consider more valuable cues at that moment, thus reducing recognition accuracy in real-world environments. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for identifying unsafe driving behaviors based on multi-dimensional feature fusion, so as to solve the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for identifying unsafe driving behaviors based on multi-dimensional feature fusion includes: Obtain the driver's raw monitoring data, wherein the raw monitoring data includes driver monitoring data, driving scenario data, and driving route data; The driver state feature vector is obtained based on the driver monitoring data, and the driver state evaluation value is obtained based on the driver state feature vector. The driving task load index is obtained based on the driving scenario data and driving route data. The driving risk value is obtained based on the driver status assessment value and the driving task load index. The driving risk value is compared with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined period of time, the driver is determined to be in a high-risk state, and a warning intervention is triggered.

[0006] Preferably, the step of obtaining the driver state feature vector based on the driver monitoring data includes: Based on the driver monitoring data, visual image data stream, physiological sensor data stream, and vehicle operation data stream are acquired. A visual feature sequence is obtained by performing continuous frame analysis on the visual image data stream, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence; The physiological sensor data stream is preprocessed to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; The operation feature sequence is obtained by performing sliding window statistics on the vehicle operation data stream, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; The visual feature sequence, physiological feature sequence, and operational feature sequence are timestamped and normalized to generate a synchronized standardized feature matrix. Multi-scale feature extraction is performed on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, which are then fused to form a driver state feature vector.

[0007] Preferably, the step of obtaining the driver state evaluation value based on the driver state feature vector includes: Based on the driver's state feature vector, obtain the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set; The fatigue trend, the proportion of abnormal duration, and the frequency of eye movement instability are obtained based on the physiological fatigue feature set. The fatigue trend, the proportion of abnormal duration, and the frequency of eye movement instability are then fused to obtain the physiological fatigue risk value. Based on the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set, a multidimensional state anomaly consistency coefficient is obtained; The driver's state assessment value is obtained based on the multidimensional state anomaly consistency coefficient and the physiological fatigue risk value.

[0008] Preferably, the step of obtaining the driving task load index based on the driving scenario data and driving route data includes: A traffic environment complexity parameter set is obtained based on the driving scenario data, and a road alignment load parameter set is extracted from the driving path data, wherein the road alignment load parameter set contains multiple sets of feature parameters characterizing driving difficulty. Based on the traffic environment complexity parameter set and the road alignment load parameter set, obtain the consistency coefficient between the environment and the road load; A historical normal driving database is obtained, and the road alignment load parameter set is normalized based on the historical normal driving database to obtain the basic road load index. Traffic flow status identifiers and real-time disturbance indicators are extracted from the traffic environment complexity parameter set. The real-time disturbance indicators include traffic density change rate, environmental visibility attenuation coefficient and frequency of disturbance events. The traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of the disturbance events are fused to obtain the real-time environmental disturbance load index.

[0009] Preferably, the step of fusing the traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of the disturbance events to obtain the real-time environmental disturbance load index includes: The traffic density change rate sequence, environmental visibility attenuation coefficient sequence, and disturbance event frequency sequence within the current monitoring period are obtained to form a set of environmental disturbance time series parameters; Calculate the perturbation coordination coefficients between each parameter sequence based on the environmental perturbation time series parameter set; Based on the historical normal driving database, the traffic density change rate, environmental visibility attenuation coefficient and frequency of interference events are normalized to obtain normalized disturbance intensity values. Extract the current traffic scenario code from the traffic flow status identifier and obtain the weight parameter set corresponding to the current traffic scenario code, wherein the weight parameter set includes traffic density change rate weight, environmental visibility attenuation coefficient weight and interference event frequency weight. Based on the set of weight parameters, a weighted fusion algorithm is used to fuse the normalized disturbance intensity values ​​to obtain the initial environmental disturbance load index. The initial environmental disturbance load index is corrected based on the disturbance coordination coefficient to obtain the real-time environmental disturbance load index.

[0010] Preferably, the step of obtaining the driving risk value based on the driver state assessment value and the driving task load index includes: A set of driver status deterioration parameters is obtained based on the driver status assessment value, and a set of driving task demand parameters is obtained based on the driving task load index. The set of driving task demand parameters includes multiple sets of feature parameters for risk quantification. Based on the driver state degradation parameter set and the driving task requirement parameter set, obtain the state-task mismatch coefficient; Obtain the historical safe driving database, and normalize the driver status assessment value and driving task load index based on the historical safe driving database to obtain the normalized status risk value and normalized task load value. The current risk scenario identifier is obtained based on the driving scenario data, and the state risk weight and task load weight corresponding to the current risk scenario identifier are obtained. Based on the state risk weight and task load weight, a dual-weight fusion algorithm is used to fuse the normalized state risk value and the normalized task load value to obtain an initial comprehensive risk index. The initial comprehensive risk index is dynamically corrected based on the state-task mismatch coefficient to obtain the driving risk value.

[0011] This invention also discloses an unsafe driving behavior recognition system based on multi-dimensional feature fusion, comprising: The raw monitoring data acquisition module is used to acquire the driver's raw monitoring data, which includes driver monitoring data, driving scenario data, and driving route data. The driver status assessment value acquisition module is used to acquire a driver status feature vector based on the driver monitoring data, and to acquire a driver status assessment value based on the driver status feature vector. The driving task load index acquisition module is used to acquire the driving task load index based on the driving scenario data and driving path data. The driving risk value acquisition module is used to acquire the driving risk value based on the driver status assessment value and the driving task load index. The warning module is used to compare the driving risk value with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined time, the driver is determined to be in a high-risk state, and a warning intervention is triggered.

[0012] Preferably, the driver state assessment value acquisition module includes: The driver monitoring data analysis unit is used to acquire visual image data stream, physiological sensor data stream and vehicle operation data stream based on the driver monitoring data. A visual feature sequence acquisition unit is used to perform continuous frame analysis on the visual image data stream to obtain a visual feature sequence, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence. A physiological feature sequence acquisition unit is used to perform signal preprocessing on the physiological sensing data stream to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; An operation feature sequence acquisition unit is used to perform sliding window statistics on the vehicle operation data stream to obtain an operation feature sequence, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; The feature matrix generation unit is used to perform timestamp alignment and normalization on the visual feature sequence, physiological feature sequence and operational feature sequence to generate a synchronously standardized feature matrix. The fusion unit is used to perform multi-scale feature extraction on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, and fuse them to form a driver state feature vector.

[0013] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a method for identifying unsafe driving behaviors based on multi-dimensional feature fusion.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a method for identifying unsafe driving behaviors based on multi-dimensional feature fusion.

[0015] The beneficial effects of this application are as follows: This invention obtains driving risk values ​​by fusing driver state assessment values ​​and driving task load indexes, and performs dynamic early warnings, which solves the problems of insufficient data robustness, rigid fusion, and lack of state-task mismatch assessment. It has the advantages of improving recognition accuracy, reducing false alarms and false alarms, and enhancing the robustness of the system in complex environments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a method flow according to an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the system structure according to an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the internal structure of a computer device according to an embodiment of this application.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0021] like Figure 1 As shown, this application provides a method for identifying unsafe driving behaviors based on multi-dimensional feature fusion, including: S1. Obtain the driver's raw monitoring data, wherein the raw monitoring data includes driver monitoring data, driving scenario data, and driving route data; S2. Obtain the driver state feature vector based on the driver monitoring data, and obtain the driver state evaluation value based on the driver state feature vector; S3. Obtain the driving task load index based on the driving scenario data and driving route data; S4. Obtain the driving risk value based on the driver status assessment value and the driving task load index; S5. Compare the driving risk value with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined duration, the driver is determined to be in a high-risk state, and a warning intervention is triggered.

[0022] As described in steps S1-S5 above, the raw monitoring data refers to the initial data set used to comprehensively assess the driver's state, driving scenario, and driving path. Driver monitoring data refers to data that directly reflects the driver's physiological, behavioral, and operational state, such as images, physiological signals, and vehicle operation signals collected by cameras, sensors, and other devices. Driving scenario data refers to data describing the environmental conditions surrounding the vehicle, such as traffic flow, weather conditions, and light intensity. Driving path data refers to data describing the characteristics of the vehicle's driving route, such as road type, curvature, gradient, and speed limit information. The driver state feature vector refers to a numerical representation extracted and fused from the driver monitoring data that quantifies the driver's current physiological and cognitive state. The driver state assessment value refers to a quantitative indicator calculated based on the driver state feature vector, comprehensively reflecting the degree of driver fatigue, distraction, or operational instability. The driving task load index refers to a quantitative indicator comprehensively assessed based on driving scenario data and driving path data, reflecting the degree of cognitive and operational demands of the current driving task on the driver. The driving risk value refers to an indicator that quantifies the potential danger of the current driving behavior after comprehensively considering the driver state assessment value and the driving task load index.

[0023] The process involves obtaining a driver state feature vector from driver monitoring data and then using this vector to generate a driver state assessment value. Obtaining the driver state feature vector can be achieved by performing preliminary processing on the driver monitoring data, such as simple edge detection or color analysis on image data, filtering physiological signals, and sampling vehicle operation data. This yields a series of raw features reflecting the driver's state. Subsequently, the driver state assessment value is obtained by comparing these raw features with preset static thresholds. For example, if the driver's blinking frequency falls below a certain fixed threshold, it is considered a possible sign of fatigue, and a simple fatigue assessment value is generated accordingly.

[0024] The driving task load index is obtained based on driving scenario data and driving route data. The driving task load index can be obtained by assessing the driving scenario data and driving route data.

[0025] The driving risk value is obtained based on the driver's condition assessment value and the driving workload index. The driving risk value can be obtained by simply averaging or weighting the driver's condition assessment value and the driving workload index.

[0026] The system compares a driving risk value with a preset risk threshold. If the driving risk value consistently exceeds the preset risk threshold for a predetermined duration, the driver is determined to be in a high-risk state, and a warning intervention is triggered. The preset risk threshold can be a fixed value; for example, a driving risk value exceeding 0.7 is considered high risk. The predetermined duration can also be a fixed time interval, such as 5 seconds. When the driving risk value consistently exceeds the fixed threshold for the specified duration, the driver is determined to be in a high-risk state. At this point, a simple audible and visual alarm can be triggered, such as a buzzer or flashing warning lights, to alert the driver.

[0027] In one embodiment, the step of obtaining a driver state feature vector based on the driver monitoring data includes: S201. Obtain visual image data stream, physiological sensor data stream and vehicle operation data stream based on the driver monitoring data; S202. Perform continuous frame analysis on the visual image data stream to obtain a visual feature sequence, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence. S203. Perform signal preprocessing on the physiological sensing data stream to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; S204. Perform sliding window statistics on the vehicle operation data stream to obtain an operation feature sequence, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; S205. Perform timestamp alignment and normalization on the visual feature sequence, physiological feature sequence and operational feature sequence to generate a synchronous standardized feature matrix. S206. Perform multi-scale feature extraction on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, and fuse them to form a driver state feature vector.

[0028] As described in steps S201-S206 above, visual image data streams, physiological sensor data streams, and vehicle operation data streams are acquired based on driver monitoring data to comprehensively perceive the driver's state from multiple dimensions and modalities. Specifically, the visual image data stream can capture real-time images of the driver's face, eyes, and upper body using cameras installed in the driver's cab. For example, an infrared camera can be used to acquire clear images under different lighting conditions, or a visible light camera can be used in conjunction with image enhancement technology. The physiological sensor data stream can collect real-time physiological signals such as the driver's electrocardiogram, skin conductance, and respiration using wearable sensors (such as smart bracelets and smartwatches) or non-contact sensors (such as seat-integrated sensors and steering wheel-integrated sensors). The vehicle operation data stream can acquire data from the vehicle's CAN bus system, including data from steering wheel angle sensors, accelerator pedal position sensors, brake pedal pressure sensors, and vehicle speed sensors.

[0029] By acquiring visual image data streams, physiological sensor data streams, and vehicle operation data streams, this application comprehensively perceives the driver's state from multiple dimensions and modalities. Continuous frame analysis of the visual image data stream captures dynamic changes in the driver's eye closure, gaze direction, and head posture, avoiding information omissions that may occur with single-frame images and more accurately reflecting the driver's fatigue and distraction. Signal preprocessing of the physiological sensor data stream makes physiological indicators such as heart rate variability trends and respiratory rhythms more stable and reliable. Sliding window statistics of the vehicle operation data stream can quantify operational features such as steering wheel angle entropy and longitudinal acceleration jitter in real time, accurately capturing subtle abnormalities in the driver's operation and reflecting their control stability. By aligning and normalizing these heterogeneous feature sequences with timestamps, this application eliminates time delays and dimensional differences between different data sources, generating a synchronized and standardized feature matrix. Based on this, multi-scale feature extraction is performed to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, ensuring comprehensive capture of subtle changes in the driver's state from different time, frequency, and complexity dimensions, ultimately fusing them to form a driver state feature vector.

[0030] In one embodiment, the step of obtaining a driver state evaluation value based on the driver state feature vector includes: S207. Obtain the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set based on the driver state feature vector; S208. Obtain fatigue trend, abnormal duration percentage and eye movement instability frequency based on the physiological fatigue feature set, and fuse the fatigue trend, abnormal duration percentage and eye movement instability frequency to obtain physiological fatigue risk value. S209. Based on the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set, obtain the multidimensional state anomaly consistency coefficient. S210. Obtain the driver's state assessment value based on the multidimensional state anomaly consistency coefficient and the physiological fatigue risk value.

[0031] As described in steps S207-S210 above, fatigue trend, abnormal duration percentage, and eye movement instability frequency are obtained based on the physiological fatigue feature set. These three indicators are then fused to obtain a physiological fatigue risk value. This step aims to extract three key indicators from the physiological fatigue feature set and fuse them to quantify physiological fatigue risk. The fatigue trend reflects the changing pattern of the driver's fatigue state over time, such as whether the fatigue level gradually worsens or decreases; the abnormal duration percentage measures the proportion of abnormal physiological states (such as prolonged eye closure, excessively low or high heart rate) to the total monitoring time; and the eye movement instability frequency quantifies the frequency of abnormal eye movements (such as blinking frequency and eye movement speed). These three indicators characterize the severity and dynamic changes of physiological fatigue from different perspectives. For example, fatigue trends can be obtained through time-series analysis methods such as moving averages or exponential smoothing on time-series data in the physiological fatigue feature set; the proportion of abnormal duration can be obtained by statistically analyzing the length of time when physiological indicators exceed the normal range and calculating their proportion; the frequency of eye movement feature instability can be obtained by detecting abnormal fluctuations or patterns in the eye movement feature sequence and counting their frequency. Finally, a weighted summation can be used to combine these three indicators into a single physiological fatigue risk value.

[0032] In one embodiment, the step of obtaining the driving task load index based on the driving scenario data and driving route data includes: S301. Obtain a traffic environment complexity parameter set based on the driving scenario data, and extract a road alignment load parameter set from the driving path data, wherein the road alignment load parameter set contains multiple sets of feature parameters characterizing driving difficulty. S302. Based on the traffic environment complexity parameter set and the road alignment load parameter set, obtain the consistency coefficient between the environment and the road load; S303. Obtain the historical normal driving database, and normalize the road alignment load parameter set based on the historical normal driving database to obtain the basic road load index. Extract traffic flow status identifiers and real-time disturbance indicators from the traffic environment complexity parameter set. The real-time disturbance indicators include traffic density change rate, environmental visibility attenuation coefficient and frequency of disturbance events. S304. The traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of the disturbance events are fused to obtain the real-time environmental disturbance load index.

[0033] As described in steps S301-S304 above, driving scenario data refers to real-time or near-real-time information used to describe the environmental conditions surrounding the vehicle. This data can originate from onboard sensors, such as images or video streams acquired through cameras, obstacle distance and speed information acquired through radar or lidar, and information about nearby objects acquired through ultrasonic sensors. Additionally, driving scenario data can also be obtained from other vehicles or roadside units via vehicle-to-everything (V2X) communication, such as traffic light status and road construction information ahead. Driving path data refers to information describing the vehicle's route and road geometry. This data can originate from vehicle position and speed information provided by the onboard GPS module, combined with high-precision map data, to obtain information such as road curvature, slope, number of lanes, and speed limits. Another method is to use the vehicle's inertial measurement unit and wheel speed sensor data, combined with trajectory estimation algorithms, to construct or update the vehicle's driving trajectory in real time. This system addresses the inaccuracy of traditional load assessment methods by accurately evaluating driving task load through a multi-dimensional and dynamic approach. First, it extracts a set of traffic environment complexity parameters from driving scenario data and a set of road alignment load parameters from driving path data. These two sets quantify the inherent difficulty of the driving task from both environmental and road dimensions. The system obtains a consistency coefficient between environmental and road loads, enabling it to identify the synergy between these factors and avoid biases that might arise from isolated assessments. For example, when a curve (high road load) is accompanied by complex traffic (high environmental load), the consistency coefficient reflects a higher overall load. The system normalizes the road alignment load parameter set using a historical normal driving database to obtain a basic road load index. This normalization eliminates differences between different drivers, vehicles, or measurement conditions, ensuring the comparability of the load index. Simultaneously, to capture the dynamic changes in driving task load, it extracts traffic flow status indicators and real-time disturbance indices from the traffic environment complexity parameter set. These real-time disturbance indicators, including traffic density change rate, environmental visibility attenuation coefficient, and frequency of disturbance events, can promptly reflect the additional stress on drivers caused by sudden events or environmental changes. Finally, by fusing these real-time disturbance indicators, a real-time environmental disturbance load index is obtained.

[0034] In one embodiment, the step of fusing the traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of disturbance events to obtain the real-time environmental disturbance load index includes: S3041. Obtain the traffic density change rate sequence, environmental visibility attenuation coefficient sequence, and interference event frequency sequence within the current monitoring period to form an environmental disturbance time series parameter set. S3042. Calculate the disturbance coordination coefficient between each parameter sequence based on the environmental disturbance time series parameter set; S3043. Based on the historical normal driving database, the traffic density change rate, the environmental visibility attenuation coefficient and the frequency of interference events are normalized to obtain the normalized disturbance intensity value. S3044. Extract the current traffic scenario code from the traffic flow status identifier and obtain the weight parameter set corresponding to the current traffic scenario code, wherein the weight parameter set includes traffic density change rate weight, environmental visibility attenuation coefficient weight and interference event frequency weight. S3045. Based on the weight parameter set, the normalized disturbance intensity values ​​are fused using a weighted fusion algorithm to obtain the initial environmental disturbance load index. S3046. The initial environmental disturbance load index is corrected according to the disturbance coordination coefficient to obtain the real-time environmental disturbance load index.

[0035] As described in steps S3041-S3046 above, the traffic density change rate sequence, environmental visibility attenuation coefficient sequence, and interference event frequency sequence for the current monitoring period are obtained to form an environmental disturbance time series parameter set, aiming to capture the dynamic changes of environmental disturbance factors in real time. The traffic density change rate sequence can reflect the fluctuation of road traffic flow. For example, through vehicle detectors, floating car data, or camera image analysis, the rate of change of the number of vehicles passing through a certain cross section per unit time within a continuous time window is calculated. The environmental visibility attenuation coefficient sequence characterizes the clarity of the driver's field of vision. For example, the detection distance and clarity of obstacles or road signs ahead can be evaluated through vehicle-mounted lidar, millimeter-wave radar, or visual sensors, or weather data such as fog, rain, and snow can be obtained through meteorological sensors and converted into visibility attenuation coefficients. The interference event frequency sequence records sudden events that may affect the driver's attention, such as the frequency of events such as pedestrians entering the road, non-motorized vehicles suddenly changing lanes, and traffic accidents identified by vehicle-mounted cameras. By acquiring the traffic density change rate sequence, environmental visibility attenuation coefficient sequence, and disturbance event frequency sequence within the current monitoring period, a dynamic time-series parameter set for environmental disturbances was constructed. This ensures that the perception of environmental disturbances is real-time and continuous, rather than a static snapshot, laying a data foundation for subsequent refined analysis. Subsequently, the disturbance synergy coefficients between the parameter sequences were calculated based on this environmental disturbance time-series parameter set. This aims to quantify the mutual influence and linkage effects between different disturbance factors, identifying whether they act independently or mutually reinforce or weaken each other, thereby avoiding evaluation bias caused by viewing factors in isolation. Based on this, to eliminate the influence of different physical dimensions and establish a unified evaluation benchmark, the traffic density change rate, environmental visibility attenuation coefficient, and disturbance event frequency were normalized based on a historical normal driving database to obtain normalized disturbance intensity values. This standardization process allows different types of disturbance data to be compared and fused on the same scale. To achieve scenario adaptability, the system extracts the current traffic scenario code from the traffic flow state identifier and obtains a preset weight parameter set accordingly. This weight parameter set dynamically adjusts the importance of each disturbance factor based on different traffic scenarios. Next, based on these scenario-specific weight parameter sets, the system uses a weighted fusion algorithm to fuse the normalized disturbance intensity values, thereby obtaining the initial environmental disturbance load index. This fusion process considers the relative contribution of each disturbance factor in the current scenario, making the initial load index more context-relevant. Finally, to further improve the accuracy of the assessment, the system corrects the initial environmental disturbance load index based on the previously calculated disturbance synergy coefficient, thus obtaining the real-time environmental disturbance load index.

[0036] In one embodiment, the step of obtaining a driving risk value based on the driver state assessment value and the driving task load index includes: S401. Obtain a set of driver status deterioration parameters based on the driver status assessment value, and obtain a set of driving task demand parameters based on the driving task load index, wherein the set of driving task demand parameters includes multiple sets of feature parameters for risk quantification. S402. Obtain the state-task mismatch coefficient based on the driver state degradation parameter set and the driving task requirement parameter set; S403. Obtain the historical safe driving database, and normalize the driver status assessment value and driving task load index based on the historical safe driving database to obtain the normalized status risk value and normalized task load value. S404. Obtain the current risk scenario identifier based on the driving scenario data, and obtain the state risk weight and task load weight corresponding to the current risk scenario identifier; S405. Based on the state risk weight and task load weight, a dual-weight fusion algorithm is used to fuse the normalized state risk value and the normalized task load value to obtain an initial comprehensive risk index. S406. The initial comprehensive risk index is dynamically corrected based on the state-task mismatch coefficient to obtain the driving risk value.

[0037] As described in steps S401-S406 above, the driver condition deterioration parameter set refers to a set of indicators that quantify the degree of driver condition deterioration. It can be based on a set of physiological fatigue characteristics, a set of cognitive distraction characteristics, and a set of operational instability characteristics. These characteristics can be comprehensively evaluated using an expert system or machine learning model (e.g., support vector machine, neural network) to extract parameters reflecting the trend of driver fatigue, distraction, and operational instability. For example, these parameters may include fatigue level, distraction level, and operational stability score. Alternatively, it can be achieved by performing time series analysis on driver condition assessment values ​​to extract statistical indicators such as the rate of change, volatility, and duration of abnormalities as driver condition deterioration parameters. For example, these indicators may include the rate of decline in driver condition assessment values, short-term fluctuation amplitude, and duration of consecutive low scores. The driving task requirement parameter set refers to a set of indicators that quantifies the degree of driver capability requirements of the current driving task. It can comprehensively assess the difficulty and complexity of the current driving task based on driving scenario data (e.g., a set of traffic environment complexity parameters) and driving path data (e.g., a set of road alignment load parameters) through preset rules or machine learning models (e.g., decision trees, random forests). For example, this can include road curvature, traffic density, number of intersections, and weather conditions (rain, snow, fog). Alternatively, it can analyze dynamic data such as vehicle speed, acceleration, and steering angle, combined with map information (e.g., curves, slopes, lane widths), to extract parameters reflecting the real-time load of the driving task. These parameters can include average vehicle speed, vehicle speed standard deviation, steering angle change rate, and lane keeping difficulty.

[0038] The state-task mismatch coefficient is an indicator that quantifies the degree of mismatch between a driver's current state and the demands of the driving task. It can be achieved by constructing a multi-dimensional matching model, taking a set of driver state deterioration parameters and a set of driving task demand parameters as input, and using methods such as fuzzy logic, neural networks, or regression analysis to output a coefficient between 0 and 1, where 0 represents a perfect match and 1 represents a severe mismatch. For example, a mismatch function can be defined so that the mismatch coefficient approaches 1 when both driver fatigue and task load are high. Alternatively, a series of matching rules can be set; for example, if the driver's fatigue level exceeds a threshold and the current road curvature is greater than a certain value, the mismatch coefficient increases; if the driver's distraction level is high and traffic density is high, the mismatch coefficient also increases. The mismatch coefficient is calculated by accumulating or weighted averaging the trigger values ​​of these rules. A historical safe driving database refers to a collection of historical data storing a large number of drivers under safe driving conditions, including various driver state assessment values ​​and driving task load indices. It can collect driving behavior data from a large number of drivers over a long period of time in a controlled environment or on real roads, and have experts label their driving status and workload to ensure that the data is acquired under safe and normal driving conditions. Alternatively, it can use onboard sensors and external environmental sensors to continuously collect driving data on actual roads, and use anomaly detection algorithms to filter out unsafe driving behavior data, retaining only the data determined to be safe driving, to build a dynamically updated database. By introducing a state-task mismatch coefficient, normalization processing, weight allocation, and dynamic correction mechanisms, this scheme achieves accurate calculation of driving risk values. Specifically, the scheme first extracts a set of driver state degradation parameters from the driver state assessment value and a set of driving task requirement parameters from the driving task load index. These parameter sets provide more granular information to quantify the degree of driver state deterioration and the complexity of the driving task. Based on this, by comparing the driver state degradation parameter set and the driving task requirement parameter set, the state-task mismatch coefficient is calculated, directly quantifying the degree of mismatch between the driver's current state and the driving task requirements, thus solving the problem of neglecting state-task mismatch in traditional methods. To ensure effective integration of data from different sources and scales, the scheme uses a historical safe driving database to normalize the driver state assessment value and the driving task load index, obtaining normalized state risk values ​​and normalized task load values, thereby unifying them to a comparable scale. Simultaneously, considering the different emphases of risk factors in different driving scenarios, the scheme obtains the current risk scenario identifier based on the driving scenario data and assigns corresponding state risk weights and task load weights to this scenario. These weights reflect the relative contributions of driver state risk and driving task load to overall risk in a specific scenario. Subsequently, a dual-weight fusion algorithm is used to fuse the normalized state risk value and normalized task load value with their corresponding weights to obtain an initial comprehensive risk index.

[0039] like Figure 2 As shown, the present invention also provides an unsafe driving behavior recognition system based on multi-dimensional feature fusion, comprising: The raw monitoring data acquisition module 1 is used to acquire the driver's raw monitoring data, wherein the raw monitoring data includes driver monitoring data, driving scenario data and driving route data; The driver status assessment value acquisition module 2 is used to acquire a driver status feature vector based on the driver monitoring data, and to acquire a driver status assessment value based on the driver status feature vector. The driving task load index acquisition module 3 is used to acquire the driving task load index based on the driving scenario data and driving path data. The driving risk value acquisition module 4 is used to acquire the driving risk value based on the driver status assessment value and the driving task load index. The warning module 5 is used to compare the driving risk value with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined time, the driver is determined to be in a high-risk state and a warning intervention is triggered.

[0040] In one embodiment, the driver state assessment value acquisition module 2 includes: The driver monitoring data analysis unit is used to acquire visual image data stream, physiological sensor data stream and vehicle operation data stream based on the driver monitoring data. A visual feature sequence acquisition unit is used to perform continuous frame analysis on the visual image data stream to obtain a visual feature sequence, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence. A physiological feature sequence acquisition unit is used to perform signal preprocessing on the physiological sensing data stream to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; An operation feature sequence acquisition unit is used to perform sliding window statistics on the vehicle operation data stream to obtain an operation feature sequence, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; The feature matrix generation unit is used to perform timestamp alignment and normalization on the visual feature sequence, physiological feature sequence and operational feature sequence to generate a synchronously standardized feature matrix. The fusion unit is used to perform multi-scale feature extraction on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, and fuse them to form a driver state feature vector.

[0041] like Figure 3As shown, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a method for identifying unsafe driving behaviors based on multi-dimensional feature fusion.

[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a method for identifying unsafe driving behaviors based on multi-dimensional feature fusion.

[0043] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0044] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0045] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent results or equivalent process transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for identifying unsafe driving behaviors based on multi-dimensional feature fusion, characterized in that, include: Obtain the driver's raw monitoring data, wherein the raw monitoring data includes driver monitoring data, driving scenario data, and driving route data; The driver state feature vector is obtained based on the driver monitoring data, and the driver state evaluation value is obtained based on the driver state feature vector. The driving task load index is obtained based on the driving scenario data and driving route data. The driving risk value is obtained based on the driver status assessment value and the driving task load index. The driving risk value is compared with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined period of time, the driver is determined to be in a high-risk state, and a warning intervention is triggered.

2. The method for identifying unsafe driving behaviors based on multi-dimensional feature fusion according to claim 1, characterized in that, The step of obtaining the driver state feature vector based on the driver monitoring data includes: Based on the driver monitoring data, visual image data stream, physiological sensor data stream, and vehicle operation data stream are acquired. A visual feature sequence is obtained by performing continuous frame analysis on the visual image data stream, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence; The physiological sensor data stream is preprocessed to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; The operation feature sequence is obtained by performing sliding window statistics on the vehicle operation data stream, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; The visual feature sequence, physiological feature sequence, and operational feature sequence are timestamped and normalized to generate a synchronized standardized feature matrix. Multi-scale feature extraction is performed on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, which are then fused to form a driver state feature vector.

3. The method for identifying unsafe driving behaviors based on multi-dimensional feature fusion according to claim 1, characterized in that, The step of obtaining the driver state evaluation value based on the driver state feature vector includes: Based on the driver's state feature vector, obtain the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set; The fatigue trend, the proportion of abnormal duration, and the frequency of eye movement instability are obtained based on the physiological fatigue feature set. The fatigue trend, the proportion of abnormal duration, and the frequency of eye movement instability are then fused to obtain the physiological fatigue risk value. Based on the physiological fatigue feature set, cognitive distraction feature set, and operational instability feature set, a multidimensional state anomaly consistency coefficient is obtained; The driver's state assessment value is obtained based on the multidimensional state anomaly consistency coefficient and the physiological fatigue risk value.

4. The method for identifying unsafe driving behaviors based on multi-dimensional feature fusion according to claim 1, characterized in that, The steps for obtaining the driving task load index based on the driving scenario data and driving route data include: A traffic environment complexity parameter set is obtained based on the driving scenario data, and a road alignment load parameter set is extracted from the driving path data, wherein the road alignment load parameter set contains multiple sets of feature parameters characterizing driving difficulty. Based on the traffic environment complexity parameter set and the road alignment load parameter set, obtain the consistency coefficient between the environment and the road load; A historical normal driving database is obtained, and the road alignment load parameter set is normalized based on the historical normal driving database to obtain the basic road load index. Traffic flow status identifiers and real-time disturbance indicators are extracted from the traffic environment complexity parameter set. The real-time disturbance indicators include traffic density change rate, environmental visibility attenuation coefficient and frequency of disturbance events. The traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of the disturbance events are fused to obtain the real-time environmental disturbance load index.

5. The method for identifying unsafe driving behaviors based on multi-dimensional feature fusion according to claim 4, characterized in that, The step of fusing the traffic density change rate, the environmental visibility attenuation coefficient, and the frequency of disturbance events to obtain the real-time environmental disturbance load index includes: The traffic density change rate sequence, environmental visibility attenuation coefficient sequence, and disturbance event frequency sequence within the current monitoring period are obtained to form a set of environmental disturbance time series parameters; Calculate the perturbation coordination coefficients between each parameter sequence based on the environmental perturbation time series parameter set; Based on the historical normal driving database, the traffic density change rate, environmental visibility attenuation coefficient and frequency of interference events are normalized to obtain normalized disturbance intensity values. Extract the current traffic scenario code from the traffic flow status identifier and obtain the weight parameter set corresponding to the current traffic scenario code, wherein the weight parameter set includes traffic density change rate weight, environmental visibility attenuation coefficient weight and interference event frequency weight. Based on the set of weight parameters, a weighted fusion algorithm is used to fuse the normalized disturbance intensity values ​​to obtain the initial environmental disturbance load index. The initial environmental disturbance load index is corrected based on the disturbance coordination coefficient to obtain the real-time environmental disturbance load index.

6. The method for identifying unsafe driving behaviors based on multi-dimensional feature fusion according to claim 1, characterized in that, The steps for obtaining a driving risk value based on the driver status assessment value and the driving task load index include: A set of driver status deterioration parameters is obtained based on the driver status assessment value, and a set of driving task demand parameters is obtained based on the driving task load index. The set of driving task demand parameters includes multiple sets of feature parameters for risk quantification. Based on the driver state degradation parameter set and the driving task requirement parameter set, obtain the state-task mismatch coefficient; Obtain the historical safe driving database, and normalize the driver status assessment value and driving task load index based on the historical safe driving database to obtain the normalized status risk value and normalized task load value. The current risk scenario identifier is obtained based on the driving scenario data, and the state risk weight and task load weight corresponding to the current risk scenario identifier are obtained. Based on the state risk weight and task load weight, a dual-weight fusion algorithm is used to fuse the normalized state risk value and the normalized task load value to obtain an initial comprehensive risk index. The initial comprehensive risk index is dynamically corrected based on the state-task mismatch coefficient to obtain the driving risk value.

7. A system for recognizing unsafe driving behaviors based on multi-dimensional feature fusion, characterized in that, include: The raw monitoring data acquisition module is used to acquire the driver's raw monitoring data, which includes driver monitoring data, driving scenario data, and driving route data. The driver status assessment value acquisition module is used to acquire a driver status feature vector based on the driver monitoring data, and to acquire a driver status assessment value based on the driver status feature vector. The driving task load index acquisition module is used to acquire the driving task load index based on the driving scenario data and driving path data. The driving risk value acquisition module is used to acquire the driving risk value based on the driver status assessment value and the driving task load index. The warning module is used to compare the driving risk value with a preset risk threshold. If the driving risk value continues to exceed the preset risk threshold for a predetermined period of time, the driver is determined to be in a high-risk state, and a warning intervention is triggered.

8. The unsafe driving behavior recognition system based on multi-dimensional feature fusion according to claim 7, characterized in that, The driver status assessment value acquisition module includes: The driver monitoring data analysis unit is used to acquire visual image data stream, physiological sensor data stream and vehicle operation data stream based on the driver monitoring data. A visual feature sequence acquisition unit is used to perform continuous frame analysis on the visual image data stream to obtain a visual feature sequence, wherein the visual feature sequence includes an eye closure state sequence, a gaze direction sequence, and a head posture angle sequence. A physiological feature sequence acquisition unit is used to perform signal preprocessing on the physiological sensing data stream to obtain a physiological feature sequence, wherein the physiological feature sequence includes a heart rate variability trend sequence and a respiratory rhythm sequence; An operation feature sequence acquisition unit is used to perform sliding window statistics on the vehicle operation data stream to obtain an operation feature sequence, wherein the operation feature sequence includes a steering wheel angle entropy sequence and a longitudinal acceleration jitter sequence; The feature matrix generation unit is used to perform timestamp alignment and normalization on the visual feature sequence, physiological feature sequence and operational feature sequence to generate a synchronously standardized feature matrix. The fusion unit is used to perform multi-scale feature extraction on the synchronous standardized feature matrix to obtain time-domain statistical features, frequency-domain energy spectrum features, and time-frequency-domain complexity features, and fuse them to form a driver state feature vector.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images