Vehicle safety control methods, vehicles and storage media

Abstract

This application provides a vehicle safety control method, a vehicle, and a storage medium. The method, applied in the field of intelligent sensing technology, includes: acquiring a state feature vector characterizing the driver's current driving state, the state feature vector being composed of feature parameters from multiple data sources; determining the deviation degree of the current driving state based on the state feature vector and the driver's driving behavior baseline distribution; identifying the driver's abnormal category when the deviation degree meets preset abnormal conditions; and determining the driver's driving risk level based on the abnormal category and the vehicle's current driving conditions, thereby controlling the vehicle to execute safety control actions matching the driving risk level. This method can improve driving safety by using multi-dimensional data monitoring to detect abnormal driving states adapted to individual driver habits and perform targeted, graded vehicle safety control.

Description

Technical Field

[0001] This application relates to the field of intelligent sensing, and more specifically, to a method for safety control of vehicles, vehicles, and storage media in the field of intelligent sensing. Background Technology

[0002] In related technologies, when it is necessary to carry out safety control for abnormal driver conditions, single-dimensional data or dedicated sensors are usually used to periodically collect driver data to assess the driver's driving status. In this way, appropriate safety controls can be implemented in case of abnormal driving conditions to ensure driving safety.

[0003] However, the use of a single data source or reliance on dedicated sensors in related technologies results in limited data dimensions, low monitoring reliability, high costs, and delayed response. The accuracy of status detection results is also low, making it difficult to meet the safe driving needs of drivers and thus failing to effectively avoid safety risks caused by various driver abnormalities. Improvements are urgently needed. Summary of the Invention

[0004] This application provides a vehicle safety control method, a vehicle, and a storage medium. The method can perform joint monitoring of the four dimensions of face, operation, posture, and vehicle dynamics. By comparing the real-time feature vector with the driver's personal driving behavior benchmark distribution, it determines the degree of deviation of the driver's current driving state from the driving behavior benchmark. It adapts to the driver's personalized driving habits, so that when the deviation is large, it can determine the abnormal driving state. Then, based on the abnormality category and the current driving condition of the vehicle, it can perform targeted and graded vehicle safety control, effectively improving driving safety.

[0005] Firstly, a vehicle safety control method is provided, comprising: acquiring a state feature vector characterizing the current driving state of a driver, the state feature vector being composed of feature parameters from multiple data sources; obtaining the degree of deviation of the current driving state based on the state feature vector and a driving behavior benchmark distribution characterizing the driver's driving habits, so as to identify the driver's abnormal category when the degree of deviation meets preset abnormal conditions; obtaining the driver's driving risk level based on the abnormal category and the vehicle's current driving conditions, so as to control the vehicle to perform safety control actions matching the driving risk level.

[0006] The above technical solution can collect multi-source data features, such as four types of feature parameters: driver vision, steering wheel, seat, and vehicle bus. This enables joint monitoring of the entire state across four dimensions: face, operation, posture, and vehicle dynamics. By comparing real-time feature vectors with the driver's personal driving behavior baseline distribution, the degree of deviation of the driver's current driving state from the driving behavior baseline can be determined. This allows for adaptation to the driver's personalized driving habits. In cases of significant deviation, abnormal driving states can be identified, and targeted, graded vehicle safety control can be implemented based on the type of abnormality and the vehicle's current driving conditions, effectively improving driving safety.

[0007] Optionally, in one embodiment of this application, the feature parameters of the multiple data sources include: visual feature parameters, steering wheel feature parameters, seat feature parameters, and communication bus feature parameters; before obtaining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, the method further includes: dividing the historical state feature vector that meets the preset benchmark conditions into behavior pattern clusters under multiple preset driving modes of the vehicle; constructing behavior feature matrices corresponding to multiple preset driving modes based on the behavior pattern clusters; calculating the covariance between historical visual feature parameters, historical steering wheel feature parameters, historical seat feature parameters, and historical communication bus feature parameters in the corresponding behavior pattern cluster based on the behavior feature matrix; calculating the corresponding feature mean vector based on the behavior feature matrix; and constructing a driving behavior benchmark distribution corresponding to multiple preset driving modes by combining the covariance and the feature mean vector.

[0008] The above technical solution can filter normal historical driving feature vectors, divide them into multiple driving behavior pattern clusters, construct a feature matrix for each driving behavior pattern cluster, and then construct a normal behavior baseline distribution for each driving mode by calculating the covariance and feature mean vector between each pair of multi-source features within the cluster, so as to effectively reduce the probability of false alarms in the scene.

[0009] Optionally, in one embodiment of this application, the step of obtaining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits includes: determining the current driving mode of the vehicle based on the current driving conditions; matching the corresponding target behavior pattern cluster based on the current driving mode; and calculating the deviation degree of the current driving state based on the state feature vector, the covariance and feature mean vector corresponding to the target behavior pattern cluster.

[0010] The above technical solution can determine the current driving mode of the vehicle based on the current driving conditions, and then match the behavior cluster corresponding to the mode. By using the mean vector and covariance matrix of the behavior cluster, the deviation of the state feature vector from the normal benchmark of the mode can be calculated. Through targeted benchmark judgment, the detection accuracy can be effectively improved and the probability of misjudgment can be reduced.

[0011] Optionally, in one embodiment of this application, the step of obtaining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits includes: obtaining the occurrence frequency of multiple preset driving modes from the vehicle's historical driving data, and assigning corresponding weights to the multiple preset driving modes based on the occurrence frequency; constructing a corresponding driving behavior benchmark distribution model by combining the covariance, the feature mean vector, and the weights; and calculating the driving behavior benchmark distribution model using the state feature vector to obtain the deviation degree.

[0012] The above technical solution can statistically analyze the frequency of each driving mode in historical data, assign mode weights to each mode, and combine the mean, covariance, and mode occurrence weights of all modes to construct a global hybrid driving behavior benchmark model, forming a global comprehensive probability benchmark. Then, it can calculate the overall probability deviation of the current state feature vector, which is consistent with the proportion of the driver's real driving habits and compatible with a variety of daily driving scenarios.

[0013] Optionally, in one embodiment of this application, identifying the abnormal category of the driver includes: inputting the state feature vector into a pre-constructed classification model to map the state feature vector to a feature space of a target dimension; in the feature space, using a preset classification hyperplane, calculating the distance from the state feature vector to each classification hyperplane to obtain a first probability vector characterizing the distance between the state feature vector and multiple preset abnormal categories, so as to determine the abnormal category based on the first probability vector.

[0014] The above technical solution allows state features to be input into a classification model, mapped to a feature space, and the distance from the state features to the boundaries of each category to be calculated using the classification model. This yields a first probability vector, which is used to determine the abnormal type of the driving state. This approach uses high-dimensional nonlinear boundaries for precise segmentation, thereby increasing the classification accuracy of small sample abnormal data.

[0015] Optionally, in one embodiment of this application, determining the anomaly category based on the first probability vector includes: inputting the state feature vector into a pre-constructed random forest model, calculating the classification results of multiple decision trees in the random forest model based on the preset anomaly category; statistically analyzing the classification results to obtain the proportion of the number of multiple preset anomaly categories; and obtaining a second probability vector representing the state feature vector corresponding to multiple preset anomaly categories based on the proportion of the number of categories, so as to determine the anomaly category based on the second probability vector and the first probability vector.

[0016] The above technical solution can utilize the random forest model to independently determine the anomaly type using multiple decision trees. Then, by statistically analyzing the proportion of classification results of all trees, a second probability vector with stronger anti-interference and anti-noise capabilities can be generated. Combining the anomaly category identification results of the classification model and the random forest model for the state feature vector can make up for the noise resistance shortcomings of the classification model and improve the accuracy of anomaly category identification.

[0017] Optionally, in one embodiment of this application, determining the anomaly category based on the second probability vector and the first probability vector includes: calculating a corresponding fusion probability vector by combining the first probability vector, the second probability vector, and a first preset weight; determining the anomaly category based on the fusion probability vector; and determining the confidence level of the anomaly category based on the fusion probability corresponding to the anomaly category, so as to obtain the driving risk level by combining the anomaly category and the confidence level.

[0018] The above technical solution can be used to weight and sum the first probability and the second probability according to the preset weights to obtain the fusion probability vector, and select the highest probability as the final anomaly category. The corresponding value is the classification confidence score, which is used for subsequent risk assessment.

[0019] Optionally, in one embodiment of this application, obtaining the driver's driving risk level based on the anomaly category and the vehicle's current driving conditions includes: obtaining the vehicle's current speed and the road type of the current driving route based on the current driving conditions; accumulating the duration of the driver's anomaly based on the time when the deviation meets preset anomaly conditions; matching target risk membership degrees corresponding to the anomaly category, the confidence level, the current speed, the road type, and the duration of the anomaly based on multiple preset risk intervals and multiple preset risk interval mappings; calculating a corresponding comprehensive risk membership degree by combining the target risk membership degree and a second preset weight; and determining the driving risk level based on the comprehensive risk membership degree.

[0020] The above technical solution can combine current vehicle speed, road type, duration of abnormality, abnormality category, and confidence level to match the corresponding risk membership degree. Then, through fuzzy calculation, a comprehensive risk membership degree is obtained, resulting in a driving risk level. By combining the driver's abnormality category and the vehicle's driving environment to comprehensively quantify the risk, the vehicle's subsequent safety control actions can be more in line with the actual situation, thereby improving driving safety.

[0021] Secondly, a vehicle safety control device is provided, comprising: an acquisition module for acquiring a state feature vector characterizing the driver's current driving state, the state feature vector being composed of feature parameters from multiple data sources; a calculation module for obtaining the deviation degree of the current driving state based on the state feature vector and a driving behavior benchmark distribution characterizing the driver's driving habits, so as to identify the driver's abnormal category when the deviation degree meets preset abnormal conditions; and a control module for obtaining the driver's driving risk level based on the abnormal category and the vehicle's current driving conditions, so as to control the vehicle to perform safety control actions matching the driving risk level.

[0022] Optionally, in one embodiment of this application, the feature parameters of the multiple data sources include: visual feature parameters, steering wheel feature parameters, seat feature parameters, and communication bus feature parameters; wherein, the vehicle safety control device further includes: a partitioning module, used to partition historical state feature vectors that meet preset benchmark conditions into behavioral pattern clusters under multiple preset driving modes of the vehicle; a first construction module, used to construct behavioral feature matrices corresponding to multiple preset driving modes based on the behavioral pattern clusters; a first calculation module, used to calculate the covariance between historical visual feature parameters, historical steering wheel feature parameters, historical seat feature parameters, and historical communication bus feature parameters in the corresponding behavioral pattern clusters based on the behavioral feature matrices; a second calculation module, used to calculate the corresponding feature mean vector based on the behavioral feature matrix; and a second construction module, used to combine the covariance and the feature mean vector to construct a driving behavior benchmark distribution corresponding to multiple preset driving modes.

[0023] Optionally, in one embodiment of this application, the calculation module includes: a determining unit, configured to determine the current driving mode of the vehicle based on the current driving condition; a first matching unit, configured to match a corresponding target behavior pattern cluster based on the current driving mode; and a first calculation unit, configured to calculate the deviation degree of the current driving state based on the state feature vector, the covariance and feature mean vector corresponding to the target behavior pattern cluster.

[0024] Optionally, in one embodiment of this application, the calculation module includes: a first acquisition unit, configured to acquire the occurrence frequencies of multiple preset driving modes from the vehicle's historical driving data, and assign corresponding weights to the multiple preset driving modes based on the occurrence frequencies; a construction unit, configured to construct a corresponding driving behavior benchmark distribution model by combining the covariance, the feature mean vector, and the weights; and a second calculation unit, configured to calculate the driving behavior benchmark distribution model using the state feature vector to obtain the degree of deviation.

[0025] Optionally, in one embodiment of this application, the calculation module includes: a mapping unit, configured to input the state feature vector into a pre-constructed classification model to map the state feature vector to a feature space of a target dimension; and a third calculation unit, configured to calculate the distance from the state feature vector to each classification hyperplane in the feature space using a preset classification hyperplane, to obtain a first probability vector characterizing the distance between the state feature vector and multiple preset anomaly categories, so as to determine the anomaly category based on the first probability vector.

[0026] Optionally, in one embodiment of this application, the third calculation unit includes: a calculation subunit, used to input the state feature vector into a pre-constructed random forest model, and calculate the classification results of multiple decision trees in the random forest model based on the preset anomaly categories; a statistics subunit, used to count the classification results to obtain the proportion of the number of multiple preset anomaly categories; and a determination subunit, used to obtain a second probability vector representing the state feature vector corresponding to multiple preset anomaly categories based on the proportion of the number, so as to determine the anomaly category based on the second probability vector and the first probability vector.

[0027] Optionally, in one embodiment of this application, the determining subunit includes: a calculation unit, used to calculate a corresponding fusion probability vector by combining the first probability vector, the second probability vector, and the first preset weight; and a determining unit, used to determine the anomaly category based on the fusion probability vector, and to determine the confidence level of the anomaly category based on the fusion probability corresponding to the anomaly category, so as to obtain the driving risk level by combining the anomaly category and the confidence level.

[0028] Optionally, in one embodiment of this application, the control module includes: a second acquisition unit, configured to acquire the current vehicle speed and the road type of the current driving road based on the current driving conditions; an accumulation unit, configured to accumulate the duration of the driver's abnormality based on the moment when the deviation degree meets a preset abnormality condition; a second matching unit, configured to match the abnormality category, the confidence level, the current vehicle speed, the road type, and the duration of the abnormality with a preset risk membership degree mapped from multiple preset risk intervals, respectively; and a fourth calculation unit, configured to calculate a corresponding comprehensive risk membership degree by combining the target risk membership degree and a second preset weight, and determine the driving risk level based on the comprehensive risk membership degree.

[0029] Thirdly, a vehicle is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to perform the method of the first aspect or any possible implementation thereof.

[0030] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0031] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0032] Figure 1 This is a schematic diagram illustrating the principle of a vehicle safety control method according to an embodiment of this application; Figure 2 This is a flowchart of a vehicle safety control method provided according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating the principle of Mahalanobis distance calculation according to an embodiment of this application; Figure 4 This is a flowchart of a vehicle safety control method according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a vehicle safety control device according to an embodiment of this application; Figure 6 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application.

[0033] Among them, 1-data acquisition layer, 11-camera, 12-steering wheel data acquisition component, 13-seat pressure sensor array, 14-controller area network bus, 2-algorithm processing layer, 21-feature extraction module, 22-behavior modeling module, 23-anomaly detection module, 24-classification and recognition module, 25-risk assessment module, 3-early warning decision layer, 4-action execution layer, 41-voice actuator, 42-visual actuator, 43-tactile actuator, 44-alarm device; 10-vehicle safety control device, 100-acquisition module, 200-computation module, 300-control module; 601-memory, 602-processor, 603-communication interface. Detailed Implementation

[0034] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0035] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0036] With the continuous improvement of automotive intelligence, functional safety has become one of the core considerations in vehicle design and development. As the core operator of the vehicle, the driver's stability and safety directly determine driving safety. Therefore, driver status monitoring technology has become an indispensable component of the functional safety system of intelligent cockpits. The core requirement of driver status monitoring is to capture abnormal driver states in real time and issue timely and effective warnings, thereby avoiding road accidents caused by driver fatigue, drunk driving, emotional instability, and other abnormal situations, and protecting the lives and property of passengers and other road users.

[0037] Currently, various technologies related to driver status monitoring have formed their own application scenarios based on different detection principles and data sources. However, they all have unavoidable limitations and cannot meet the requirements of comprehensiveness, real-time performance, and accuracy of driver status monitoring in actual driving.

[0038] For example, relying on a single visual data source to analyze eye features by collecting driver facial images to determine fatigue status has obvious limitations in its application scope. It cannot cover other abnormal driver states besides fatigue, and is greatly affected by environmental factors such as changes in lighting and facial occlusion. The reliability of monitoring fluctuates significantly, and the early warning response is delayed, making it difficult to deal with sudden abnormal situations. The excessively high false alarm rate also reduces users' trust in the system and may even lead users to actively turn off the monitoring function, rendering the monitoring system ineffective.

[0039] For example, technologies that rely on specialized sensors to monitor specific abnormal states, such as detecting intoxication through alcohol sensors or monitoring health status through physiological sensors, are highly dependent on these sensors. They require additional specialized hardware, which not only increases vehicle manufacturing costs but also reduces the technology's accessibility. Furthermore, these technologies have limited functionality, only monitoring one type of abnormal state and failing to provide comprehensive monitoring of the driver's entire condition. Some technologies also require active driver intervention, making them less practical and unable to provide real-time monitoring during driving, making it difficult to capture dynamically changing abnormal driver states.

[0040] There are also voice analysis-based monitoring technologies that judge the emotional state by collecting the driver's voice signals. However, such technologies are significantly affected by the in-vehicle noise environment and rely on the driver to speak continuously. They cannot play a monitoring role when the driver is driving silently, and they also cannot achieve comprehensive coverage of abnormal states.

[0041] In summary, the aforementioned technologies have significant limitations. They rely on a single data source or dedicated sensors, resulting in limited data dimensions, insufficient functional coverage, low monitoring reliability, high costs, and delayed response. They cannot achieve comprehensive, real-time, and accurate monitoring of various abnormal driver states without increasing hardware costs. This makes it difficult to meet the high standards of functional safety required by intelligent cockpit systems and also fails to effectively mitigate safety risks caused by various driver abnormalities.

[0042] To improve the aforementioned technical problems, this application embodiment can collect multi-source data related to state characteristics, such as data from cameras, steering wheel torque sensors, seat pressure sensors, and controller area network buses, and perform data stitching to determine the degree of deviation between the driver's current driving state and the normal baseline. In the event that the driver's state is abnormal, real-time classification and graded warning of abnormal driver state (fatigue, intoxication, emotional state, drug-induced state) can be performed, effectively reducing the probability of misjudgment and significantly improving driving safety.

[0043] The system architecture of the embodiments of this application will be described next.

[0044] like Figure 1As shown, the vehicle architecture in this application embodiment may include a data acquisition layer 1 for collecting multi-source data related to state characteristics, an algorithm processing layer 2 for calculating the degree of deviation, determining the anomaly category, and determining the driving risk level, an early warning decision layer 3 for determining the risk range and safety control strategy, and an action execution layer 4 for performing safety control actions.

[0045] Among them, the data acquisition layer 1 can collect visual parameters (reuse the existing in-vehicle camera 11 to collect the driver's facial data), steering wheel parameters (the steering wheel data acquisition component 12, which includes a steering torque sensor and a steering angle sensor, collects relevant parameters of the steering wheel), seat parameters (collected through the seat pressure sensor array 13), and communication bus parameters (collecting data such as vehicle speed, throttle opening, braking force, and wheel speed from the controller local area network bus 14).

[0046] The algorithm processing layer 2 can execute the feature extraction module 21, behavior modeling module 22, anomaly detection module 23, classification and recognition module 24, and risk assessment module 25.

[0047] The feature extraction module 21 can extract corresponding features from the collected visual parameters, steering wheel parameters, seat parameters and communication bus parameters to obtain visual feature parameters, steering wheel feature parameters, seat feature parameters and communication bus feature parameters.

[0048] The behavior modeling module 22 can use Gaussian mixture models to establish individual driving behavior benchmarks for drivers.

[0049] The anomaly detection module 23 can detect abnormal deviations by calculating the Mahalanobis distance between the driver's current driving state and the driving behavior benchmark.

[0050] The classification and recognition module 24 can construct and train a classification model and a random forest model to identify abnormal categories of drivers.

[0051] The risk assessment module 25 can use the fuzzy comprehensive evaluation method to determine the driving risk level by combining the vehicle's current driving conditions and the driver's abnormality category.

[0052] The early warning decision layer 3 can determine various thresholds for judgment in advance and set safety control actions for different driving risk levels in advance.

[0053] The action execution layer 4 may include a voice actuator 41 (such as a speaker), a visual actuator 42 (such as a central control screen and a head-up display), a tactile actuator 43 (such as a steering wheel vibration motor), and a warning device 44 (such as a warning light) to interact with the driver and provide warnings.

[0054] Figure 2This is a schematic flowchart of a vehicle control method provided in an embodiment of this application.

[0055] For example, such as Figure 2 As shown, the method 100 includes: In step S1, a state feature vector is obtained to characterize the driver's current driving state. The state feature vector is composed of feature parameters from multiple data sources.

[0056] In practical implementation, the state feature vector can be composed of feature parameters from multiple data sources based on the characteristics of driver state changes (changes from the inside out), which may include: visual feature parameters, steering wheel feature parameters, seat feature parameters, and communication bus feature parameters. Visually, for example, a camera can determine the driver's mental state. Characteristics such as driver fatigue and intoxication will first manifest on the face. Therefore, visual feature parameters collected by the camera can be used to identify the driver's eyes and expressions, and then used for subsequent state judgment. In terms of actions, for example, changes in steering wheel control can determine the driver's ability to control the vehicle, so as to quickly detect when the driver is sick or affected by medication, causing their hands to tremble or their movements to become distorted. In terms of posture, for example, by using relevant sensors in the seat, the driver's ability to control their own posture can be determined, so as to quickly detect any abnormalities in the driver when they are unable to sit still. Furthermore, actual vehicle control data can be obtained through the communication bus to determine the driver's control over the vehicle at the current stage, i.e., driving quality, and to assess the driver's state from an objective perspective.

[0057] The data from the above four aspects can match the characteristics of changes in driver state. For example, if the driver is intoxicated, the camera can first capture data related to the driver's eyes (such as unfocused gaze), the steering wheel can capture abnormal hand movements of the driver (such as hand tremors caused by intoxication), the seat-related sensors can determine whether the driver's sitting posture is abnormal (such as slouching caused by intoxication) by determining data such as the driver's center of gravity and contact surface, and finally, the vehicle control data captured through the communication bus can determine the driver's ability to control the whole vehicle (such as unstable vehicle speed caused by intoxication).

[0058] By linking the data from the above four aspects, the driver's condition can be identified more comprehensively, avoiding the omission of early signals. Moreover, all of the above data can reuse existing sensors in the vehicle, without the need for additional hardware installation, without increasing additional costs, and is easy to promote and apply.

[0059] In actual implementation, the embodiments of this application can collect multi-source data that can be used to identify the driver's driving state, including driver facial image I_t (including eyes, facial expression, and gaze direction) data collected by a camera; steering wheel related data, including torque sequence T_t, steering angle sequence A_t, and steering frequency F_t; seat related data, including pressure distribution matrix P_t and pressure center coordinates (x_t, y_t); and communication bus parameters obtained through the controller local area network bus, including vehicle speed V_t, throttle opening G_t, braking force B_t, and wheel speed W_t.

[0060] From the data collected above, relevant feature parameters can be extracted in the embodiments of this application.

[0061] For example, embodiments of this application can extract visual feature parameters from driver facial image I_t (including eyes, facial expression, and gaze direction) data, including: PERCLOS percentage of time with eyes closed, Blink_freq blink frequency (times / minute), Gaze_offset gaze offset (angle with the front), Face_expression facial expression feature vector (7 categories: anger / disgust / fear / happiness / sadness / surprise / neutrality, taking the 3 categories with the highest confidence), Head_pose head posture angle (pitch, yaw, roll), Pupil_size pupil size change rate, Brow_raise eyebrow raising degree, and Mouth_open mouth opening degree.

[0062] This application embodiment can extract steering wheel feature parameters from relevant steering wheel data, including: Torque_std torque standard deviation (sliding window 1 second), Torque_jerk number of torque mutations (threshold > 0.5 Nm), Steer_freq steering frequency (FFT main frequency component), Steer_angle_std steering angle standard deviation, Steer_speed_mean average steering speed, and Steer_symmetry left and right steering symmetry index.

[0063] This application embodiment can extract seat feature parameters from relevant seat data, including: Pressure_entropy (pressure distribution entropy H=-Σ(p_i·ln(p_i)), Center_shift (pressure center of gravity offset distance relative to the reference position), Posture_jerk (number of posture mutations (pressure distribution change rate > threshold)), and Weight_distribution (body weight distribution characteristics (front-to-back / left-to-right proportions)).

[0064] The embodiments of this application can extract communication bus characteristic parameters from the communication bus parameters, including: Speed_fluctuation standard deviation of speed fluctuation, Acceleration_jerk number of rapid accelerations (acceleration > 3 m / s²), Braking_jerk number of rapid brakings (deceleration > 4 m / s²), Lateral_acc lateral acceleration, and Yaw_rate yaw rate.

[0065] It is understandable that the above-mentioned feature parameters can be divided into instantaneous feature parameters and time window feature parameters according to their physical meaning. Instantaneous feature parameters can reflect the snapshot state at the current moment, while time window feature parameters can reflect the statistical trend over a period of time.

[0066] Among them, the visual feature parameter, camera feature, can be extracted based on the facial image I_t (including eye state, facial expression, and gaze direction) captured by the driver monitoring camera, as follows: (1) PERCLOS (percentage of time with eyes closed) is used to characterize the cumulative fatigue level of the driver over a period of time. It is a time window feature, and the time window can be set to 60 seconds. In this embodiment, the driver's eye opening and closing status can be detected frame by frame within the past 60 seconds, and the percentage of frames with eyes closed is counted. For example, if a total of 600 frames of images are collected within 60 seconds (corresponding to a sampling frequency of 10fps), and 30 frames are detected as being in the closed-eye state, then PERCLOS = 30 / 600 = 5%.

[0067] (2) Blink_freq (blink frequency) is used to reflect the frequency of the driver's eye movements over a period of time. It is a time window feature. The time window can be 60 seconds. In this application embodiment, the number of blinking actions that occurred in the past 60 seconds can be counted (one blink = a complete cycle of the eye from opening to closing and then opening again), and the result is count / minute.

[0068] (3) Gaze_offset (gaze offset) is used to reflect the driver's attention direction at the current moment. It is an instantaneous feature. In this application embodiment, the current frame image can be taken, the pupil center can be located by pupil detection, the eyeball center can be located by iris edge detection, and the angle between the line connecting the two centers and the front (horizontal direction) can be calculated in degrees.

[0069] (4) Face_expression (facial expression feature), used to reflect the driver's facial expression state at the current moment, is determined based on the current frame image, does not involve time window accumulation, is calculated independently for each frame, and is an instantaneous feature. In this embodiment, the current frame facial image can be cropped to a fixed size (e.g., 48x48 pixels) and input into a pre-trained facial expression convolutional neural network model (the training dataset is a facial expression recognition dataset, including anger, disgust, fear, happiness, sadness, surprise, neutrality, etc.). The facial expression convolutional neural network model performs forward inference on the current frame image and outputs the probability values ​​of different expressions (e.g., for the above seven expressions, the output probability values ​​can be [0.05, 0.02, 0.01, 0.80, 0.05, 0.02, 0.05], indicating an 80% probability of "happiness"). The three probability values ​​with the highest confidence are taken as features.

[0070] (5) Head_pose (head pose angle) is used to reflect the driver's current head orientation. It is an instantaneous feature. In this application embodiment, the three rotation angles of the head relative to the camera can be estimated by solving the Perspective-n-Point problem based on the 3D face key point model (such as the 68 key points of Dlib). The unit is degrees.

[0071] (6) Pupil_size (pupil size change rate) is used to reflect the instantaneous change trend of the driver's pupil size. It is an instantaneous feature. In this embodiment, the pupil diameter (pixels) can be detected in the current frame, divided by the face width (pixels) for normalization (to eliminate the influence of distance change), and then subtracted from the normalized pupil diameter of the previous frame to obtain the change rate.

[0072] (7) Brow_raise (degree of eyebrow raising) is used to reflect the degree of eyebrow raising of the driver at the current moment. It is an instantaneous feature. In the embodiments of this application, the key points of eyebrows and eyes can be located in the current frame, the vertical distance from the eyebrow key point to the corresponding eye key point can be calculated, and then normalized by dividing by the face height.

[0073] (8) Mouth_open (mouth opening degree) is used to reflect the degree of mouth opening of the driver at the current moment. It is an instantaneous ratio and an instantaneous feature. In the embodiments of this application, the key points of the upper lip and lower lip can be located in the current frame, the vertical distance between them can be calculated, and then normalized by dividing by the face height.

[0074] The steering wheel feature parameters are extracted based on relevant data collected from the steering wheel (torque sequence T_t, steering angle sequence A_t, steering frequency F_t), as detailed below: (1) Torque_std (torque standard deviation) is used to reflect the fluctuation range of steering operation over a period of time. It is a time window feature. The time window can be 1 second (i.e. the most recent 50 sampling points). In this application embodiment, 50 torque sampling values ​​within the most recent 1 second can be taken and the standard deviation of these 50 values ​​can be calculated. The unit is Nm.

[0075] (2) Torque_jerk (number of torque mutations) is used to reflect the frequency of mutations in steering operations over a period of time. It is a time window feature. The time window can be 60 seconds (i.e., the most recent 3000 sampling points). In this embodiment, the torque sequence within the most recent 60 seconds can be traversed to count the number of times the absolute value of the difference between two adjacent sampling values ​​exceeds 0.5 Nm, and then converted into the number of times / minute.

[0076] (3) Steer_freq (steering frequency) is used to reflect the dominant frequency of steering operation (such as low-frequency left and right swaying that may occur when fatigued). It is a time window feature. The time window can be 10 seconds (i.e. the most recent 500 sampling points). In this application embodiment, FFT (Fast Fourier Transform) can be performed on the steering angle sequence within the most recent 10 seconds to find the frequency component with the largest amplitude in the spectrum, in Hz.

[0077] (4) Steer_angle_std (Steering angle standard deviation) is used to reflect the dispersion of steering angle over a period of time. It is a time window feature. The time window can be 1 second (i.e., the most recent 50 sampling points). In this application embodiment, the 50 steering angle sampling values ​​within the most recent 1 second can be taken and the standard deviation can be calculated. The unit is degrees.

[0078] (5) Steer_speed_mean (average steering speed) is used to reflect the steering operation speed over a period of time. It is a time window feature. The time window can be 1 second (i.e., the most recent 50 sampling points). In this application embodiment, the first difference (i.e., the rate of change of angle) of the steering angle sequence in the most recent 1 second can be calculated to obtain 49 angular velocity values ​​and calculate their average value in degrees / second.

[0079] (6) Steer_symmetry (left-right steering symmetry) is used to reflect the degree of left-right symmetry of the driver's long-term driving habits. It is a time window feature, and the time window can be 60 seconds. In the embodiment of this application, the sum of the left turn and right turn amplitudes can be counted separately in the steering angle sequence of the most recent 60 seconds, and the ratio of the two can be calculated. Among them, a ratio of 1.0 indicates complete symmetry, and a ratio deviating from 1.0 indicates the existence of steering preference.

[0080] Seat pressure characteristic parameters are extracted based on relevant data collected from the seat (pressure distribution matrix P_t, pressure centroid coordinates (x_t, y_t)), as follows: (1) Pressure_entropy, used to reflect the balance of sitting posture, is an instantaneous characteristic. In this embodiment, the pressure readings of 16 sensors at the current moment can be taken, and the pressure value of each sensor can be divided by the sum of the pressure values ​​of all sensors to obtain the pressure ratio p_i of each sensor (i=1 to 16). Then, the information entropy H = -(p_1) is calculated. ln(p_1) + p_2 ln(p_2) + ... + p_16 ln(p_16)). Among them, when the driver's sitting posture is uniform and stable, the pressure is evenly distributed on each sensor, and the entropy value is high; when the driver's body is tilted to one side or leans forward, the pressure is concentrated on some sensors, and the entropy value decreases.

[0081] (2) Center_shift (pressure center of gravity offset distance) is used to reflect the degree to which the driver's current body center of gravity deviates from the normal position, and is an instantaneous feature. The x_i and y_i (i=1 to 16) involved in this feature calculation are the physical installation position coordinates of the i-th pressure sensor on the seat (calibrated at the factory, fixed, unit: cm), which are not the same set of variables as the pressure center of gravity coordinates (x_t, y_t) mentioned above. This application embodiment can extract the pressure values ​​w_i of the 16 sensors at the current moment and their installation positions (x_i, y_i), using the formula x_c = (w1·x1 + w2·x2 + ... + w 16 ·x 16 ) / (w1+ w2+... + w 16 Calculate the x-axis coordinate of the pressure center of gravity, and similarly calculate the y-axis coordinate y_c; then calculate the Euclidean distance between the pressure center of gravity (x_c, y_c) and the driver's personal reference position (x0, y0), in cm. The reference position (x0, y0) is the average pressure center of gravity obtained during the first 7 days of model learning.

[0082] (3) Posture_jerk (number of posture changes) is used to reflect the frequency of sudden changes in the driver's body posture within a period of time. It is a time window feature. The time window can be 30 seconds (i.e. the most recent 600 frames). In this embodiment, the Euclidean distance between the 16-dimensional pressure distribution vector of the current frame and the previous frame can be calculated frame by frame within the most recent 30 seconds. When the distance exceeds the preset threshold, it is counted as one posture change. The total number is counted and converted into the number of times / minute.

[0083] (4) Weight_distribution (weight distribution characteristics) is used to reflect whether the driver's current weight is evenly distributed on the seat. It is an instantaneous characteristic. In this embodiment, the 16 sensors can be divided into 8 in the front row and 8 in the back row according to their positions. The total pressure in the front row and the total pressure in the back row are summed respectively, and the front-to-back ratio is calculated. Similarly, they can be divided into 8 on the left side and 8 on the right side, and the left-to-right ratio is calculated.

[0084] The characteristic parameters of the controller area network (CLAN) bus are extracted based on relevant parameters obtained from the CLAN bus (vehicle speed V_t, throttle opening G_t, braking force B_t, wheel speed W_t), as follows: (1) Speed_fluctuation (speed fluctuation standard deviation) is used to reflect the degree of fluctuation of vehicle speed over a period of time. It is a time window feature. The time window can be 10 seconds (i.e. the most recent 500 sampling points). In this application embodiment, 500 vehicle speed sampling values ​​within the most recent 10 seconds can be taken to calculate the standard deviation, with the unit being km / h.

[0085] (2) Acceleration_jerk (number of rapid accelerations) is used to reflect the frequency of rapid acceleration within a period of time. It is a time window feature. The time window can be 60 seconds. In this application embodiment, the first-order difference of the vehicle speed sequence can be obtained to obtain the acceleration sequence. The number of times the absolute value of acceleration exceeds 3m / s² is counted and converted into the number of times / minute.

[0086] (3) Braking_jerk (number of emergency brakings), which reflects the frequency of recent emergency brakings. It is a time window feature, and the time window can be 60 seconds. In this embodiment, it can count the number of times the acceleration is less than -4m / s², which is converted into number of times / minute. This feature reflects the frequency of recent emergency brakings.

[0087] (4) Lateral_acc (lateral acceleration) is used to reflect the current lateral motion state of the vehicle. It is an instantaneous feature. In this embodiment, the lateral acceleration sensor reading at the current moment can be obtained, and the unit is m / s².

[0088] (5) Yaw_rate: This reflects the current rotation rate of the vehicle around the vertical axis. It is an instantaneous feature. In this embodiment, the yaw rate sensor reading at the current moment can be obtained, and the unit is degrees / second.

[0089] In this embodiment, the extracted dimensional features can be concatenated into a single state feature vector, which can then be used as a whole for subsequent Mahalanobis distance calculation. Based on the above data, this embodiment can form the final feature vector: x_t ∈ R^23 (8+6+4+5=23 dimensions).

[0090] It is understandable that for some data, such as the number of rapid accelerations, data may not be collected within the time window. In this embodiment, the spliced ​​state feature vector is used as a whole in the subsequent calculation of the degree of deviation. Even if the feature parameter is 0 at some time, the frequency of the feature being 0 during normal driving has been statistically analyzed in the subsequent benchmark used during the learning phase (i.e., the value of 0 often occurs during normal driving), so it will not affect the actual use.

[0091] In step S2, the deviation degree of the current driving state is obtained based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, so as to identify the abnormal category of the driver when the deviation degree meets the preset abnormal conditions.

[0092] As one possible implementation, embodiments of this application can utilize state feature vectors and a driving behavior baseline distribution constructed using historical driver data to determine the degree of deviation between the current driving state and the normal driving state, thereby determining an abnormal state. For example, embodiments of this application can set a certain degree threshold; if the value corresponding to the degree of deviation is greater than the degree threshold, the driver's driving state is determined to be abnormal. In this case, embodiments of this application can further utilize classification models or other methods to identify the driver's abnormal category, so as to subsequently combine the abnormal category with the vehicle's current driving conditions to determine the corresponding driving risk level, thereby implementing appropriate safety controls.

[0093] The calculation of the degree of deviation, the identification of abnormal categories, and the determination of driving risk levels will be described in detail below.

[0094] Optionally, in one embodiment of this application, before obtaining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, the method further includes: dividing the historical state feature vector that meets preset benchmark conditions into behavior pattern clusters under multiple preset driving modes of the vehicle; constructing behavior feature matrices corresponding to multiple preset driving modes based on the behavior pattern clusters; calculating the covariance between historical visual feature parameters, historical steering wheel feature parameters, historical seat feature parameters, and historical communication bus feature parameters in the corresponding behavior pattern cluster based on the behavior feature matrices; calculating the corresponding feature mean vector based on the behavior feature matrix; and constructing a driving behavior benchmark distribution corresponding to multiple preset driving modes by combining the covariance and the feature mean vector. The driving behavior benchmark distribution is used to characterize the distribution of multi-source data features under normal driving conditions, with different driving behavior benchmark distributions corresponding to different driving conditions or driving modes.

[0095] In some embodiments, the basis for constructing the driving behavior baseline distribution is a 23-dimensional feature vector, which is formed by splicing together 8 visual features extracted by the camera, 6 control features of the steering wheel, 4 pressure features of the seat, and 5 vehicle features of the controller area network bus in a fixed order. In this application embodiment, the entire 23-dimensional vector is modeled as a whole to capture the inherent relationship between different features (such as the coordinated changes of eye features and steering features when driving while fatigued).

[0096] This application embodiment can divide historical state feature vectors that meet preset benchmark conditions, such as normal driving data of drivers collected in the first 7 days of a new car (approximately 1000 sample points, each sample point being a 23-dimensional feature vector), into multiple behavioral pattern clusters under preset driving modes. This can be achieved through the EM (Expectation Maximization) iterative algorithm (clustering these sample points into K clusters, calculating the center (μ_k), distribution range (Σ_k), and proportion (π_k) of each cluster), and finally clustering to form K behavioral pattern clusters (such as typical normal driving modes such as high-speed cruise mode, urban following mode, parking waiting mode, lane changing and overtaking mode, and low-speed driving mode).

[0097] Based on each behavior pattern cluster, this embodiment can construct a corresponding behavior feature matrix, which organizes all historical 23-dimensional feature vectors within the cluster into a matrix form, serving as the basis for parameter calculation. Based on this behavior feature matrix, the covariance between each pair of the four types of features (visual, steering wheel, seat, and communication bus) within the corresponding cluster can be calculated. This covariance describes the degree of correlation between different features. Simultaneously, a corresponding feature mean vector is calculated, containing 23 values ​​representing the average level of each of the 23 features within the pattern cluster. Combining the calculated covariance and feature mean vector, a unique driving behavior benchmark distribution can be constructed for each preset driving mode. These benchmark distributions collectively constitute a personalized driving behavior benchmark system for the driver, providing a standard reference for subsequent deviation calculations. During parameter estimation, relevant parameters can be updated periodically, such as every 7 days (through online learning), continuously adapting to the slow changes in driver habits. During the learning period, a group benchmark based on a large amount of driver data is used as a temporary substitute for the general model.

[0098] The EM (Expectation-Maximization) iterative algorithm automatically divides samples into K clusters and calculates the covariance and feature mean vector of all samples in each cluster. The iterative learning process of the EM algorithm includes: the E-step (expectation step), where, given the current K sets of parameters, for each training sample, the probability of it belonging to each Gaussian component is calculated; and the M-step (maximization step), where, based on the probability values ​​obtained in the E-step, the K sets of parameters are updated, including updating the weights (the larger the sum of probabilities assigned to a component, the larger its weight), updating the mean (the average value of the samples corresponding to a component, which is the new mean vector), and updating the covariance (the degree of dispersion of the samples corresponding to a component, which is the new covariance matrix). When the change in the log-likelihood function value (an indicator of the model's fit to the data) between two consecutive iterations is less than 0.00001, the parameters are considered to have converged, and the iteration stops. If convergence is not achieved after 100 iterations, the iteration is also forcibly stopped to avoid infinite loops.

[0099] The determination of the mixture component K can be started by trying different values ​​starting with K=2. For each K value, a Gaussian Mixture Model (GMM) is trained, and the AIC (Akaike Information Criterion) value is calculated. AIC considers both model goodness of fit and model complexity (number of parameters). A smaller AIC value indicates a better balance between goodness of fit and simplicity. Finally, the K value with the smallest AIC is selected.

[0100] Optionally, in one embodiment of this application, the deviation degree of the current driving state is obtained based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, including: determining the current driving mode of the vehicle based on the current driving conditions; matching the corresponding target behavior pattern cluster based on the current driving mode; and calculating the deviation degree of the current driving state based on the state feature vector, the covariance and feature mean vector corresponding to the target behavior pattern cluster.

[0101] In actual implementation, the embodiments of this application can determine the current driving mode of the vehicle (such as highway cruising, urban following, etc.) based on the current driving conditions (such as vehicle speed, road type, etc.), and then match the most suitable target behavior pattern cluster from multiple constructed behavior pattern clusters based on the current driving mode.

[0102] The covariance and feature mean vector corresponding to the target behavior pattern cluster are Σ_k (a 23x23 covariance matrix that describes the fluctuation range of each of the 23 features in the cluster and the degree of pairwise correlation) and μ_k (a 23-dimensional mean vector that represents the average level of the 23 features in the cluster) obtained by the EM algorithm for the pattern cluster.

[0103] The core of calculating the degree of deviation is the Mahalanobis distance, and the formula is: D 2 =(x_t-μ_{k*})^T×Σ_{k*} -1 ×(x_t-μ_{k*}), where x_t is the 23-dimensional state feature vector at the current time, μ_{k*} is the feature mean vector of the target behavior pattern cluster, and Σ_{k*} -1 It is the inverse matrix of the covariance matrix of the target behavior pattern cluster. Its function is to standardize the deviation between real-time features and mean vector, unify the fluctuation scale of different features, and eliminate the redundant calculation caused by the correlation between features.

[0104] For example, the Mahalanobis distance can be calculated by subtracting the 23 components of the mean vector mu from the 23 components of the state feature vector x_t, resulting in a 23-dimensional difference vector (representing the degree to which each feature deviates from the normal average). A linear transformation is then performed on the difference vector using the inverse of the covariance matrix (to eliminate the influence of differences in fluctuation amplitude and correlation between different dimensions, making the contribution of dimensions with large fluctuations smaller and the contribution of dimensions with small fluctuations larger). Finally, the inner product of the transformed vector and itself is calculated to obtain a single numerical value D. 2 The entire calculation process takes a 23-dimensional vector (the currently collected features) as input, performs a comprehensive calculation, and outputs a scalar value (Mahaviran distance), which is a combined distance value used to compare with a threshold to determine whether an anomaly is detected. Figure 3 As shown in the figure, the elliptical structure represents the equiprobability density surface (similar to contour lines in geography) of K Gaussian components in 23-dimensional space. Each elliptical structure represents a typical region of a normal driving behavior pattern in the 23-dimensional feature space. When the driver's state feature vector falls inside a certain elliptical structure, it indicates that the current state belongs to the normal driving behavior represented by that pattern; if it falls far outside all circles, it is judged as abnormal.

[0105] The inverse of the 23×23 covariance matrix can be solved using the Cholesky decomposition method, which decomposes the symmetric positive definite matrix into the product of a lower triangular matrix and its transpose, thus transforming the matrix inversion into solving a system of two trigonometric equations. This method is computationally efficient and has good numerical stability.

[0106] The final calculated D 2 This is a single scalar value; the larger the value, the further the current driving state deviates from the driving behavior baseline distribution corresponding to the target behavior pattern cluster, i.e., the greater the degree of deviation; D 2 The smaller the value, the closer it is to normal driving conditions.

[0107] This application embodiment can continuously record Mahalanobis distance values ​​(3600 data points) over a period of time, such as the most recent 60 minutes, and calculate the mean and standard deviation of this data set. The threshold is set to the mean plus three times the standard deviation. Under normal distribution conditions, data exceeding three times the standard deviation have an extremely low probability of occurrence (less than 0.3%), and therefore are reasonably considered anomalies.

[0108] In certain special scenarios (such as switching the driving mode from highway mode to city road mode), the currently most matching mode cluster may change to another mode cluster.

[0109] Optionally, in one embodiment of this application, the deviation degree of the current driving state is obtained based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, including: obtaining the occurrence frequency of multiple preset driving modes from the vehicle's historical driving data, and assigning corresponding weights to the multiple preset driving modes based on the occurrence frequency; constructing a corresponding driving behavior benchmark distribution model by combining the covariance, feature mean vector and weights; and calculating the driving behavior benchmark distribution model using the state feature vector to obtain the deviation degree.

[0110] The baseline distribution model of driving behavior is constructed using a Gaussian mixture model (GMM), and its expression is: p(x)=π_1·N(x|μ_1,Σ_1)+π_2·N(x|μ_2,Σ_2)+...+π_K·N(x|μ_K,Σ_K).

[0111] This application embodiment can statistically analyze the occurrence frequency p(x) of multiple preset driving modes (i.e., K behavior pattern clusters, such as the behavior pattern clusters corresponding to highway cruise mode, city following mode, parking waiting mode, lane changing and overtaking mode, and low-speed driving mode) from the vehicle's historical driving data. Based on the occurrence frequency, each mode is assigned a corresponding weight π_k, and the sum of all K weights is 1 (100%). The higher the occurrence frequency, the greater the weight, and the mode with the largest weight is the normal driving mode that the driver is most often in.

[0112] Subsequently, by combining the covariance (Σ_k, the covariance matrix of the k-th mode, representing the magnitude of fluctuation and correlation between each pair of the 23 features in the corresponding mode. The values ​​on the diagonal represent the fluctuation range (variance) of each feature itself, and the values ​​on the off-diagonal represent the correlation (covariance) between two different features. For example, when fatigued, the blinking frequency and steering standard deviation may increase simultaneously, and this correlation is reflected in the off-diagonal elements of the covariance matrix), the feature mean vector (μ_k, representing the average level of each of the 23 features in the k-th normal driving mode. For example, in highway cruising mode, the mean of steering angle may be close to 0 (straight driving), while in urban following mode, the mean of steering angle may be larger (frequent small corrections in direction)) and the assigned weights (π_k, representing the frequency of the k-th mode during normal driving), a complete GMM driving behavior baseline distribution model is constructed. This model integrates the baseline parameters of all normal driving modes to form a global driving behavior baseline distribution.

[0113] The deviation is calculated by substituting the real-time 23-dimensional state feature vector x_t into the GMM model and calculating the probability p(x) of this vector in the global baseline distribution. The higher the probability, the closer the current driving state is to the driver's normal driving habits, and the smaller the deviation; the lower the probability, the further the deviation from the normal driving mode, and the greater the deviation. Furthermore, Mahalanobis distance can be used to further quantify the deviation, ensuring the accuracy of the judgment.

[0114] In addition, such as Figure 3 As shown, embodiments of this application can use an adaptive threshold to determine anomalies instead of a fixed threshold. For example, the Mahalanobis distance D can be calculated once per second. 2 3600 D in the last 60 minutes 2 The values ​​form a sliding window, and the average value μ_D within this window is calculated. 2 and standard deviation σ_D 2 The threshold is set to threshold=μ_D 2 + 3×σ_D 2 If the current Mahalanobis distance exceeds the threshold, it is considered to be outside the reasonable fluctuation range and is judged as abnormal, further ensuring the scientific nature and adaptability of the deviation determination.

[0115] In practical applications, taking cold start as an example, this embodiment of the application can adopt a phased transition strategy when there is not enough historical data within the sliding window to calculate a reliable adaptive threshold: In the first phase (the first 5 minutes, when historical data is insufficient), a global default threshold can be set based on empirical values ​​derived from a large number of pre-calibrated driver datasets.

[0116] In the second stage (5 to 30 minutes), linear interpolation can be used to smoothly transition between the default threshold and the adaptive threshold calculated in real time.

[0117] In the third stage (30 minutes later), historical data is sufficient and the system no longer relies on default values.

[0118] In summary, the embodiments of this application can utilize the degree of deviation for anomaly determination. For example, when D 2 When the threshold is exceeded, it is considered an anomaly, triggering subsequent anomaly category classification and identification. Simultaneously, the anomaly start time is recorded for calculating the duration. When D... 2 When the value is ≤threshold, it is considered normal driving, and monitoring continues while the behavioral baseline model is slowly updated to adapt to minor changes in driving habits.

[0119] Optionally, in one embodiment of this application, identifying the abnormal category of the driver includes: inputting a state feature vector into a pre-built classification model to map the state feature vector to a feature space of a target dimension; in the feature space, using a preset classification hyperplane, calculating the distance from the state feature vector to each classification hyperplane to obtain a first probability vector characterizing the distance between the state feature vector and multiple preset abnormal categories, so as to determine the abnormal category based on the first probability vector.

[0120] In some embodiments, the present application may employ a classification model, such as an SVM (Support Vector Machine) model, to classify and identify anomaly categories.

[0121] The purpose of the classification model is to find the optimal classification hyperplane in the feature space, maximizing the margin between different anomaly categories. In this embodiment, the preset anomaly categories can be: fatigue, drunk driving, emotional abnormality, and drug influence. For the classification problem of these four types of anomalies, the classification model can adopt a one-to-one strategy, that is, train a binary classifier for every two categories, requiring a total of 6 binary classifiers to be trained (fatigue vs. drunk driving, fatigue vs. emotional abnormality, fatigue vs. drug abnormality, drunk driving vs. emotional abnormality, drunk driving vs. drug abnormality, emotional abnormality vs. drug abnormality).

[0122] The training data for the classification model can include normal driving data and the four types of abnormal data mentioned above, for a total of five categories, with the ratio of normal data to abnormal data being approximately 3:1. Since there are relatively few abnormal data, SMOTE (Synthetic Minority Oversampling) is used to artificially generate some abnormal samples to ensure that the number of samples in the five categories is basically balanced, thus preventing the model from favoring the category with more samples.

[0123] Meanwhile, the classification model is configured with specific parameters to ensure classification performance: the kernel function uses RBF (Radial Basis Function), which maps the input state feature vector to the feature space of the target dimension. This mathematical transformation makes it easier to distinguish different anomaly categories that might otherwise be intertwined; the penalty parameter C=5.0 controls the tolerance of the classification model for classification errors. The larger the C value, the stricter the classification requirements of the model for training samples, and vice versa, a certain degree of classification error is allowed. C=5.0 is a balance point determined through 5-fold cross-validation experiments, which can balance classification accuracy and model generalization ability; the kernel parameter γ=0.1 controls the influence range of the RBF kernel function. The larger the γ, the smaller the influence range of a single training sample and the more complex the model, and vice versa, the smoother the model. γ=0.1 is also the optimal value determined through cross-validation.

[0124] In this embodiment, the 23-dimensional state feature vector x_t that triggers the anomaly at the current moment can be used as input and fed into a pre-trained classification model. The classification model first maps the 23-dimensional state feature vector to the target dimension feature space using a preset RBF kernel function. Then, in this feature space, using preset classification hyperplanes corresponding to six pre-trained binary classifiers, the distance from the current state feature vector to each classification hyperplane is calculated. Finally, these distance values ​​are converted into four probability values ​​through Platt scaling, forming a first probability vector. This probability vector represents the degree of matching between the current state feature vector and the four preset anomaly categories (the closer the distance, the higher the corresponding probability value). Based on this first probability vector, the system can determine the driver's anomaly category.

[0125] Optionally, in one embodiment of this application, determining the anomaly category based on the first probability vector includes: inputting the state feature vector into a pre-constructed random forest model, calculating the classification results of multiple decision trees in the random forest model based on preset anomaly categories respectively; statistically analyzing the classification results to obtain the proportion of multiple preset anomaly categories; and obtaining a second probability vector representing the state feature vector corresponding to multiple preset anomaly categories based on the proportion of categories, so as to determine the anomaly category based on the second probability vector and the first probability vector.

[0126] In other embodiments, anomaly categories can be classified and identified using a random forest model.

[0127] The random forest module can improve classification accuracy and stability by constructing multiple decision trees and combining their judgments. It can work in conjunction with classification models to obtain more accurate results in identifying anomaly categories.

[0128] In practical applications, this embodiment can simultaneously construct multiple decision trees, such as 100 independent decision trees, based on computational limitations and accuracy requirements. Each decision tree is trained on a random subset of the training data. Each decision tree has a maximum allowed number of branches, such as 10 branches, to limit its depth, prevent overgrowth, and avoid learning noise from the training data, thus preventing overfitting. Each tree randomly selects multiple features, such as 5 features. When splitting a node in each decision tree, 5 features are randomly selected from 23 feature dimensions for consideration. This allows different decision trees to focus on different feature combinations, effectively enhancing the diversity of the entire random forest model and avoiding the impact of single feature bias on the classification results.

[0129] It is important to note that when a node in a decision tree contains fewer than a certain number of samples, such as 5, the splitting operation of that node should be stopped to prevent overfitting of the model and ensure the generalization ability of the decision tree to adapt to anomaly recognition in different driving scenarios.

[0130] The training data for the random forest model can include normal driving data and four preset abnormal data categories (fatigue, drunk driving, emotional abnormality, and drug influence), for a total of five categories. The ratio of normal data to abnormal data is approximately 3:1. Since abnormal data is relatively scarce, to avoid the model biasing towards the category with a larger number of samples, SMOTE (Synthetic Minority Oversampling) is used to artificially generate some abnormal samples, ensuring a relatively balanced sample size across the five categories and guaranteeing the fairness of model training and classification accuracy.

[0131] In actual implementation, this embodiment can input the state feature vector into a pre-constructed random forest model; then, calculate the classification results of 100 independent decision trees in the model based on four preset anomaly categories (each decision tree outputs a classification judgment); next, count the classification results of all decision trees and calculate the proportion of classification results corresponding to each preset anomaly category (i.e., the proportion of the number of times a certain anomaly category is judged by the decision tree to the total number of judgments of the 100 decision trees); finally, based on this proportion, obtain a second probability vector, which represents the degree of matching between the current state feature vector and multiple preset anomaly categories. Combining the second probability vector with the first probability vector output by the classification model, the driver's anomaly category is finally determined.

[0132] Optionally, in one embodiment of this application, determining the anomaly category based on the second probability vector and the first probability vector includes: calculating a corresponding fusion probability vector by combining the first probability vector, the second probability vector, and a first preset weight; determining the anomaly category based on the fusion probability vector; and determining the confidence level of the anomaly category based on the fusion probability corresponding to the anomaly category, so as to obtain the driving risk level by combining the anomaly category and the confidence level.

[0133] In the embodiments of this application, the first probability vector and the second probability vector are both probability distribution vectors corresponding to multiple abnormal categories. Each value in the vector can correspond to abnormal categories such as fatigue, drunk driving, emotional abnormality, and drug effects, and the sum of the probability values ​​in each vector is 1.

[0134] In this embodiment, the first probability vector and the second probability vector can be weighted and fused. For example, the first probability vector output by the classification model is p_svm=[0.6,0.2,0.15,0.05] (corresponding to fatigue, intoxication, mood, and medication, respectively), and the second probability vector output by the random forest model is p_rf=[0.3,0.5,0.15,0.05]. They can be weighted and fused with certain weights, such as multiplying each probability of the first probability vector by 0.6, and the second probability vector... Each probability is multiplied by 0.4, and then summed to obtain the following probabilities: fatigue probability: 0.6×0.6+0.4×0.3=0.36+0.12=0.48; intoxication probability: 0.6×0.2+0.4×0.5=0.12+0.20=0.32; emotional probability: 0.6×0.15+0.4×0.15=0.09+0.06=0.15; and drug probability: 0.6×0.05+0.4×0.05=0.03+0.02=0.05. The final fusion result is p_final=[0.48,0.32,0.15,0.05]. In this embodiment, the category with the highest probability can be selected as the final classification result: fatigue (probability 0.48), with a confidence level of 0.48 (i.e., 48% certainty of judging it as fatigue).

[0135] In this embodiment, weights can be assigned based on the characteristics of the model. Classification models, which typically have higher classification accuracy in high-dimensional spaces and are more sensitive to outlier data, can be assigned higher weights, such as 0.6. Although the accuracy of a single tree in a random forest is slightly lower, it has better generalization ability and robustness through the collective decision-making of multiple trees. In this embodiment, the optimal weight combination determined through 5-fold cross-validation experiments during the offline training phase can be used directly during deployment.

[0136] In step S3, the driver's driving risk level is obtained based on the anomaly category and the vehicle's current driving condition, so as to control the vehicle to perform safety control actions that match the driving risk level.

[0137] The embodiments of this application can determine the corresponding driving risk level by combining the anomaly category and the current driving conditions of the vehicle, thereby carrying out corresponding safety controls.

[0138] For example, at a low-risk level, the driver may have a slight deviation from the normal driving state, which does not significantly affect driving safety. Only a mild reminder is needed to correct the state. In this embodiment, the driver may be reminded to pay attention to their driving state in both Chinese and English, so as to avoid interfering with normal driving. The vehicle can automatically record the current driving state and warning information to the local log without interrupting the driver's normal driving operation, thus balancing reminder and driving continuity.

[0139] At the medium-risk level, the corresponding driver's abnormal condition is quite obvious, posing a potential threat to driving safety. It is necessary to urge the driver to take measures through multi-dimensional warnings. The embodiment of this application can clearly announce "Please stop and rest" to strengthen the reminder to the driver to stop driving in time and alleviate the abnormal condition; the vehicle's central control screen displays a yellow warning icon to intuitively convey the warning information and attract the driver's attention; "Resumption recommended" is displayed on the windshield head-up display interface, without the driver having to look down to check, improving the timeliness of the warning; the vehicle can automatically record logs and report abnormal information to the cloud platform, which is convenient for background monitoring and data traceability, while not forcibly interrupting driving, leaving time for the driver to adjust independently.

[0140] At high-risk levels, the corresponding driver's condition is severely abnormal (such as severe fatigue or severe intoxication), seriously threatening driving safety. Immediate and strong intervention measures are required to prevent accidents. This application embodiment can repeatedly broadcast "Stop immediately!" to strongly urge the driver to stop urgently; the central control screen displays a red warning icon that flashes continuously, and the head-up display simultaneously displays "Danger! Stop immediately!", strengthening emergency warnings through multiple visual channels; the steering wheel vibrates at a 20Hz frequency and 70% intensity, forcibly reminding the driver through tactile feedback to prevent the driver from missing the warning due to impaired consciousness; the interior LED lights flash red to further enhance the warning atmosphere inside the vehicle, reminding passengers to pay attention to the driver's condition; the IPB (Integrated Power Brake) pre-fill function is activated to prepare for active braking in advance. If the driver does not respond in time, active braking can be quickly triggered to reduce the risk of an accident; an emergency rescue call is automatically dialed via a 4G / 5G T-Box, simultaneously sending information such as the vehicle's real-time location and the driver's abnormal condition to ensure rapid arrival of rescue personnel and maximize the safety of passengers.

[0141] Optionally, in one embodiment of this application, the driver's driving risk level is obtained based on the anomaly category and the vehicle's current driving conditions, including: obtaining the vehicle's current speed and the road type of the current driving route based on the current driving conditions; accumulating the duration of the driver's anomaly based on the time when the deviation meets preset anomaly conditions; matching the corresponding target risk membership degree based on multiple preset risk intervals and preset risk interval mappings, respectively, for the anomaly category, confidence level, current speed, road type, and anomaly duration; calculating the corresponding comprehensive risk membership degree by combining the target risk membership degree and a second preset weight; and determining the driving risk level based on the comprehensive risk membership degree.

[0142] As one possible approach, embodiments of this application can combine current driving environment conditions and use fuzzy comprehensive evaluation method to comprehensively assess the specific driving risk level (low / medium / high), providing a decision-making basis for subsequent early warning strategies.

[0143] For example, the driving risk level is divided into three intervals: low risk [0, 0.4], medium risk [0.4, 0.7], and high risk [0.7, 1.0]. The assessment process revolves around five factors: anomaly category, confidence level, anomaly duration, current vehicle speed, and road type, and is combined with a second preset weight to complete the comprehensive calculation.

[0144] This application embodiment can obtain the vehicle's current speed and the road type of the current driving route based on the current driving conditions. The road types are ranked by danger level as follows: highway > urban expressway > urban road. The current speed is ranked by danger level as follows: the higher the speed, the greater the risk. This application embodiment can also accumulate the duration of the driver's abnormality based on the moment when the deviation meets preset abnormal conditions. The longer the duration of the abnormal state, the higher the risk.

[0145] Furthermore, in this application embodiment, multiple risk intervals and corresponding preset risk membership degrees (membership degree values ​​are between 0 and 1) can be preset, which are five factors: anomaly category, confidence level, current vehicle speed, road type, and anomaly duration, and the corresponding target risk membership degree can be matched.

[0146] For example, the target risk membership degree corresponding to anomaly category, confidence level, current vehicle speed, road type, and anomaly duration can be as follows: Anomaly categories (weight can be set to 0.3) correspond to different preset risk membership degrees, and are ordered by degree of danger as follows: drunk driving > fatigue > emotional state > drug-induced. Among them, drunk driving corresponds to a high-risk membership degree of 1.0, fatigued driving corresponds to a medium-risk membership degree of 0.7, emotional abnormality corresponds to a medium-risk membership degree of 0.5, and drug-induced effects correspond to a low-risk membership degree of 0.3. The classification confidence level (weight can be set to 0.25) is used. The higher the confidence level (closer to 1), the greater the risk. The confidence level from 0 to 1 is converted into the membership degree corresponding to the three risk levels of low, medium and high through linear mapping, which is used as the target risk membership degree of the factor. The duration of the anomaly (weight can be set to 0.2) is preset to three risk ranges: less than 30 seconds corresponds to low risk membership of 1.0, 30 to 60 seconds corresponds to medium risk membership of 0.7, and more than 60 seconds corresponds to high risk membership of 1.0. The target risk membership is matched according to the cumulative duration of the anomaly. The current vehicle speed (weight can be set to 0.15) is preset to three risk ranges: below 60km / h corresponds to low risk membership degree of 0.3, 60 to 100km / h corresponds to medium risk membership degree of 0.7, and above 100km / h corresponds to high risk membership degree of 1.0. The corresponding target risk membership degree is matched according to the current vehicle speed. Road type (weight can be set to 0.1), the preset risk membership degree correspondence is: highway corresponds to high risk membership degree 1.0, urban expressway corresponds to medium risk membership degree 0.7, urban road corresponds to low risk membership degree 0.3. Based on the obtained current road type, the corresponding target risk membership degree is matched.

[0147] By combining the target risk membership degree and corresponding weights, this embodiment of the application can calculate the comprehensive risk membership degree and determine the driving risk level: the weight vector is w = [0.3, 0.25, 0.2, 0.15, 0.1], and the target risk membership degrees matched by the five factors (corresponding to low, medium, and high risk levels) constitute a 5x3 single-factor evaluation matrix R. Using the fuzzy comprehensive evaluation method, the weight vector w is subjected to fuzzy matrix operations with the single-factor evaluation matrix R (the operation rule is to take the smaller value first and then the larger value) to obtain the comprehensive evaluation vector B. After normalizing B, the comprehensive risk membership degree is obtained. The risk level corresponding to the maximum value of the comprehensive risk membership degree is taken as the final driving risk level.

[0148] The determination of driving risk level is illustrated with an example.

[0149] Assuming the anomaly category is fatigued driving (corresponding to a medium-risk membership degree of 0.7), the classification confidence is 0.8 (obtained through linear mapping), the anomaly duration is 45 seconds (corresponding to a medium-risk membership degree of 0.7), the current vehicle speed is 90 km / h (corresponding to a medium-risk membership degree of 0.7), and the current road type is an urban expressway (corresponding to a medium-risk membership degree of 0.7), after fuzzy matrix operations and normalization, if the comprehensive evaluation vector is [low 0.15, medium 0.55, high 0.30], then the maximum value of the comprehensive risk membership degree is 0.55, corresponding to medium risk. Therefore, the final driving risk level is medium risk.

[0150] In summary, such as Figure 4 As shown, embodiments of this application may include the following steps: Step S1, Data Acquisition. This embodiment of the application can reuse in-vehicle cameras, combine relevant data acquisition sensors from the steering wheel, seat pressure sensors, and a controller area network bus to acquire multi-source data characterizing the driver's state.

[0151] Step S2, Feature Extraction. This embodiment of the application can extract visual features, steering wheel features, seat features, and communication bus features from multi-source data, and then concatenate them to obtain a holistic multi-dimensional feature vector, i.e., a state feature vector.

[0152] Step S3: Calculate the Mahalanobis distance using the behavioral baseline distribution. This embodiment of the application can utilize a behavioral baseline distribution constructed from historical normal data to determine the degree of deviation between the driver's current driving state and the normal state using Mahalanobis distance. This embodiment of the application can calculate the degree of deviation by constructing an overall Gaussian model of multiple driving modes, or by matching the corresponding behavioral baseline according to the current driving mode, thereby determining the degree of deviation between the actual state and the normal state under the current driving mode.

[0153] Step S4, Anomaly Detection. Determine if the Mahalanobis distance is greater than a certain distance threshold. If it is greater than the certain distance threshold, proceed to step S5; otherwise, proceed to step S1.

[0154] Step S5, Classification and Identification. This embodiment of the application can combine the characteristics of classification models and random forest models to comprehensively identify anomaly categories.

[0155] Step S6, Risk Assessment. This embodiment of the application can assess the driving risk level by combining the anomaly category and the vehicle's current driving conditions.

[0156] Step S7, Warning Decision. This embodiment of the application can match corresponding warning actions to different driving risk levels to achieve vehicle safety control.

[0157] In summary, the embodiments of this application can collect four types of feature parameters: driver vision, steering wheel, seat, and vehicle bus. These four types of features are concatenated into a unified state feature vector to perform joint monitoring of the entire state across four dimensions: face, operation, posture, and vehicle dynamics. By comparing the real-time feature vector with the driver's personal driving behavior baseline distribution, the degree of deviation of the driver's current driving state from the driving behavior baseline is determined. This adapts to the driver's personalized driving habits, and in cases of significant deviation, an abnormal driving state is identified. Based on the type of abnormality and the current driving conditions of the vehicle, targeted and graded vehicle safety control is implemented, effectively improving driving safety.

[0158] Figure 5 This is a schematic diagram of the structure of a vehicle safety control device provided in an embodiment of this application.

[0159] For example, such as Figure 5 As shown, the vehicle's safety control device 10 may include: Specifically, the acquisition module 100 is used to acquire a state feature vector that characterizes the driver's current driving state. The state feature vector is composed of feature parameters from multiple data sources.

[0160] The calculation module 200 is used to obtain the degree of deviation of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, so as to identify the abnormal category of the driver when the degree of deviation meets the preset abnormal conditions.

[0161] The control module 300 is used to obtain the driver's driving risk level based on the anomaly category and the vehicle's current driving conditions, so as to control the vehicle to perform safety control actions that match the driving risk level.

[0162] Optionally, in one embodiment of this application, the feature parameters of the multiple data sources include: visual feature parameters, steering wheel feature parameters, seat feature parameters, and communication bus feature parameters, wherein the vehicle safety control device 10 further includes: The partitioning module is used to divide the historical state feature vectors that meet the preset benchmark conditions into behavioral pattern clusters under multiple preset driving modes of the vehicle.

[0163] The first construction module is used to construct behavioral feature matrices corresponding to multiple preset driving modes based on behavioral pattern clusters.

[0164] The first calculation module is used to calculate the covariance between historical visual feature parameters, historical steering wheel feature parameters, historical seat feature parameters, and historical communication bus feature parameters in the corresponding behavior pattern cluster based on the behavior feature matrix.

[0165] The second calculation module is used to calculate the corresponding feature mean vector based on the behavioral feature matrix.

[0166] The second construction module is used to combine covariance and feature mean vectors to construct a benchmark distribution of driving behavior corresponding to multiple preset driving modes.

[0167] Optionally, in one embodiment of this application, the computing module 200 includes: The determining unit is used to determine the current driving mode of the vehicle based on the current driving conditions.

[0168] The first matching unit is used to match the corresponding target behavior pattern cluster based on the current driving mode.

[0169] The first calculation unit is used to calculate the degree of deviation of the current driving state based on the state feature vector, the covariance and feature mean vector corresponding to the target behavior pattern cluster.

[0170] Optionally, in one embodiment of this application, the computing module 200 includes: The first acquisition unit is used to acquire the frequency of occurrence of multiple preset driving modes from the vehicle's historical driving data, and assign corresponding weights to the multiple preset driving modes based on the frequency of occurrence.

[0171] The building unit is used to combine covariance, feature mean vector, and weights to construct the corresponding benchmark distribution model of driving behavior.

[0172] The second calculation unit is used to calculate the baseline distribution model of driving behavior using state feature vectors to obtain the degree of deviation.

[0173] Optionally, in one embodiment of this application, the computing module 200 includes: The mapping unit is used to input the state feature vector into a pre-built classification model to map the state feature vector to the feature space of the target dimension.

[0174] The third calculation unit is used to calculate the distance from the state feature vector to each classification hyperplane in the feature space using a preset classification hyperplane, and obtain a first probability vector representing the distance between the state feature vector and multiple preset anomaly categories, so as to determine the anomaly category based on the first probability vector.

[0175] Optionally, in one embodiment of this application, the third computing unit includes: The computational subunit is used to input the state feature vector into the pre-built random forest model and calculate the classification results of multiple decision trees in the random forest model based on the preset anomaly categories.

[0176] The statistical subunit is used to statistically analyze the classification results and obtain the proportion of multiple preset abnormal categories.

[0177] A sub-unit is determined to obtain a second probability vector corresponding to multiple preset anomaly categories based on the quantity ratio, so as to determine the anomaly category based on the second probability vector and the first probability vector.

[0178] Optionally, in one embodiment of this application, determining the subunit includes: The calculation component is used to combine the first probability vector, the second probability vector, and the first preset weight to calculate the corresponding fusion probability vector.

[0179] The determination component is used to determine the anomaly category based on the fusion probability vector, and to determine the confidence level of the anomaly category based on the fusion probability corresponding to the anomaly category, so as to combine the anomaly category and the confidence level to obtain the driving risk level.

[0180] Optionally, in one embodiment of this application, the control module 300 includes: The second acquisition unit is used to acquire the vehicle's current speed and the road type of the current road based on the current driving conditions.

[0181] The cumulative unit is used to accumulate the duration of the driver's abnormality based on the time when the degree of deviation meets the preset abnormal conditions.

[0182] The second matching module is used to match the target risk membership degree based on multiple preset risk intervals and multiple preset risk interval mappings, which are respectively anomaly category, confidence level, current vehicle speed, road type and anomaly duration.

[0183] The fourth calculation unit is used to combine the target risk membership degree and the second preset weight to calculate the corresponding comprehensive risk membership degree, and determine the driving risk level based on the comprehensive risk membership degree.

[0184] Figure 6 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0185] When the processor 602 executes the program, it implements the vehicle safety control method provided in the above embodiments.

[0186] Furthermore, the vehicle also includes: Communication interface 606 is used for communication between memory 601 and processor 602.

[0187] The memory 601 is used to store computer programs that can run on the processor 602.

[0188] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0189] If the memory 601, processor 602, and communication interface 606 are implemented independently, then the communication interface 606, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0190] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 606 are integrated on a single chip, then the memory 601, processor 602, and communication interface 606 can communicate with each other through an internal interface.

[0191] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0192] Furthermore, embodiments of this application also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform the vehicle safety control method provided in embodiments of this application.

[0193] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0194] When the functional modules are divided according to their respective functions, the device may also include an acquisition module, a calculation module, and a control module. It should be noted that all relevant content regarding the steps involved in the above method embodiments can be referenced to the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0195] It should be understood that the device provided in this embodiment is used to execute the above-described vehicle safety control method, and therefore can achieve the same effect as the above-described implementation method.

[0196] When using integrated units, the device may include a processing module and a storage module. When applied to an automobile, the processing module can be used to control and manage the vehicle's movements. The storage module can be used to support the vehicle in executing program code, etc.

[0197] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as disclosed in this application. The processor may also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0198] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the vehicle safety control method provided in the above embodiments.

[0199] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described related method steps to implement a vehicle safety control method provided in the above embodiment.

[0200] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement a vehicle safety control method provided in the above embodiment.

[0201] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0202] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0203] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0204] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vehicle safety control method, characterized in that, Includes the following steps: Obtain a state feature vector that characterizes the driver's current driving state; the state feature vector is composed of feature parameters from multiple data sources. The deviation degree of the current driving state is obtained based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, so as to identify the abnormal category of the driver when the deviation degree meets the preset abnormal conditions; The driver's driving risk level is determined based on the anomaly category and the vehicle's current driving conditions, so as to control the vehicle to perform safety control actions that match the driving risk level.

2. The method according to claim 1, characterized in that, The feature parameters of the multiple data sources include: visual feature parameters, steering wheel feature parameters, seat feature parameters, and communication bus feature parameters; before obtaining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits, the method further includes: The historical state feature vectors that meet the preset benchmark conditions are divided into behavioral pattern clusters under multiple preset driving modes of the vehicle. Based on the behavior pattern cluster, construct multiple behavior feature matrices corresponding to the preset driving modes respectively; Based on the behavioral feature matrix, calculate the covariance among the historical visual feature parameters, historical steering wheel feature parameters, historical seat feature parameters, and historical communication bus feature parameters in the corresponding behavioral pattern cluster; Calculate the corresponding feature mean vector based on the behavioral feature matrix; By combining the covariance and the feature mean vector, a benchmark distribution of driving behavior corresponding to multiple preset driving modes is constructed.

3. The method according to claim 2, characterized in that, The step of determining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits includes: Based on the current driving conditions, determine the current driving mode of the vehicle; Match the corresponding target behavior pattern cluster based on the current driving mode; Based on the state feature vector, the covariance and feature mean vector corresponding to the target behavior pattern cluster, the deviation degree of the current driving state is calculated.

4. The method according to claim 2, characterized in that, The step of determining the deviation degree of the current driving state based on the state feature vector and the driving behavior benchmark distribution used to characterize the driver's driving behavior habits includes: From the vehicle's historical driving data, the frequency of occurrence of multiple preset driving modes is obtained, and based on the frequency of occurrence, corresponding weights are assigned to each of the multiple preset driving modes; By combining the covariance, the feature mean vector, and the weights, a corresponding benchmark distribution model for driving behavior is constructed. The deviation degree is obtained by calculating the driving behavior baseline distribution model using the state feature vector.

5. The method according to claim 1, characterized in that, The identification of the driver's anomaly category includes: The state feature vector is input into a pre-built classification model to map the state feature vector to the feature space of the target dimension. In the feature space, using a preset classification hyperplane, the distance from the state feature vector to each classification hyperplane is calculated to obtain a first probability vector characterizing the distance between the state feature vector and multiple preset anomaly categories, so as to determine the anomaly category based on the first probability vector.

6. The method according to claim 5, characterized in that, Determining the anomaly category based on the first probability vector includes: The state feature vector is input into a pre-constructed random forest model, and the classification results of multiple decision trees in the random forest model based on the preset anomaly category are calculated respectively. The classification results are statistically analyzed to obtain the percentage of each of the preset anomaly categories. Based on the quantity ratio, a second probability vector is obtained to characterize the state feature vector corresponding to multiple preset anomaly categories, so as to determine the anomaly category based on the second probability vector and the first probability vector.

7. The method according to claim 6, characterized in that, Determining the anomaly category based on the second probability vector and the first probability vector includes: By combining the first probability vector, the second probability vector, and the first preset weight, the corresponding fusion probability vector is calculated; The anomaly category is determined based on the fusion probability vector, and the confidence level of the anomaly category is determined based on the fusion probability corresponding to the anomaly category, so as to obtain the driving risk level by combining the anomaly category and the confidence level.

8. The method according to claim 7, characterized in that, The process of determining the driver's driving risk level based on the anomaly category and the vehicle's current driving conditions includes: Based on the current driving conditions, obtain the vehicle's current speed and the road type of the current road being traveled; The duration of the driver's abnormality is accumulated based on the time when the degree of deviation meets the preset abnormal conditions; Based on multiple preset risk intervals and multiple preset risk membership degrees mapped by the preset risk intervals, the target risk membership degrees are matched with the anomaly category, the confidence level, the current vehicle speed, the road type, and the anomaly duration, respectively. By combining the target risk membership degree and the second preset weight, the corresponding comprehensive risk membership degree is calculated, and the driving risk level is determined based on the comprehensive risk membership degree.

9. A vehicle, characterized in that, The vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 8.

Images