A driving behavior risk detection method, system, device and storage medium

Abstract

A method, system, device, and storage medium for detecting driving behavior risks relate to the field of driving safety. In this method, vehicle internal data, external environmental data, driver biometric data, in-vehicle video data, and in-vehicle audio data are collected; a first feature is extracted from the vehicle internal data, a second feature is extracted from the external environmental data, a third feature is extracted from the driver biometric data, a fourth feature is extracted from the in-vehicle video data, and a fifth feature is extracted from the in-vehicle audio data; the first, second, third, fourth, and fifth features are fused to obtain a feature vector, which is input into a preset machine learning model to obtain a risk detection score; and when the risk detection score exceeds a threshold, risk information is displayed on the in-vehicle display screen and a voice prompt is given. Implementing the technical solution provided by this application can comprehensively assess driving risks by integrating multi-source data, providing timely warnings and improving driving safety.

Description

Technical Field

[0001] The present application relates to the technical field of driving safety, and specifically to a method, system, device, and storage medium for detecting driving behavior risks. Background Art

[0002] With the continuous development of intelligent transportation systems, how to effectively monitor and evaluate the risks of driving behavior has become a key link in ensuring road traffic safety.

[0003] To address various risks that may arise during driving, most driving behavior monitoring systems currently on the market rely on a single data source (such as vehicle speed, braking frequency, etc.) to determine whether driving behavior poses risks. However, this method often ignores the impact of the external driving environment, resulting in incomplete risk assessment and difficulty in accurately predicting the probability of potential accidents.

[0004] Therefore, there is an urgent need for a technical solution that can integrate multiple data sources and realize multi-dimensional risk assessment to improve the effectiveness of driving behavior risk detection. Summary of the Invention

[0005] The present application provides a method, system, device and storage medium for detecting driving behavior risks, which can comprehensively and carefully assess the risks of driving behavior. This multi-dimensional data collection and analysis improves the accuracy of risk detection.

[0006] In a first aspect of the present application, a method for detecting driving behavior risk is provided, which is applied to a driving behavior risk detection platform. The method comprises:

[0007] Collecting vehicle interior data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data, wherein the vehicle interior data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation; the external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and changes in distance from the vehicle; the driver biometric data includes facial expressions; the in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration; and the in-vehicle audio data includes the tone and intonation of the driver's speech;

[0008] extracting a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data;

[0009] fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and inputting the feature vector into a preset machine learning model to obtain a risk detection score;

[0010] When the risk detection score exceeds a threshold, risk information is displayed on the vehicle display screen and a voice prompt is given.

[0011] Optionally, extracting a first feature from the vehicle interior data includes:

[0012] calculating an average speed of the first speed within a preset time period, calculating a standard deviation of the target acceleration within the preset time period, determining a first extreme value of the target acceleration within the preset time period, and determining a second extreme value of the first speed within the preset time period;

[0013] When the first extreme value is greater than or equal to a first threshold, constructing a first feature according to the first extreme value, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0014] When the first extreme value is less than the first threshold, determining whether the standard deviation is greater than or equal to a second threshold;

[0015] When the standard deviation is greater than or equal to the second threshold, constructing a first feature based on the standard deviation, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0016] When the standard deviation is less than the second threshold, a first feature is constructed according to the standard deviation, the average speed, the current continuous driving time, and the frequency of activating the accelerator and the brake.

[0017] Optionally, extracting a second feature from the external environment data includes:

[0018] determining whether a distance change rate is greater than or equal to a third threshold value based on the distance change, and determining whether the target vehicle is located in a target direction of the vehicle when the distance change rate is greater than or equal to the third threshold value;

[0019] When the target vehicle is located in a target direction of the vehicle, calculating a relative speed between the first speed and the second speed;

[0020] A second feature is constructed according to the feature value of the weather condition, the feature value of the road type, and the relative speed.

[0021] Optionally, extracting a third feature from the driver's biometric data includes:

[0022] Locating the driver's facial region in the captured image or video frame using a facial detection algorithm;

[0023] In the facial area, key feature points of the face are marked using a feature point detection algorithm, wherein the key feature points include the corners of the eyes, the corners of the mouth, and the eyebrows;

[0024] Facial expression features are extracted based on the key feature points. The facial expression features include the degree of opening and closing of the eyes, the degree of curvature of the eyebrows, and the degree of upward or downward movement of the corners of the mouth. A third feature is constructed based on the facial expression features.

[0025] Optionally, extracting the fourth feature from the in-vehicle video data includes:

[0026] Acquiring a gaze movement trajectory of the driver, and determining a distraction degree characteristic based on the gaze movement trajectory;

[0027] Detecting and tracking the driver's hand position in the image frame, classifying the hand posture, identifying the driver's gesture, and extracting gesture features from the gesture, the gesture features including gesture type, duration, and motion trajectory;

[0028] detecting the duration of the driver's eyes being closed, and obtaining the driver's head posture and mouth movements, and determining fatigue driving characteristics based on the duration of the eyes being closed, the head posture, and the mouth movements;

[0029] A fourth feature is constructed based on the attention distraction degree feature, the gesture feature, and the fatigue driving feature.

[0030] Optionally, extracting the fifth feature from the in-vehicle audio data includes:

[0031] Calculating volume differences and pitch differences between adjacent audio frames, determining volume features based on the volume differences, and determining pitch features based on the pitch differences, wherein the volume features include maximum volume, minimum volume, average volume, and standard deviation of volume changes, and the pitch features include average pitch and standard deviation of pitch changes;

[0032] Converting the in-vehicle audio data into text, calculating the number of words in the text, determining a speech rate feature based on the number of words, extracting content from the text, and determining an emotional feature based on the content;

[0033] A fifth feature is constructed based on the volume feature, the tone feature, the speech rate feature, and the emotion feature.

[0034] Optionally, fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector includes:

[0035] performing a standardization process on the first feature, the second feature, the third feature, the fourth feature, and the fifth feature;

[0036] Perform weighted summation on the standardized first feature, the second feature, the third feature, the fourth feature, and the fifth feature to form a feature vector.

[0037] In a second aspect of the present application, a driving behavior risk detection system is provided, comprising an acquisition module, a feature module, a fusion module, and a display module, wherein:

[0038] a collection module configured to collect vehicle internal data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data, wherein the vehicle internal data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation; the external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and a change in distance from the vehicle; the driver biometric data includes facial expressions; the in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration; and the in-vehicle audio data includes the tone and intonation of the driver's speech;

[0039] a feature module configured to extract a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data;

[0040] a fusion module configured to fuse the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and input the feature vector into a preset machine learning model to obtain a risk detection score;

[0041] The display module is configured to display risk information on the vehicle display screen and give a voice prompt when the risk detection score exceeds a threshold.

[0042] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0043] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any one of the methods described above is executed.

[0044] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0045] 1. By collecting multiple types of data (vehicle internal data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data), it can comprehensively and meticulously assess the risks of driving behavior. This multi-dimensional data collection and analysis makes risk detection more accurate and reliable;

[0046] 2. By collecting and analyzing data in real time, it can promptly detect potential risk behaviors during driving, such as driver fatigue, inattention, speeding, etc., thereby effectively preventing traffic accidents;

[0047] 3. By processing feature vectors using machine learning models, risk behavior patterns can be automatically identified and corresponding risk detection scores can be assigned. This intelligent processing method not only improves the efficiency and accuracy of risk detection, but also reduces the subjectivity and uncertainty of human judgment.

[0048] 4. When risky behavior is detected, the system can display risk information on the vehicle display screen and give voice prompts to remind the driver to pay attention to safety. This intuitive feedback method helps the driver to correct bad driving habits in time and improve driving safety;

[0049] 5. By detecting and warning of driving behavior risks in advance, this method helps drivers take preventive measures before potential dangers occur, thereby avoiding or reducing the occurrence of traffic accidents and protecting the lives and property of drivers and passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 1 is a flow chart of a method for detecting driving behavior risks disclosed in an embodiment of the present application;

[0051] Figure 2 This is a module diagram of a driving behavior risk detection system disclosed in an embodiment of the present application;

[0052] Figure 3 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0053] Explanation of the accompanying drawings: 201, acquisition module; 202, feature module; 203, fusion module; 204, display module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0055] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0056] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0057] This embodiment discloses a method for detecting driving behavior risk, which is applied to a driving behavior risk detection platform. Figure 1 This is a flow chart of the method for detecting driving behavior risks disclosed in the embodiment of the present application, such as Figure 1 As shown, the method includes the following steps:

[0058] S101. Collecting vehicle interior data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data. The vehicle interior data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation. The external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and a change in distance from the vehicle. The driver biometric data includes facial expressions. The in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration. The in-vehicle audio data includes the tone and intonation of the driver's speech.

[0059] S102, extracting a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver's biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data;

[0060] S103: Fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and inputting the feature vector into a preset machine learning model to obtain a risk detection score;

[0061] S104: When the risk detection score exceeds a threshold, risk information is displayed on the vehicle display screen and a voice prompt is given.

[0062] The first speed records the vehicle's current speed in real time and is a key parameter for assessing driving safety. The target acceleration records the vehicle's current acceleration value, reflecting smooth driving. The current continuous driving time records the driver's continuous driving time from the start of driving to the current moment, used to assess driver fatigue. The number and frequency of accelerator and brake operations are recorded, reflecting the driver's stability and responsiveness in vehicle control. Current weather information, such as sunny, rainy, and snowy, is obtained through on-board sensors or external data sources. Weather conditions significantly impact driving behavior. The vehicle identifies the type of road it is traveling on, such as highways, urban roads, and rural roads, as different road types have different requirements for driving behavior. The speed of the nearest target vehicle is obtained in real time to assess relative speed and potential collision risk. The distance between the vehicle and surrounding vehicles is recorded over time to reflect traffic density and the congestion level of the driving environment. The driver's facial expressions, such as smiles, frowns, and yawns, are captured through the on-board camera to assess their emotional state and fatigue. Video analysis technology tracks the driver's eye movements to assess driver attention. Identify driver gestures while driving, such as when talking on the phone, drinking, or eating, which can distract the driver. Record the length of time the driver's eyes are closed to assess driver fatigue. Use voice recognition technology to analyze the driver's tone and intonation, such as nervousness, anger, or calmness, to assess the driver's emotional state.

[0063] After collecting the above data, key features that reflect driving behavior risk need to be extracted from each data source. These features may include statistics (such as mean, standard deviation, maximum, minimum, etc.), category labels (such as weather condition classification), and time series data (such as speed and acceleration trends). First features: Features extracted from internal vehicle data, such as speed fluctuation range, acceleration change rate, and continuous driving duration threshold. Second features: Features extracted from external environmental data, such as the impact of weather conditions on driving, the safety factor of road types, and statistics on relative speed and distance changes. Third features: Features extracted from driver biometric data, such as facial expression classification labels, blink frequency, and facial expression duration. Fourth features: Features extracted from in-vehicle video data, such as the stability of eye gaze trajectory, the categories and frequency of gestures, and thresholds for eye closure duration. Fifth features: Features extracted from in-vehicle audio data, such as voice tone and intonation classification labels and emotional fluctuations.

[0064] The extracted features are fused to form a multidimensional feature vector. This process may involve steps such as feature standardization, feature selection, and feature transformation to ensure that different features have equal weight and comparability within the feature vector. The fused feature vector is then fed into a pre-set machine learning model (such as a neural network, support vector machine, or decision tree). This pre-set machine learning model has been trained and validated using extensive historical data to accurately identify risky patterns in driving behavior. The model outputs a risk detection score to assess the risk level of the current driving behavior. When the risk detection score exceeds a preset threshold, the current driving behavior is considered high risk. At this point, the system displays risk information (such as "Please Drive Safely" or "Avoid Fatigue Driving") on the vehicle's display screen and provides voice prompts to remind the driver to exercise caution. These prompts are intended to help drivers promptly identify and correct bad driving habits, reducing the risk of traffic accidents.

[0065] By collecting vehicle internal data, external environmental data, driver biometrics, in-vehicle video data, and in-vehicle audio data, the system can comprehensively and multi-dimensionally analyze driving behavior, thereby more accurately assessing risks. Real-time collection and processing of this data enables the system to promptly identify potential risky behaviors during driving, such as speeding, fatigued driving, and distracted driving, and provide a timely risk detection score. When the risk score exceeds a preset threshold, the system immediately displays risk information on the in-vehicle display and alerts the driver through voice prompts, effectively preventing traffic accidents. Using a machine learning model to process the fused feature vectors, it achieves intelligent detection of driving behavior risks. This approach not only improves the efficiency and accuracy of risk detection but also reduces the need for manual intervention, making the driving process more automated and intelligent. By promptly detecting and warning of driving behavior risks, the method helps drivers take preventive measures before potential dangers occur, thereby avoiding or reducing the occurrence of traffic accidents. This not only protects the lives of drivers and passengers but also reduces the risk of vehicle damage and property loss.

[0066] Optionally, extracting a first feature from the vehicle interior data includes:

[0067] calculating an average speed of the first speed within a preset time period, calculating a standard deviation of the target acceleration within the preset time period, determining a first extreme value of the target acceleration within the preset time period, and determining a second extreme value of the first speed within the preset time period;

[0068] When the first extreme value is greater than or equal to a first threshold, constructing a first feature according to the first extreme value, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0069] When the first extreme value is less than the first threshold, determining whether the standard deviation is greater than or equal to a second threshold;

[0070] When the standard deviation is greater than or equal to the second threshold, constructing a first feature based on the standard deviation, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0071] When the standard deviation is less than the second threshold, a first feature is constructed according to the standard deviation, the average speed, the current continuous driving time, and the frequency of activating the accelerator and the brake.

[0072] Calculate the average vehicle speed at the first speed over a preset time period (such as the past one minute or five minutes). This parameter reflects the vehicle's average driving state during this period. Calculate the standard deviation of the target acceleration over the preset time period. The standard deviation is a statistical measure of data fluctuation and is used here to reflect acceleration fluctuations, specifically the frequency and magnitude of vehicle acceleration or deceleration during this period. First Extreme Value: Determine the maximum or minimum value of the target acceleration over the preset time period (depending on the specific situation, but assumed to be the maximum value here). This parameter reflects the maximum acceleration variation experienced by the vehicle during this period. Second Extreme Value: Determine the maximum or minimum value of the first speed over the preset time period (also depending on the specific situation, but assumed to be the maximum value here). This parameter reflects the vehicle's maximum speed during this period. If the first extreme value of the target acceleration exceeds a preset threshold (indicating that the vehicle has experienced significant acceleration variation), a first feature is constructed based on the first extreme value, the second extreme value (maximum speed), the current continuous driving time, and the frequency of accelerator and brake activation. These parameters collectively reflect the vehicle's driving state when experiencing significant acceleration variations, the driver's fatigue level, and the frequency of driver operation. If the first extreme value of the target acceleration does not exceed the first threshold, but the standard deviation exceeds the second preset threshold (indicating significant acceleration fluctuations), the first feature is constructed based on the standard deviation, the second extreme value (maximum speed), the current continuous driving duration, and the frequency of accelerator and brake activations. These parameters reflect the vehicle's driving state and the frequency of maneuvers during periods of significant acceleration fluctuations. If neither the first extreme value nor the standard deviation of the target acceleration exceeds their respective thresholds (indicating relatively stable driving), the first feature is constructed based on the standard deviation (which, while small, still reflects some fluctuation), the average speed, the current continuous driving duration, and the frequency of accelerator and brake activations. These parameters reflect the vehicle's speed during stable driving, the driver's fatigue level, and the frequency of maneuvers. Through this conditional assessment and feature construction process, corresponding feature vectors can be extracted based on different driving behavior characteristics (such as large acceleration variations, large acceleration fluctuations, and stable driving). These feature vectors are then input into the machine learning model to calculate the risk detection score. By considering multiple driving behavior characteristics, it can more comprehensively reflect the driver's driving behavior and potential risks, thereby improving the accuracy and reliability of risk detection.

[0073] By calculating statistics such as the average speed of the first speed over a preset time period, and the standard deviation and extreme value of the target acceleration over a preset time period, dynamic changes during vehicle driving, such as speed stability and acceleration fluctuations, can be more precisely captured. These statistics, as features, can more accurately reflect certain characteristics of driving behavior. By comparing the extreme value (first extreme value) of the target acceleration with a preset threshold, a preliminary assessment can be made regarding potential risky behaviors such as sudden acceleration or deceleration. When the first extreme value exceeds the threshold, a first feature is constructed by combining it with other relevant features (such as the second extreme value, the current continuous driving duration, and the frequency of accelerator and brake activation), helping to more accurately identify high-risk driving behaviors. By comparing the standard deviation of the target acceleration with another preset threshold, the stability of driving behavior can be further refined. A large standard deviation indicates significant acceleration fluctuations, potentially indicating unstable driving behavior; a small standard deviation indicates relatively stable acceleration. Constructing different feature combinations based on these two conditions provides a more comprehensive description of driving behavior. Through conditional judgment, different feature combinations can be constructed based on different situations, adapting to different driving scenarios. For example, when driving on a highway, acceleration fluctuations may be relatively small, while on congested urban roads, acceleration fluctuations may be larger. By constructing different feature combinations, we can more accurately reflect driving behavior in different scenarios.

[0074] Optionally, extracting a second feature from the external environment data includes:

[0075] determining whether a distance change rate is greater than or equal to a third threshold value based on the distance change, and determining whether the target vehicle is located in a target direction of the vehicle when the distance change rate is greater than or equal to the third threshold value;

[0076] When the target vehicle is located in a target direction of the vehicle, calculating a relative speed between the first speed and the second speed;

[0077] A second feature is constructed according to the feature value of the weather condition, the feature value of the road type, and the relative speed.

[0078] The system determines the first vehicle currently closest to the vehicle and calculates the rate of change in the distance between the vehicle and the first vehicle. This rate of change reflects the relative speed of movement between the two vehicles, that is, whether they are approaching or receding. The system then determines whether the rate of change is greater than or equal to a preset third threshold. This third threshold is used to determine whether there is a potential collision risk. A large rate of change indicates that the relative speed between the two vehicles is high, potentially posing a collision risk. When the rate of change is greater than or equal to the third threshold, the system further determines whether the target vehicle is in the vehicle's target direction. This "target direction" may refer to the vehicle's current direction of travel or the driver's intended direction (such as indicated by a turn signal). If the target vehicle is in the vehicle's target direction, the relative movement between the two vehicles is potentially more dangerous because they are moving in the same direction. When the target vehicle is determined to be in the vehicle's target direction, the system calculates the relative speed between the two vehicles. This relative speed is determined by comparing the two speeds (the first speed and the second speed). Relative speed is an important feature because it directly reflects the speed of approach between the two vehicles, thereby helping to assess the potential collision risk. The system constructs a second feature based on the characteristic values of weather conditions, road type, and the calculated relative speed. Weather conditions (e.g., rainy, snowy, sunny) and road types (e.g., highways, urban roads, rural roads, etc.) can affect vehicle stability and safety. Therefore, incorporating these factors as features into risk detection models can improve their accuracy and adaptability.

[0079] By determining whether the distance change rate is greater than or equal to a third threshold, potential changes in the external environment, such as acceleration or deceleration of the vehicle ahead or road congestion, can be preliminarily identified. This helps the vehicle perceive changes in the surrounding environment more promptly, enabling more accurate driving decisions. When the distance change rate meets the criteria, the vehicle further determines whether the target vehicle is in its target direction. This step helps the vehicle more accurately identify potential risk vehicles relative to its driving direction, avoiding unnecessary interference and misjudgment. By calculating the relative speed between the first and second speeds, the speed difference between the vehicle and the target vehicle can be more intuitively reflected. This feature is important for assessing the relative position and potential collision risk between the two vehicles. By constructing a second feature based on the characteristic values of weather conditions, road type, and relative speed, this method comprehensively considers the impact of the external environment on driving behavior. For example, driving on rainy or snowy roads can increase braking distances due to slippery roads, while driving on highways often involves higher speeds, increasing potential risk. By incorporating the characteristic values of these environmental factors, the risk level of driving behavior can be more accurately assessed. The second feature constructed through the above steps more comprehensively reflects the impact of the external environment on driving behavior and the relative relationship between the vehicle and the target vehicle. This helps the vehicle to more accurately judge potential risks and take preventive measures in advance, such as slowing down and avoiding, to improve driving safety.

[0080] Optionally, extracting a third feature from the driver's biometric data includes:

[0081] Locating the driver's facial region in the captured image or video frame using a facial detection algorithm;

[0082] In the facial area, key feature points of the face are marked using a feature point detection algorithm, wherein the key feature points include the corners of the eyes, the corners of the mouth, and the eyebrows;

[0083] Facial expression features are extracted based on the key feature points. The facial expression features include the degree of opening and closing of the eyes, the degree of curvature of the eyebrows, and the degree of upward or downward movement of the corners of the mouth. A third feature is constructed based on the facial expression features.

[0084] Facial detection algorithms, such as Haar features, HOG features combined with a support vector machine (SVM) classifier, or deep learning-based facial detection methods (such as convolutional neural networks (CNNs)), accurately locate the driver's facial region within captured images or video frames. These algorithms can identify faces against complex backgrounds and determine their location and size. A rectangular bounding box is generated, accurately locating the driver's face. Key facial feature points are marked within the facial region, reflecting its shape and details. Feature point detection algorithms, such as the Active Shape Model (ASM) and Active Appearance Model (AAM), or deep learning-based methods (such as the 68-point facial feature detection in the Dlib library), can be used to identify and mark the locations of key facial features, such as the corners of the eyes, mouth, and eyebrows. The resulting coordinates of a series of facial feature points accurately describe the shape and details of the face. Based on the locations of these facial feature points, facial expression features are extracted, which can reflect the driver's emotions and attention state. By calculating the distance or area change between feature points in the eye region, it is possible to determine whether the eyes are open or closed, and the degree of openness. By calculating the curvature or tilt angle of feature points in the eyebrow region, it is possible to determine whether the eyebrows are raised, lowered, or straight, as well as the degree of curvature. By calculating the positional changes of feature points in the mouth corner region, it is possible to determine whether the corners of the mouth are raised (indicating happiness or a smile) or lowered (indicating sadness or frustration), as well as the degree of the raised or lowered corners. This generates a series of facial expression feature values that quantitatively describe the dynamic changes in the eyes, eyebrows, and mouth corners, thereby reflecting the driver's facial expression and emotional state. The extracted facial expression feature values are integrated into a sub-feature vector, which serves as the third feature for subsequent driving behavior risk detection. Specifically, feature values such as the degree of eye opening and closing, the degree of eyebrow curvature, and the degree of raised or lowered mouth corners can be normalized (e.g., mapped to a range of 0-1) and then combined in a specific order to form a sub-feature vector.

[0085] By using facial detection and feature point detection algorithms, the driver's facial region can be accurately located and key facial features can be marked. This helps more precisely capture changes in the driver's facial expressions, enabling more accurate identification of states such as fatigue, distraction, and anger. In addition to locating key facial features, facial expression characteristics such as the degree of eye opening and closing, eyebrow curvature, and the degree of upturned or downturned corners of the mouth are further extracted. These features provide a more nuanced reflection of the driver's emotional state and attention level, facilitating a more comprehensive assessment of driving risk. Facial detection and feature point detection algorithms generally offer high real-time and robust performance, operating stably under varying lighting conditions, head posture, and occlusion. This means that this method can extract facial expression features in real time in a variety of complex driving environments, providing timely and reliable information for driving behavior risk detection. Facial expressions are an important indicator of a driver's psychological and emotional state. By extracting and analyzing facial expression features, this method can provide important insights for driving behavior risk detection. For example, when a driver displays signs of fatigue or distraction, the system can issue a timely warning, reminding the driver to pay attention.

[0086] Optionally, extracting the fourth feature from the in-vehicle video data includes:

[0087] Acquiring a gaze movement trajectory of the driver, and determining a distraction degree characteristic based on the gaze movement trajectory;

[0088] Detecting and tracking the driver's hand position in the image frame, classifying the hand posture, identifying the driver's gesture, and extracting gesture features from the gesture, the gesture features including gesture type, duration, and motion trajectory;

[0089] detecting the duration of the driver's eyes being closed, and obtaining the driver's head posture and mouth movements, and determining fatigue driving characteristics based on the duration of the eyes being closed, the head posture, and the mouth movements;

[0090] A fourth feature is constructed based on the attention distraction degree feature, the gesture feature, and the fatigue driving feature.

[0091] The driver's facial image is captured using an onboard camera, and image processing techniques (such as pupil tracking and head pose estimation) are used to track the driver's gaze trajectory. This gaze trajectory is analyzed to determine whether the driver's gaze frequently strays from the road ahead or remains focused on non-driving areas (such as the rearview mirror or a mobile phone) for extended periods. The driver's level of distraction is quantified based on the frequency and duration of these gaze deviations and used as part of the fourth feature. Within the image frames, the driver's hand position is detected and continuously tracked using image processing or deep learning techniques. Based on hand position and trajectory, hand gestures are classified into categories such as gripping the steering wheel, talking on the phone, or eating. Specific gestures are further identified, including gesture type (such as pointing or waving), duration, and trajectory. The identified gestures are converted into feature vectors, including gesture type, duration, and trajectory, which are used as components of the fourth feature. Image processing techniques are used to detect the driver's eye closure and calculate the duration of eye closure. Furthermore, the driver's head posture (such as nodding or shaking the head) and mouth movements (such as yawning or talking) are monitored. The driver's fatigue level is comprehensively assessed by combining eye closure duration, head posture, and mouth movements. For example, prolonged eye closure, frequent nodding, or yawning may indicate driver fatigue. The aforementioned distraction, gesture, and fatigue driving characteristics are combined to form a fourth feature. This feature vector comprehensively reflects the driver's behavior while driving, providing important evidence for subsequent driving behavior analysis, risk assessment, and early warning.

[0092] By capturing the driver's gaze trajectory, it is possible to determine whether the driver's attention is distracted, which is crucial for promptly detecting distracted driving and improving driving safety. Detecting and tracking the driver's hand position and posture, and recognizing gestures, can further understand the driver's operating habits and behavioral intentions while driving, helping to promptly detect abnormal driving behavior. By detecting the driver's eye closure duration, head posture, and mouth movements, combined with this information, it is possible to accurately determine whether the driver is driving fatigued. This is crucial for preventing traffic accidents caused by fatigue. Extracted features such as attention distraction, gesture features, and fatigue driving characteristics provide rich data support for driving behavior analysis. These features can be used to train and optimize driving behavior models, enhancing the intelligence level of intelligent driving systems. By real-time monitoring and analysis of features such as the driver's gaze, hand posture, gestures, and fatigue level, intelligent driving assistance systems can more intelligently assess the driver's condition, issue timely warnings, or take intervention measures, thereby improving driving safety and comfort.

[0093] Optionally, extracting the fifth feature from the in-vehicle audio data includes:

[0094] Calculating volume differences and pitch differences between adjacent audio frames, determining volume features based on the volume differences, and determining pitch features based on the pitch differences, wherein the volume features include maximum volume, minimum volume, average volume, and standard deviation of volume changes, and the pitch features include average pitch and standard deviation of pitch changes;

[0095] Converting the in-vehicle audio data into text, calculating the number of words in the text, determining a speech rate feature based on the number of words, extracting content from the text, and determining an emotional feature based on the content;

[0096] A fifth feature is constructed based on the volume feature, the tone feature, the speech rate feature, and the emotion feature.

[0097] The in-vehicle audio data is segmented into multiple consecutive audio frames, and noise, including music, is removed from the audio frames. The volume difference between adjacent audio frames is then calculated by comparing the audio energy or amplitude of each frame. This volume difference reflects the volume fluctuation of the driver's voice. Based on this volume difference, the maximum volume (i.e., the highest volume value in an audio frame), the minimum volume (i.e., the lowest volume value in an audio frame), the average volume (i.e., the average volume across all audio frames), and the standard deviation of the volume variation (reflecting the degree of volume fluctuation) can be calculated. Pitch is another important characteristic of speech, reflecting the fundamental frequency of speech. By calculating the pitch difference between adjacent audio frames, the pitch variation of the driver's voice can be understood. Based on this pitch difference, the average pitch (i.e., the average pitch across all audio frames) and the standard deviation of the pitch variation (reflecting the degree of pitch fluctuation) can be calculated. Using speech recognition technology, the in-vehicle audio data is converted into text. This step forms the basis for subsequent analysis. The number of words in the text is calculated, and combined with the total duration of the audio data, the speaking rate (i.e., the number of words uttered per unit time) can be calculated. The speaking rate reflects the speed of the driver's speech. Extract the content from the text and perform sentiment analysis using natural language processing techniques. Sentiment analysis can identify the type of emotion expressed in the text, such as positive, negative, or neutral. Emotional features reflect the driver's emotional state while driving. The volume features (maximum volume, minimum volume, average volume, and standard deviation of volume variation), pitch features (average pitch, standard deviation of pitch variation), speech rate features, and emotion features calculated above are integrated to construct the fifth feature.

[0098] By calculating the volume and pitch differences between adjacent audio frames, volume and pitch features can be extracted. These features can reflect the driver's emotional changes and attention state while driving. For example, a sudden increase in volume may indicate excitement or nervousness, while a change in pitch may reflect fatigue or emotional fluctuations. After converting in-vehicle audio data to text, the number of words in the text can be counted to determine speech rate features. Speech rate is an important indicator of the driver's speaking speed and mental activity, and is crucial for determining whether the driver is fatigued, distracted, or nervous. By extracting the content from the text, the driver's emotional characteristics can be further analyzed. Emotion is a key factor influencing driving behavior. By identifying the driver's emotional state, potential driving risks can be promptly identified. For example, negative emotions such as anger and anxiety may lead to dangerous driving behavior. By combining information from multiple dimensions such as volume, pitch, speech rate, and emotion, a more comprehensive driving behavior profile can be constructed. These features provide rich data support for intelligent driving assistance systems, helping them to more accurately assess the driver's state and issue timely warnings or take intervention measures, thereby improving driving safety.

[0099] Optionally, fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector includes:

[0100] performing a standardization process on the first feature, the second feature, the third feature, the fourth feature, and the fifth feature;

[0101] Perform weighted summation on the standardized first feature, the second feature, the third feature, the fourth feature, and the fifth feature to form a feature vector.

[0102] Normalization is a crucial step in data preprocessing. It helps eliminate dimensional differences between features and makes their values comparable. Normalization is particularly important in feature fusion, as different features may come from different data sources or use different units of measurement. Direct fusion can lead to inaccurate or difficult-to-interpret results. Common normalization methods include: Z-score normalization: This method subtracts the mean from each feature value and then divides it by the standard deviation. This method results in a mean of 0 and a standard deviation of 1. Min-Max normalization: This method scales each feature value to a specified range (typically between 0 and 1). This method is achieved by calculating the minimum and maximum values of the feature values. Before fusion, the first, second, third, fourth, and fifth features are each normalized. This ensures that they are equally weighted and comparable during fusion. Weighted summation is a common method for fusing multiple features into a feature vector. In this process, each feature is assigned a weight that reflects its importance or influence within the overall dataset. Weights are often determined based on feature importance, relevance, or empirical knowledge. In practical applications, weights can be determined using machine learning algorithms (such as principal component analysis and linear discriminant analysis) or expert scoring. The standardized value of each feature is multiplied by its corresponding weight, and then all weighted values are summed to obtain each component of the feature vector. The weighted sum is used as a component of the feature vector, forming a multidimensional vector.

[0103] Normalization is a crucial step in data preprocessing. It unifies the data ranges of different features onto the same scale, eliminating dimensional differences between features. Normalization makes the numerical values of each feature comparable, facilitating subsequent feature fusion and model training. Weighted summation is a common method for fusing multiple features into a single feature vector. In this weighted summation, the weight of each feature reflects its importance in the overall feature vector. Proper weight distribution ensures that the feature vector fully reflects the information of each feature, thereby improving its representational power. Feature fusion is a key technique in machine learning. It combines multiple individual features into a more representative composite feature. Fusion of multiple features enables the construction of more complex and accurate models, enhancing their generalization capabilities. This allows the model to better adapt to diverse data distributions and changing scenarios, improving prediction accuracy and stability. Both normalization and weighted summation are relatively simple and efficient computational processes. Through these two steps, multiple features can be quickly fused into a feature vector, providing strong support for subsequent model training and prediction. At the same time, since the normalization process eliminates the dimensional differences between features, it can avoid problems such as numerical overflow or underflow during the calculation process, thereby improving the accuracy and stability of the calculation.

[0104] This embodiment also discloses a driving behavior risk detection system. Figure 2 This is a module diagram of the driving behavior risk detection system disclosed in the embodiment of the present application, such as Figure 2 As shown, the system includes an acquisition module 201, a feature module 202, a fusion module 203 and a display module 204, wherein:

[0105] The acquisition module 201 is configured to collect vehicle internal data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data, wherein the vehicle internal data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation; the external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and a change in distance from the vehicle; the driver biometric data includes facial expressions; the in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration; and the in-vehicle audio data includes the tone and intonation of the driver's speech;

[0106] a feature module 202 configured to extract a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data;

[0107] a fusion module 203 configured to fuse the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and input the feature vector into a preset machine learning model to obtain a risk detection score;

[0108] The display module 204 is configured to display risk information on the vehicle display screen and give a voice prompt when the risk detection score exceeds a threshold.

[0109] Optionally, the feature module 202 is configured to:

[0110] calculating an average speed of the first speed within a preset time period, calculating a standard deviation of the target acceleration within the preset time period, determining a first extreme value of the target acceleration within the preset time period, and determining a second extreme value of the first speed within the preset time period;

[0111] When the first extreme value is greater than or equal to a first threshold, constructing a first feature according to the first extreme value, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0112] When the first extreme value is less than the first threshold, determining whether the standard deviation is greater than or equal to a second threshold;

[0113] When the standard deviation is greater than or equal to the second threshold, constructing a first feature based on the standard deviation, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake;

[0114] When the standard deviation is less than the second threshold, a first feature is constructed according to the standard deviation, the average speed, the current continuous driving time, and the frequency of activating the accelerator and the brake.

[0115] Optionally, the feature module 202 is configured to:

[0116] determining whether a distance change rate is greater than or equal to a third threshold value based on the distance change, and determining whether the target vehicle is located in a target direction of the vehicle when the distance change rate is greater than or equal to the third threshold value;

[0117] When the target vehicle is located in a target direction of the vehicle, calculating a relative speed between the first speed and the second speed;

[0118] A second feature is constructed according to the feature value of the weather condition, the feature value of the road type, and the relative speed.

[0119] Optionally, the feature module 202 is configured to:

[0120] Locating the driver's facial region in the captured image or video frame using a facial detection algorithm;

[0121] In the facial area, key feature points of the face are marked using a feature point detection algorithm, wherein the key feature points include the corners of the eyes, the corners of the mouth, and the eyebrows;

[0122] Facial expression features are extracted based on the key feature points. The facial expression features include the degree of opening and closing of the eyes, the degree of curvature of the eyebrows, and the degree of upward or downward movement of the corners of the mouth. A third feature is constructed based on the facial expression features.

[0123] Optionally, the feature module 202 is configured to:

[0124] Acquiring a gaze movement trajectory of the driver, and determining a distraction degree characteristic based on the gaze movement trajectory;

[0125] Detecting and tracking the driver's hand position in the image frame, classifying the hand posture, identifying the driver's gesture, and extracting gesture features from the gesture, the gesture features including gesture type, duration, and motion trajectory;

[0126] detecting the duration of the driver's eyes being closed, and obtaining the driver's head posture and mouth movements, and determining fatigue driving characteristics based on the duration of the eyes being closed, the head posture, and the mouth movements;

[0127] A fourth feature is constructed based on the attention distraction degree feature, the gesture feature, and the fatigue driving feature.

[0128] Optionally, the feature module 202 is configured to:

[0129] Calculating volume differences and pitch differences between adjacent audio frames, determining volume features based on the volume differences, and determining pitch features based on the pitch differences, wherein the volume features include maximum volume, minimum volume, average volume, and standard deviation of volume changes, and the pitch features include average pitch and standard deviation of pitch changes;

[0130] Converting the in-vehicle audio data into text, calculating the number of words in the text, determining a speech rate feature based on the number of words, extracting content from the text, and determining an emotional feature based on the content;

[0131] A fifth feature is constructed based on the volume feature, the tone feature, the speech rate feature, and the emotion feature.

[0132] Optionally, the fusion module 203 is configured to:

[0133] performing a standardization process on the first feature, the second feature, the third feature, the fourth feature, and the fifth feature;

[0134] Perform weighted summation on the standardized first feature, the second feature, the third feature, the fourth feature, and the fifth feature to form a feature vector.

[0135] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0136] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0137] The communication bus 302 is used to implement the connection and communication between these components.

[0138] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0139] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0140] The processor 301 may include one or more processing cores. Using various interfaces and circuits, the processor 301 connects to various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 305, as well as accesses data stored in the memory 305, to perform various server functions and process data. Optionally, the processor 301 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 301 but implemented as a separate chip.

[0141] Among them, the memory 305 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a method for detecting driving behavior risks.

[0142] exist Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call the application program for detecting the driving behavior risk stored in the memory 305. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0143] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0144] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0146] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0147] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory 305 includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0149] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for detecting driving behavior risk, characterized in that: Applied to a driving behavior risk detection platform, the method includes: Collecting vehicle interior data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data, wherein the vehicle interior data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation; the external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and changes in distance from the vehicle; the driver biometric data includes facial expressions; the in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration; and the in-vehicle audio data includes the tone and intonation of the driver's speech; extracting a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data; fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and inputting the feature vector into a preset machine learning model to obtain a risk detection score; When the risk detection score exceeds the threshold, the risk information is displayed on the vehicle display screen and a voice prompt is given. The extracting the second feature from the external environment data comprises: determining whether a distance change rate is greater than or equal to a third threshold value based on the distance change, and determining whether the target vehicle is located in a target direction of the vehicle when the distance change rate is greater than or equal to the third threshold value; When the target vehicle is located in a target direction of the vehicle, calculating a relative speed between the first speed and the second speed; Constructing a second feature according to the feature value of the weather condition, the feature value of the road type, and the relative speed, The extracting the fourth feature from the vehicle-mounted video data comprises: Acquiring a gaze movement trajectory of the driver, and determining a distraction degree characteristic based on the gaze movement trajectory; Detecting and tracking the driver's hand position in the image frame, classifying the hand posture, identifying the driver's gesture, and extracting gesture features from the gesture, the gesture features including gesture type, duration, and motion trajectory; detecting the duration of the driver's eyes being closed, and obtaining the driver's head posture and mouth movements, and determining fatigue driving characteristics based on the duration of the eyes being closed, the head posture, and the mouth movements; A fourth feature is constructed based on the attention distraction degree feature, the gesture feature, and the fatigue driving feature.

2. The method for detecting driving behavior risk according to claim 1, characterized in that: The extracting a first feature from the vehicle interior data comprises: calculating an average speed of the first speed within a preset time period, calculating a standard deviation of the target acceleration within the preset time period, determining a first extreme value of the target acceleration within the preset time period, and determining a second extreme value of the first speed within the preset time period; When the first extreme value is greater than or equal to a first threshold, constructing a first feature according to the first extreme value, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake; When the first extreme value is less than the first threshold, determining whether the standard deviation is greater than or equal to a second threshold; When the standard deviation is greater than or equal to the second threshold, constructing a first feature based on the standard deviation, the second extreme value, the current continuous driving time, and the frequency of activating the accelerator and the brake; When the standard deviation is less than the second threshold, a first feature is constructed according to the standard deviation, the average speed, the current continuous driving time, and the frequency of activating the accelerator and the brake.

3. The method for detecting driving behavior risk according to claim 1, characterized in that: Extracting a third feature from the driver's biometric data includes: Locating the driver's facial region in the captured image or video frame using a facial detection algorithm; In the facial area, key feature points of the face are marked using a feature point detection algorithm, wherein the key feature points include the corners of the eyes, the corners of the mouth, and the eyebrows; Facial expression features are extracted based on the key feature points. The facial expression features include the degree of opening and closing of the eyes, the degree of curvature of the eyebrows, and the degree of upward or downward movement of the corners of the mouth. A third feature is constructed based on the facial expression features.

4. The method for detecting driving behavior risk according to claim 1, characterized in that: Extracting the fifth feature from the vehicle audio data includes: Calculating volume differences and pitch differences between adjacent audio frames, determining volume features based on the volume differences, and determining pitch features based on the pitch differences, wherein the volume features include maximum volume, minimum volume, average volume, and standard deviation of volume changes, and the pitch features include average pitch and standard deviation of pitch changes; Converting the in-vehicle audio data into text, calculating the number of words in the text, determining a speech rate feature based on the number of words, extracting content from the text, and determining an emotional feature based on the content; A fifth feature is constructed based on the volume feature, the tone feature, the speech rate feature, and the emotion feature.

5. The method for detecting driving behavior risk according to claim 1, characterized in that: The fusing the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector includes: performing a standardization process on the first feature, the second feature, the third feature, the fourth feature, and the fifth feature; Perform weighted summation on the standardized first feature, the second feature, the third feature, the fourth feature, and the fifth feature to form a feature vector.

6. A driving behavior risk detection system, characterized in that: It includes acquisition module, feature module, fusion module and display module, among which: a collection module configured to collect vehicle internal data, external environment data, driver biometric data, in-vehicle video data, and in-vehicle audio data, wherein the vehicle internal data includes a first vehicle speed, target acceleration, current continuous driving duration, and frequency of accelerator and brake activation; the external environment data includes weather conditions, road type, a second speed of the nearest target vehicle around the vehicle, and a change in distance from the vehicle; the driver biometric data includes facial expressions; the in-vehicle video data includes eye gaze trajectory, hand gestures, and eye closure duration; and the in-vehicle audio data includes the tone and intonation of the driver's speech; a feature module configured to extract a first feature from the vehicle interior data, a second feature from the external environment data, a third feature from the driver biometric data, a fourth feature from the in-vehicle video data, and a fifth feature from the in-vehicle audio data; a fusion module configured to fuse the first feature, the second feature, the third feature, the fourth feature, and the fifth feature to obtain a feature vector, and input the feature vector into a preset machine learning model to obtain a risk detection score; A display module is configured to display risk information on the vehicle display screen and give a voice prompt when the risk detection score exceeds a threshold value. The extracting the second feature from the external environment data comprises: determining whether a distance change rate is greater than or equal to a third threshold value based on the distance change, and determining whether the target vehicle is located in a target direction of the vehicle when the distance change rate is greater than or equal to the third threshold value; When the target vehicle is located in a target direction of the vehicle, calculating a relative speed between the first speed and the second speed; The second feature is constructed based on the feature value of the weather condition, the feature value of the road type, and the relative speed. The extracting the fourth feature from the vehicle-mounted video data comprises: Acquiring a gaze movement trajectory of the driver, and determining a distraction degree characteristic based on the gaze movement trajectory; Detecting and tracking the driver's hand position in the image frame, classifying the hand posture, identifying the driver's gesture, and extracting gesture features from the gesture, the gesture features including gesture type, duration, and motion trajectory; detecting the duration of the driver's eyes being closed, and obtaining the driver's head posture and mouth movements, and determining fatigue driving characteristics based on the duration of the eyes being closed, the head posture, and the mouth movements; A fourth feature is constructed based on the attention distraction degree feature, the gesture feature, and the fatigue driving feature.

7. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 5 is executed.

Images