Driver driving risk analysis method, system, electronic device and storage medium

Abstract

A driver driving risk analysis method, comprising: collecting face video data, environment video data and vehicle driving data under a driving scene, and performing behavior event analysis on the face video data, the environment video data and the vehicle driving data to construct a combined event; performing driving risk analysis on the combined event to determine whether the combined event is a negative event; and counting the number of negative events, and if the number of negative events reaches a preset number threshold, determining that the current driving state of the driver has driving risk. A driver driving risk analysis system, an electronic device and a storage medium are also disclosed.

Description

Technical Field

[0001] This application relates to the field of vehicle driving technology, and in particular to a method, system, electronic device and storage medium for analyzing driving risks for drivers. Background Technology

[0002] With the increasing prevalence of vehicles, traffic accidents are also on the rise. A driver's condition has a significant impact on road safety. Therefore, it is essential to ensure drivers are in good driving condition as much as possible. During driving, factors such as excessive fatigue and inattention can impair a driver's judgment, greatly increasing the risk of traffic accidents. In recent years, research on analyzing driver risk has become increasingly prevalent. Currently, existing methods for analyzing driving risk rely on Driver Monitoring Systems (DMS) or Advanced Driving Assistance Systems (ADAS) to determine driver behavior and analyze driving risk based on single video frames from a single sensor. However, this analysis process relies on limited data, largely depending on the analyst's experience, resulting in strong subjectivity and potentially biased or inaccurate conclusions. Invention Overview

[0004] Technical issues

[0005] In view of this, embodiments of this application provide a method, system, electronic device, and storage medium for analyzing driver driving risks, which can improve the accuracy of driver driving risk analysis.

[0006] Technical solutions

[0007] The first aspect of this application provides a driver driving risk analysis method, including: collecting facial video data, environmental video data, and vehicle driving data in a driving scenario; performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct a combined event; performing driving risk analysis on the combined event to determine whether the combined event is a negative event; counting the number of negative events; and if the number of negative events reaches a preset threshold, determining that the driver's current driving state poses a driving risk.

[0008] In one possible implementation, the step of performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct a combined event includes: performing driver state detection on the facial video data and / or the environmental video data to obtain a first behavioral event representing the driver state, wherein the first behavioral event includes the event type, the event start time, the event end time, and a probability score sequence of the event between the start time and the end time; performing vehicle state detection on the vehicle driving data to obtain a second behavioral event representing the vehicle state, wherein the event start time, the event end time, and a driving parameter value sequence of the event between the start time and the end time; sorting all the obtained first behavioral events and all the second behavioral events on a timeline to obtain a behavioral event sequence; and obtaining a combined event corresponding to the target behavioral event from the behavioral event sequence based on the target behavioral event.

[0009] In one possible implementation, the step of detecting driver state on the face video data and / or the environmental video data to obtain a first behavioral event characterizing the driver state includes: performing frame segmentation processing on the face video data and / or the environmental video data to obtain a video image frame sequence; performing feature extraction processing on each video image in the video image frame sequence to obtain a feature image sequence; and inputting the feature image sequence into a preset driver state detection model to perform driver state detection and obtain a first behavioral event characterizing the driver state.

[0010] In one possible implementation, before the step of inputting the feature image sequence into a preset driver state detection model to detect the driver state and obtain a first behavioral event representing the driver state, the method further includes: performing pixel regularization processing on each frame of the feature image sequence.

[0011] In one possible implementation, before the step of sorting all the first behavioral events and all the second behavioral events obtained on the timeline to obtain a behavioral event sequence, the method further includes: collecting weather state information under the driving scenario, and fusing the weather state information with time to obtain weather events; the step of sorting all the first behavioral events and all the second behavioral events obtained on the timeline to obtain a behavioral event sequence includes: sorting all the first behavioral events and all the second behavioral events obtained along with the weather events on the timeline to obtain a behavioral event sequence.

[0012] In one possible implementation, before the step of sorting all the first behavioral events and all the second behavioral events obtained on the time axis to obtain a behavioral event sequence, the method further includes: collecting road information in the driving scenario, and fusing the road information with time to obtain road condition events; the step of sorting all the first behavioral events and all the second behavioral events obtained on the time axis to obtain a behavioral event sequence includes: sorting all the first behavioral events and all the second behavioral events obtained on the time axis together with the road condition events to obtain a behavioral event sequence.

[0013] In one possible implementation, the step of performing driving risk analysis on the combined events and determining whether the combined events are adverse events includes: numerically encoding the attribute data of each behavioral event in the combined events according to time sequence to generate a high-dimensional matrix for representing the combined events; inputting the high-dimensional matrix into a preset driving risk calculation model for numerical calculation to output the risk score corresponding to the combined events; comparing the risk score with a preset score threshold, and if the risk score reaches the preset score threshold, then the combined events are judged as adverse events.

[0014] A second aspect of this application provides a driver's driving risk analysis system, comprising a device and a platform. The device is used to collect facial video data, environmental video data, and vehicle driving data in a driving scenario; perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to generate behavioral events; and upload the behavioral events to the platform. The platform is used to sort the behavioral events uploaded by the device in chronological order to obtain a behavioral event sequence; construct combined events based on the behavioral event sequence; perform driving risk analysis on the combined events; determine whether the combined events are adverse events; and count the number of adverse events. If the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state poses a driving risk.

[0015] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method as described in any of the first aspects.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0017] A fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the steps of the method described in any of the first aspects.

[0018] Beneficial effects

[0019] The beneficial effects of the embodiments in this application compared with the prior art are:

[0020] By collecting facial video data, environmental video data, and vehicle driving data in driving scenarios, and performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct combined events, driving risk analysis is performed on the combined events to determine whether the combined events are adverse events. The number of adverse events is counted, and if the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state poses a driving risk. This method can generate combined events from all behavioral event information that occurs within a short period of time, uses a larger receptive field to generate more accurate event descriptions, prevents interference caused by device vibration, and solves the problems of single data reliance, strong subjectivity, and low accuracy in the analysis process. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating the implementation of a driver's driving risk analysis method provided in this application embodiment;

[0023] Figure 2 A flowchart illustrating a method for constructing combined events in the driver risk analysis method provided in this application embodiment;

[0024] Figure 3 A flowchart illustrating a method for driver state detection in the driver's driving risk analysis method provided in this application embodiment;

[0025] Figure 4 A flowchart illustrating one method for determining adverse events in the driver risk analysis method provided in this application embodiment;

[0026] Figure 5 A system diagram of a driver's driving risk analysis system provided in this application embodiment;

[0027] Figure 6 A basic structural block diagram of a driver's driving risk analysis device provided in an embodiment of this application;

[0028] Figure 7 This is a basic structural block diagram of an electronic device provided in an embodiment of this application.

[0029] Embodiments of the present invention

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0032] Please see Figure 1 , Figure 1 This application provides a flowchart of a method for analyzing driving risks for drivers. The method described in this application may specifically include steps S11 to S13.

[0033] S11: Collect facial video data, environmental video data, and vehicle driving data in driving scenarios, and perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct combined events.

[0034] In this embodiment, the DMS camera is installed on the dashboard in front of the driver. It captures images of the driver's face to obtain facial video data. Based on the facial information displayed in the video data, data on the driver's facial position, key facial features, facial expression, and eye opening / closing status (mental state) can be obtained. ADAS cameras are installed around the vehicle to capture images of the surrounding environment, acquiring environmental video data. Based on the surrounding environment information displayed in the images, driving scenario data such as the vehicle's lane position, the positions of surrounding vehicles, and the positions of surrounding pedestrians can be obtained. Vehicle sensors, including a six-axis sensor and a GPS sensor, can detect the vehicle's motion and collect vehicle driving data. This driving data includes speed data, angle data, and turn signal data. In this embodiment, pre-trained neural network models of different types can be used to analyze the facial video data, environmental video data, and vehicle driving data to obtain various types of behavioral events. Specifically, different types of behavioral events include, but are not limited to, events such as closing eyes, glancing down, yawning, frontal collision, lane change, acceleration, deceleration, turning, rapid acceleration, rapid deceleration, and sharp turning. It can be understood that behavioral events are sequential events with temporal information.

[0035] For example, for driver face images and road environment images, a closed-eye detection model can be used to perform behavioral event analysis on the driver face image. If the driver closes their eyes, a closed-eye event can be obtained. A yawning detection model can be used to perform behavioral event analysis on the driver face image. If the driver glances down, a glance-down event can be obtained. A yawning detection model can be used to perform behavioral event analysis on the driver face image. If the driver yawns, a yawning event can be obtained. An expression detection model can be used to perform behavioral event analysis on the driver face image. If the driver has a specific expression, such as a smiling expression, a crying expression, an angry expression, or a painful expression, the corresponding expression event can be obtained. A forward collision warning model can be used to perform behavioral event analysis on the road environment image. If the driver's current vehicle may be involved in a forward collision, a forward collision event can be obtained. A lane departure model can be used to perform behavioral event analysis on the road environment image. If the driver's current vehicle may be involved in a lane change, a lane change event can be obtained. For vehicle driving data, speed analysis models can be used to perform behavioral event analysis on speed data, obtaining acceleration events, deceleration events, rapid acceleration events, or rapid deceleration events, etc.; similarly, turn signal analysis models can be used to perform behavioral event analysis on turn signal data, obtaining turning events or sharp turning events, etc. After obtaining various types of behavioral events through behavioral event analysis, multiple behavioral events can be combined according to their time sequence to construct a combined event. In some specific implementations, behavioral events occurring within a set time period can be combined to form a combined event.

[0036] In some specific embodiments, facial video data, environmental video data, and vehicle driving data are all time-series data, continuously collected in real time by cameras and sensors, forming a data queue. In this embodiment, driving risk analysis can be performed on the driver based on the data queue and according to the first-in, first-out (FIFO) principle.

[0037] S12: Perform driving risk analysis on the combined events to determine whether the combined events are adverse events.

[0038] In this embodiment, a driving risk analysis model is pre-trained using a neural network model. This model is used to calculate scores for different attribute information of various behavioral events in a combined event, resulting in a risk score for the combined event. Based on the risk score, it is determined whether the combined event is a negative event. Specifically, a score threshold for measuring negative and positive events can be preset. After obtaining the risk score of the combined event, the risk score is compared with the preset score threshold. Combined events with risk scores reaching the threshold are judged as negative events, while combined events with risk scores below the threshold are judged as positive events.

[0039] In some specific embodiments, a deep learning model with multimodal classification capabilities can be trained to detect the collected images and information, classify a limited set of combined events and their evaluation scores, and determine whether each combined event is a poor-quality event based on the evaluation scores.

[0040] S13: Count the number of adverse events. If the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state poses a driving risk.

[0041] In this embodiment, by continuously collecting driver facial video data, vehicle environmental video data, and vehicle driving data in driving scenarios, multiple combined events can be obtained by continuously constructing combined events in chronological order. Driving risk analysis is performed on each combined event individually to determine whether it is a negative event, and the number of negative events is counted in real time. The number of negative events is used to measure the quality of the driver's driving habits; the more negative events, the worse the driver's driving habits and the higher the driving risk. In this embodiment, a preset quantity threshold can be used as a standard for judging whether the driver has driving risks. After counting the number of negative events, the number of negative events is compared with the preset quantity threshold. If the number of negative events reaches the preset quantity threshold, the judgment result is that the driver's current driving risk exists.

[0042] As can be seen from the above, the driver risk analysis method provided in this application collects facial video data, environmental video data, and vehicle driving data, and performs behavioral event analysis on these data to construct combined events. It then performs driving risk analysis on these combined events to determine whether they are adverse events. Finally, it counts the number of adverse events; if the number reaches a preset threshold, it determines that the driver's current driving state poses a driving risk. The generation of combined events includes time-series information, which can characterize not only the driving behavior itself but also the motivation behind the behavior and the driving environment at the time. This increases the receptive field, enabling more accurate judgments of the driver's behavior and improving the accuracy of driver risk analysis. This addresses the problems of limited data, strong subjectivity, and low accuracy in driver risk analysis.

[0043] In some embodiments of this application, please refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for constructing combined events in the driver's driving risk analysis method provided in this application embodiment. Specifically, it may include steps S21 to S24.

[0044] S21: Perform driver state detection on the face video data and / or the environmental video data to obtain a first behavioral event characterizing the driver state. The first behavioral event includes the type of the event, the start time of the event, the end time of the event, and a probability score sequence of the event between the start time and the end time.

[0045] S22: Perform vehicle state detection on the vehicle driving data to obtain a second behavioral event characterizing the vehicle state. The second behavioral event includes the type of the event, the start time of the event, the end time of the event, and a sequence of driving parameter values ​​between the start time and the end time.

[0046] S23: Sort all first-behavioral events and all second-behavioral events on the timeline to obtain a sequence of behavioral events;

[0047] S24: Based on the target behavior event, obtain the combined event corresponding to the target behavior event from the sequence of behavior events.

[0048] In this embodiment, different detection models can be trained for different types of behavioral events. Each detection model contains an algorithm for confirming the corresponding type of behavioral event. The input data for each detection model can be obtained from the acquired face video data, environmental video data, and vehicle driving data, according to the input requirements of each event confirmation algorithm. The event confirmation algorithm in each detection model performs behavioral analysis based on its input data to confirm whether the driver has executed the behavioral event corresponding to the event confirmation algorithm in the current state. If the event confirmation algorithm confirms that the driver has executed the corresponding behavioral event, it obtains the information of the behavioral event and generates the behavioral event.

[0049] Specifically, facial video data and environmental video data can be used to detect driver state and obtain a first behavioral event characterizing the driver's state. Specifically, the first behavioral event can include information such as the event type, start time, end time, and probability score sequence between the start and end times. For example, consider an eye-closing event. For this type of event, an eye-closing detection model can be pre-trained, and an algorithm can be configured to determine whether the driver has performed the eye-closing behavior. The input data to the eye-closing detection model can be eye feature data from each frame of the facial image in the facial video data. The event confirmation algorithm can determine whether the driver has performed the eye-closing behavior based on the eye feature data. If it is determined that the driver has performed the eye-closing behavior, an eye-closing behavior event can be generated. Among them, the type of the closed-eye behavior event is closed-eye; the start time of the closed-eye behavior event is the time when the first frame of the detected area contains a face image frame with eye features that match the closed-eye behavior; the end time of the closed-eye behavior event is the time when the last frame contains a face image frame with eye features that match the closed-eye behavior; and the probability score sequence of the event between the start time and the end time is the probability score sequence obtained by sorting the probability scores of each image frame between the start time and the end time as the driver has closed-eye behavior in chronological order.

[0050] Specifically, vehicle driving data can be used to detect vehicle status and obtain a second behavioral event characterizing the vehicle status. Specifically, the second behavioral event can include information such as the event type, the event start time, the event end time, and the sequence of driving parameter values ​​between the start and end times. For example, consider an acceleration event. For this type of event, a corresponding event confirmation algorithm can be pre-configured. This algorithm analyzes the vehicle driving data to determine whether the driver's vehicle is currently accelerating. If it is determined that the driver's vehicle is currently accelerating, an acceleration behavior event can be generated. Here, the acceleration behavior event type is acceleration; the acceleration behavior event start time is the time when the vehicle is detected to begin accelerating; the acceleration behavior event end time is the time when the vehicle is detected to stop accelerating; and the sequence of driving parameter values ​​between the start and end times is a sequence of speed parameter values ​​obtained by arranging the speed parameter values ​​corresponding to each time point between the start and end times in chronological order.

[0051] Different types of detection models and event confirmation algorithms can be used for simultaneous detection, which is timely and efficient. After obtaining the first and second behavioral events, based on the time of occurrence of each behavioral event, all the obtained first and second behavioral events can be sorted on the timeline to obtain a behavioral event sequence. A combined event can be a target behavioral event and other behavioral events on the timeline that are within the same set time period as the target behavioral event. In this embodiment, each detected behavioral event can be taken as a target behavioral event, and the combined event corresponding to the target behavioral event can be obtained from the behavioral event sequence based on the target behavioral event. For example, assuming that the set time period used to construct the combined event is 5 seconds, and the target behavioral event is the acceleration event that occurs at 13:33:07, then all behavioral events that occurred within 5 seconds before 13:33:07 can be obtained from the behavioral event sequence. That is, all behavioral events that occurred between 13:33:02 and 13:33:07 can form a combined event. This embodiment can encompass as many possible combinations of events as possible, without limiting the number of types of combined events. It can uncover more potential driving habit data, more accurately determine the driver's driving habits, and more accurately analyze the driver's driving risks.

[0052] In some embodiments of this application, please refer to Figure 3 , Figure 3 This is a flowchart illustrating a method for driver state detection in the driver's driving risk analysis method provided in this application embodiment. Specifically, it may include steps S31 to S33.

[0053] S31: Perform frame segmentation processing on the face video data and / or the environment video data to obtain a video image frame sequence;

[0054] S32: Perform feature extraction processing on each video image in the video image frame sequence to obtain a feature image sequence;

[0055] S33: Input the feature image sequence into a preset driver state detection model to detect the driver state and obtain the first behavioral event representing the driver state.

[0056] In this embodiment, by performing frame segmentation on the video data, the video data can be divided into individual image frames. These image frames are then sorted sequentially to obtain a corresponding video image frame sequence. In this embodiment, driver state detection models can be pre-trained for different types of behavioral events. Furthermore, according to the input requirements of each driver state detection model, feature extraction processing is performed on each video image frame in the video image frame sequence to obtain feature image sequences that serve as input data for various driver state detection models. For example, driver state detection models include, but are not limited to, closed-eye detection models, downward-looking detection models, yawning detection models, facial expression detection models, forward collision warning models, and lane departure detection models. It is understood that each driver state detection model corresponds to a feature image sequence; for example, the input data for a closed-eye detection model can be a feature image sequence of the eye area on a face, and the input data for a yawning detection model can be a feature image sequence of the mouth area on a face. In this embodiment, by inputting the feature image sequence into the driver state detection model, the driver state detection model calculates the probability score of each feature image corresponding to a driving behavior by performing pixel value calculations on the input feature images. The calculated probability scores are then compared one by one with a preset score threshold to determine whether the probability score exceeds the preset score threshold. If the probability scores corresponding to multiple consecutive frames (e.g., 10 consecutive frames) of feature images exceed the preset score threshold, the probability scores corresponding to the multiple consecutive frames of feature images are sorted by time to generate a probability score sequence. The timestamp of the first feature image in the multiple consecutive frames of feature images is taken as the start time of the event, and the timestamp of the last feature image in the multiple consecutive frames of feature images is taken as the end time of the event. Then, the probability score sequence, the start time, the end time, and the type of event detected by the driver state detection model are packaged together to form a behavior event for output, thereby obtaining the first behavior event representing the driver state.

[0057] In some embodiments of this application, for the feature images used for driver state analysis, pixel regularization processing can be performed on the pixel values ​​in the feature images using statistical mean and standard deviation. This can prevent overfitting of data when the feature image sequence is input into the driver state detection model for driver state detection.

[0058] In some embodiments of this application, to consider the impact of weather conditions on the analysis of driver driving risks, weather state information under the driving scenario can also be collected, such as rainy days, snowy days, foggy days, and sunny days. After obtaining the weather state information, the weather state information is fused with time to obtain weather events. All the obtained first behavioral events, all the second behavioral events, and the weather events are sorted together to obtain a behavioral event sequence. This realizes the inclusion of weather state information as an influencing factor in driving risk analysis, which can improve the accuracy of driving risk analysis.

[0059] In some embodiments of this application, to consider the impact of road conditions on the analysis of driver driving risk, road information in the driving scenario can also be collected, such as traffic lights, intersections, highways, passageways, provincial roads, etc. After obtaining the road information, the road information and time are fused to obtain road condition events. All the obtained first behavioral events, all the second behavioral events, and the road condition events are sorted together to obtain a behavioral event sequence. This realizes the inclusion of road information as an influencing factor in driving risk analysis, which can improve the accuracy of driving risk analysis.

[0060] In some embodiments of this application, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a method for determining adverse events in a driver risk analysis method provided in this application. Specifically, it may include steps S41 to S43.

[0061] S41: The attribute data of each behavioral event in the combined event are numerically encoded according to the time sequence to generate a high-dimensional matrix for representing the combined event;

[0062] S42: Input the high-dimensional matrix into the preset driving risk calculation model for numerical calculation, and output the risk score corresponding to the combined event;

[0063] S43: Compare the risk score with a preset score threshold. If the risk score reaches the preset score threshold, the combined event is judged as a bad event.

[0064] In this embodiment, a combined event may contain multiple behavioral events, each with multiple attribute data of different information dimensions, such as behavior type, probability score, occurrence time, duration, vehicle speed, and vehicle direction. In this embodiment, these attribute data can be numerically encoded according to time sequence to obtain a high-dimensional matrix representing the combined event. This high-dimensional matrix contains the attribute data of all behavioral events in the combined event. By inputting this high-dimensional matrix into a preset driving risk calculation model, the driving risk calculation model performs numerical calculations on various attributes to obtain a score, which is then output as the risk score corresponding to the combined event. It is understood that the driving risk calculation model is a convolutional neural network model trained to convergence, trained to calculate the risk score of the combined event based on the high-dimensional matrix. The convolutional neural network (CNN) model can be either a CNN or a VGG model. By training the CNN model to convergence using a large amount of sample data (e.g., a combination of various behavioral events), the model gains the ability to calculate the risk score of the combination of events based on a high-dimensional matrix. In this embodiment, a preset score threshold for measuring adverse and benign events is used. After the driving risk calculation model calculates the risk score of the combination of events, it compares the risk score with the threshold. If the risk score reaches the preset threshold, the combination of events is classified as adverse; otherwise, it is classified as benign. This embodiment also encodes the attribute data of each behavioral event in the combination of events according to time sequence to obtain a high-dimensional matrix. This high-dimensional matrix contains temporal information, representing information about the behavior itself, the motivation behind the behavior, and the environmental information at the time of the behavior. This increases the receptive field for analyzing driver driving risk and improves the accuracy of the analysis.

[0065] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0066] In some embodiments of this application, please refer to Figure 5 , Figure 5 This is a schematic diagram of a driver's driving risk analysis system provided in an embodiment of this application. Figure 5As shown, the driver risk analysis system includes a device terminal and a platform terminal. The device terminal can be an in-vehicle terminal, and the platform terminal can be a cloud server. The device terminal can train different detection models or configure different event confirmation algorithms for different types of events to confirm the existence of corresponding events. Specifically, the in-vehicle terminal can acquire facial video data from the vehicle's DMS camera, environmental video data from the vehicle's ADAS camera, vehicle driving data from the vehicle's six-axis sensor, and road condition data from the vehicle's GPS sensor. Then, depending on the data acquisition channel, different detection models are used to detect and derive the corresponding events. All the obtained events are then uploaded to the platform terminal for further driving risk analysis. After receiving the events uploaded by the device terminal, the platform terminal sorts the events according to time sequence to obtain a behavioral event sequence. Based on the behavioral event sequence, it constructs combined events, performs driving risk analysis on the combined events, determines whether the combined events are adverse events, and counts the number of adverse events. If the number of adverse events reaches a preset threshold, it determines that the driver's current driving state poses a driving risk, thus evaluating the driver's driving habits. This embodiment allows for the separate deployment of detection and analysis algorithms on the device and platform sides, reducing the hardware requirements of the algorithm deployment environment. Device-side detection, as described in this application, essentially extracts information from video evidence, significantly reducing bandwidth consumption when uploading videos to the platform, protecting user privacy, and facilitating subsequent statistical analysis of driver behaviors. Furthermore, on the platform side, these combined events that reveal driver driving habits can be stored as a chain of evidence, facilitating traffic supervision.

[0067] For some specific embodiments, please refer to Figure 6 , Figure 6 This is a basic structural block diagram of a driver's driving risk analysis device provided in an embodiment of this application. In this embodiment, the device includes units used to perform the steps in the above-described method embodiments. Please refer to the relevant descriptions in the above-described method embodiments for details. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 6 As shown, the driver's driving risk analysis device includes a construction module 61, an analysis module 62, and a judgment module 63. The construction module 61 collects facial video data, environmental video data, and vehicle driving data in a driving scenario, and performs behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct combined events. The analysis module 62 performs driving risk analysis on the combined events and determines whether the combined events are adverse events. The judgment module 63 counts the number of adverse events; if the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state poses a driving risk.

[0068] It should be understood that the driver's driving risk analysis device in this embodiment corresponds one-to-one with the driver's driving risk analysis method applied to the first vehicle terminal, and will not be described again here.

[0069] In some embodiments of this application, please refer to Figure 7 , Figure 7 This is a basic structural block diagram of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71, such as the program for the driving risk analysis methods for each driver described above. When the processor 71 executes the computer program 73, it implements the steps in each embodiment of the driving risk analysis methods for each driver described above. Alternatively, when the processor 71 executes the computer program 73, it implements the functions of each module in the embodiment corresponding to the driving risk analysis device for each driver described above. Please refer to the relevant descriptions in the embodiments for details, which will not be repeated here.

[0070] For example, the computer program 73 can be divided into one or more modules (units), which are stored in the memory 72 and executed by the processor 71 to complete this application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 73 in the electronic device 7. For example, the computer program 73 can be divided into a construction module, an analysis module, and a judgment module, with the functions of each module as described above.

[0071] The electronic device may include, but is not limited to, a processor 71 and a memory 72. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0072] The processor 71 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0073] The memory 72 can be an internal storage unit of the electronic device 7, such as a hard disk or memory. The memory 72 can also be an external storage device of the electronic device 7, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 72 can include both internal and external storage units of the electronic device 7. The memory 72 is used to store the computer program and other programs and data required by the electronic device. The memory 72 can also be used to temporarily store data that has been output or will be output.

[0074] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0075] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above. In this embodiment, the computer-readable storage medium can be either non-volatile or volatile.

[0076] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0077] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0078] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0079] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0080] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for analyzing driving risks, characterized in that, The method includes: Collect facial video data, environmental video data, and vehicle driving data in driving scenarios, and perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct combined events; Perform driving risk analysis on the combined events to determine whether the combined events are adverse events; The number of adverse events is counted. If the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state poses a driving risk. The step of performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct combined events includes: Driver state detection is performed on the face video data and / or the environmental video data to obtain a first behavioral event characterizing the driver state. The first behavioral event includes the type of the event, the start time of the event, the end time of the event, and a probability score sequence of the event between the start time and the end time. Vehicle state detection is performed on the vehicle driving data to obtain a second behavioral event characterizing the vehicle state. The second behavioral event includes the start time of the event, the end time of the event, and a sequence of driving parameter values ​​between the start time and the end time. Sort all first-behavioral events and all second-behavioral events on the timeline to obtain a sequence of behavioral events; Based on the target behavior event, obtain the combined event corresponding to the target behavior event from the sequence of behavior events; The combined event consists of a target behavior event and other behavior events that are in the same set time period as the target behavior event on the time axis. Each detected behavior event is taken as the target behavior event, and the combined event corresponding to the target behavior event is obtained from the behavior event sequence based on the target behavior event.

2. The driver risk analysis method according to claim 1, characterized in that, The step of detecting driver state from the facial video data and / or the environmental video data to obtain a first behavioral event characterizing the driver state includes: The face video data and / or the environmental video data are processed by frame segmentation to obtain a video image frame sequence; Feature extraction processing is performed on each video image in the video image frame sequence to obtain a feature image sequence; The feature image sequence is input into a preset driver state detection model to detect the driver state and obtain the first behavioral event representing the driver state.

3. The driver risk analysis method according to claim 2, characterized in that, Before the step of inputting the feature image sequence into a preset driver state detection model for driver state detection and obtaining the first behavioral event representing the driver state, the method further includes: Pixel regularization is performed on each frame of the feature image sequence.

4. The driver risk analysis method according to claim 1, characterized in that, Before the step of sorting all first behavioral events and all second behavioral events obtained on the timeline to obtain a sequence of behavioral events, the method further includes: Collect weather status information in driving scenarios, and fuse the weather status information with time data to obtain weather events; The step of sorting all first behavioral events and all second behavioral events obtained on the timeline to obtain a sequence of behavioral events includes: All first behavioral events, all second behavioral events, and the weather events obtained are sorted together on the timeline to obtain a sequence of behavioral events.

5. The driver risk analysis method according to claim 1, characterized in that, Before the step of sorting all first behavioral events and all second behavioral events obtained on the timeline to obtain a sequence of behavioral events, the method further includes: Collect road information in driving scenarios, fuse the road information with time data, and obtain road condition events; The step of sorting all first behavioral events and all second behavioral events obtained on the timeline to obtain a sequence of behavioral events includes: All first behavioral events, all second behavioral events, and the road condition events obtained are sorted together on the timeline to obtain a sequence of behavioral events.

6. The driver risk analysis method according to any one of claims 1-5, characterized in that, The step of performing driving risk analysis on the combined events and determining whether the combined events are adverse events includes: The attribute data of each behavioral event in the combined event are numerically encoded according to the time sequence to generate a high-dimensional matrix representing the combined event; The high-dimensional matrix is ​​input into a preset driving risk calculation model for numerical calculation, and the risk score corresponding to the combined event is output. The risk score is compared with a preset score threshold. If the risk score reaches the preset score threshold, the combined event is judged as a bad event.

7. A driver's driving risk analysis system, characterized in that, The driver's driving risk analysis system includes a device-side and a platform-side, wherein: The device is used to collect facial video data, environmental video data, and vehicle driving data in driving scenarios, perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data, generate behavioral events, and upload the behavioral events to the platform. The platform is used to sort the behavioral events uploaded by the device according to the time sequence to obtain a behavioral event sequence, construct a combined event based on the behavioral event sequence, perform driving risk analysis on the combined event, determine whether the combined event is a bad event, and count the number of bad events. If the number of bad events reaches a preset number threshold, it is determined that the driver's current driving state has a driving risk. The steps of performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data through the device, and constructing combined events through the platform, include: Driver state detection is performed on the face video data and / or the environmental video data to obtain a first behavioral event characterizing the driver state. The first behavioral event includes the type of the event, the start time of the event, the end time of the event, and a probability score sequence of the event between the start time and the end time. Vehicle state detection is performed on the vehicle driving data to obtain a second behavioral event characterizing the vehicle state. The second behavioral event includes the start time of the event, the end time of the event, and a sequence of driving parameter values ​​between the start time and the end time. Sort all first-behavioral events and all second-behavioral events on the timeline to obtain a sequence of behavioral events; Based on the target behavior event, obtain the combined event corresponding to the target behavior event from the sequence of behavior events; The combined event consists of a target behavior event and other behavior events that are in the same set time period as the target behavior event on the time axis. Each detected behavior event is taken as the target behavior event, and the combined event corresponding to the target behavior event is obtained from the behavior event sequence based on the target behavior event.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

Images