Driving risk analysis method and system for driver, and electronic device and storage medium

By collecting and analyzing multiple data in driving scenarios, building a combined event and conducting risk assessment, the problems of single and subjective data in the existing technology are solved, and the accuracy of driver driving risk analysis is improved.

WO2024243804A1PCT designated stage expired Publication Date: 2024-12-05SHENZHEN STREAMING VIDEO TECH

Patent Information

Application Number
PCT/CN2023/097144
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The existing driver driving risk analysis methods rely on single data, are highly subjective and have low accuracy, making it difficult to accurately judge the driver's driving status.

Method used

By collecting face video data, environmental video data and vehicle driving data in driving scenarios, a combined event is constructed, and a preset driving risk analysis model is used to determine whether the combined event is an inferior event, and the number of inferior events is counted to judge the driver's driving risk .

Benefits of technology

It improves the accuracy of driver driving risk analysis, prevents analysis deviations caused by single data, increases the receptive field, and reduces interference caused by equipment jitter.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023097144_05122024_PF_FP_ABST
    Figure CN2023097144_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A driving risk analysis method for a driver. The method comprises: collecting facial video data, environmental video data and vehicle traveling data in a driving scenario, and performing behavior event analysis on the facial video data, the environmental video data and the vehicle traveling data to construct a combined event; performing driving risk analysis on the combined event, and determining whether the combined event is an inferior event; and counting the number of inferior events, and if the number of inferior events reaches a preset number threshold value, determining that the current driving state of a driver poses a driving risk. Also disclosed are a driving risk analysis system for a driver, and an electronic device and a storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Driver's driving risk analysis method, system, electronic device and storage medium Technical Field

[0001] The present application relates to the field of vehicle driving technology, and in particular to a method, system, electronic device, and storage medium for analyzing a driver's driving risk. Background Art

[0002] With the increasing prevalence of vehicles, traffic accidents are also increasing. The driver's driving condition has a significant impact on driving safety. Therefore, it is important to ensure that the driver is in good driving condition as much as possible. Driver fatigue, distraction, and other issues while driving can lead to a decrease in driving judgment, making road accidents more likely to occur. In recent years, research on analyzing driver driving risk has increased. Currently, existing solutions for analyzing driving risk use a single sensor's single video frame to determine the driver's driving behavior and analyze the driver's driving risk. The analysis process relies on a single set of data, often relying on the analyst's experience, which is highly subjective. The resulting analysis conclusions are prone to bias and lack accuracy. Technical issues

[0003] In view of this, the embodiments of the present application provide a driver's driving risk analysis method, system, electronic device and storage medium, which can improve the accuracy of the driver's driving risk analysis. Technical Solutions

[0004] A first aspect of an embodiment of the present application provides a driver driving risk analysis method, comprising: collecting facial video data, environmental video data, and vehicle driving data in a driving scene, and performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct a combined event; performing driving risk analysis on the combined event to determine whether the combined event is a bad event; counting the number of bad events, and if the number of bad events reaches a preset number threshold, determining that the driver's current driving state has a driving risk.

[0005] In one possible implementation, the step of performing behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct a combined event includes: performing driver status detection on the facial video data and / or the environmental video data to obtain a first behavioral event representing the driver status, wherein the first behavioral event includes the type of event, the start time of the event, the end time of the event, and a sequence of probability scores of the event between the start time and the end time; performing vehicle status detection on the vehicle driving data to obtain a second behavioral event representing the vehicle status, wherein the start time of the event, the end time of the event, and a sequence of driving parameter values ​​of the event between the start time and the end time; sorting all the first behavioral events and all the second behavioral events obtained on the time axis to obtain a behavioral event sequence; and according to the target behavioral event, obtaining a combined event corresponding to the target behavioral event from the behavioral event sequence.

[0006] In one possible implementation, the step of performing driver status detection on the facial video data and / or the environmental video data to obtain a first behavioral event representing the driver status includes: performing frame processing on the facial video data and / or the environmental video data to obtain a video image frame sequence; performing feature extraction processing on each frame of the video image in the video image frame sequence to obtain a feature image sequence; and inputting the feature image sequence into a preset driver status detection model to perform driver status detection to obtain a first behavioral event representing the driver status.

[0007] In one possible implementation, the step of inputting the feature image sequence into a preset driver state detection model to perform driver state detection and obtaining a first behavioral event representing the driver state further includes: performing pixel regularization processing on each frame of the feature image in the feature image sequence.

[0008] In one possible implementation, before the step of sorting all the first behavior events and all the second behavior events obtained on the timeline to obtain a behavior event sequence, the step also includes: collecting weather status information in the driving scenario, fusing the weather status information with time, and obtaining weather events; the step of sorting all the first behavior events and all the second behavior events obtained on the timeline to obtain a behavior event sequence includes: sorting all the first behavior events and all the second behavior events obtained and the weather events together on the timeline to obtain a behavior event sequence.

[0009] In one possible implementation, before the step of sorting all the first behavior events and all the second behavior events obtained on the timeline to obtain a behavior event sequence, the step also includes: collecting road information in the driving scenario, fusing the road information with time, and obtaining road condition events; the step of sorting all the first behavior events and all the second behavior events obtained on the timeline to obtain a behavior event sequence includes: sorting all the first behavior events and all the second behavior events obtained and the road condition events together on the timeline to obtain a behavior event sequence.

[0010] In one possible implementation, the step of performing a driving risk analysis on the combined event and determining whether the combined event is a poor quality event includes: numerically encoding the attribute data of each behavioral event in the combined event in time series to generate a high-dimensional matrix for representing the combined event; inputting the high-dimensional matrix into a preset driving risk calculation model for numerical calculation, and outputting a risk score corresponding to the combined event; comparing the risk score with a preset score threshold; and if the risk score reaches the preset score threshold, determining that the combined event is a poor quality event.

[0011] A second aspect of an embodiment of the present application provides a driver's driving risk analysis system, which includes a device side and a platform side, wherein: the device side is used to collect facial video data, environmental video data, and vehicle driving data in a driving scene, perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data, generate behavioral events, and upload the behavioral events to the platform side; the platform side is used to sort the behavioral events uploaded by the device side in time sequence, obtain a behavioral event sequence, construct a combined event based on the behavioral event sequence, perform driving risk analysis on the combined event, determine whether the combined event is a bad event, and count the number of bad events. If the number of bad events reaches a preset number threshold, it is determined that the driver's current driving state has a driving risk.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and is characterized in that when the computer program is executed by a processor, it implements the steps of the method described in any one of the first aspects.

[0014] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device implements the steps of any one of the methods described in the first aspect when executing the computer program product. Beneficial effects

[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0016] By collecting facial video data, environmental video data and vehicle driving data in driving scenarios, and performing behavioral event analysis on the facial video data, environmental video data and vehicle driving data, a combined event is constructed; a driving risk analysis is performed on the combined event to determine whether the combined event is a bad event; the number of bad events is counted, and if the number of bad events reaches a preset threshold, it is determined that the driver's current driving state has a driving risk. It is possible to generate a combined event through all behavioral event information that occurs continuously over a short period of time, using a larger receptive field to generate a more accurate event description, which can prevent interference caused by device jitter and solve the problem that the data based on the analysis process is single, highly subjective and low in accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] FIG1 is a flowchart of a method for analyzing a driver's driving risk according to an embodiment of the present application;

[0019] FIG2 is a flow chart of a method for constructing a combined event in a method for analyzing a driver's driving risk provided by an embodiment of the present application;

[0020] FIG3 is a flowchart of a method for implementing driver status detection in the driver's driving risk analysis method provided in an embodiment of the present application;

[0021] FIG4 is a flowchart illustrating a method for determining adverse events in a driver's driving risk analysis method provided in an embodiment of the present application;

[0022] FIG5 is a system diagram of a driver's driving risk analysis system provided in an embodiment of the present application;

[0023] FIG6 is a basic structural block diagram of a driving risk analysis device for a driver provided in an embodiment of the present application;

[0024] FIG7 is a basic structural block diagram of an electronic device provided in an embodiment of the present application. Modes for Carrying Out the Invention

[0025] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0026] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0027] Please refer to Figure 1, which is a flowchart of a method for implementing a driver's driving risk analysis method provided in an embodiment of the present application. The method in the embodiment of the present application may specifically include steps S11 to S13.

[0028] S11: Collecting facial video data, environmental video data, and vehicle travel data in a driving scene, and performing behavioral event analysis on the facial video data, environmental video data, and vehicle travel data to construct a combined event.

[0029] In this embodiment, a DMS camera is mounted on the dashboard in front of the driver. The DMS camera can be used to capture the driver's face and obtain facial video data. Based on the facial information displayed in the facial video data, mental state data such as the driver's facial position, facial key points, driver's expression, and eye opening and closing can be obtained. ADAS cameras are mounted on all four sides of the vehicle and can be used to capture the vehicle's surroundings and obtain environmental video data. Based on the environmental information displayed in the environmental image, driving scene data such as the vehicle's lane position, the positions of surrounding vehicles, and the positions of surrounding pedestrians can be obtained. Vehicle sensors include six-axis sensors and GPS sensors installed on the vehicle. These sensors can detect the vehicle's motion state and collect vehicle driving data. It is understood that vehicle driving data includes speed data, angle data, and turn signal data. In this embodiment, different types of pre-trained neural network models can be used to perform behavioral event analysis on the facial video data, environmental video data, and vehicle driving data, respectively, to obtain various types of behavioral events. Specifically, different types of behavioral events include, but are not limited to, eye closing events, downcasting events, yawning events, front collision events, lane change events, acceleration events, deceleration events, turning events, sudden acceleration events, sudden deceleration events, and sharp turning events. It is understood that behavioral events are time-series events with time-series information.

[0030] For example, for the driver's facial image and the road environment image, the driver's facial image can be analyzed for behavioral events using the eye closure detection model. If the driver has closed eyes, an eye closure event can be obtained. The driver's facial image can be analyzed for behavioral events using the yawn detection model. If the driver has a downward glance, a downward glance event can be obtained. The driver's facial image can be analyzed for behavioral events using the yawn detection model. If the driver has a yawn, a yawn event can be obtained. The driver's facial image can be analyzed for behavioral events using the expression detection model. If the driver has a specific expression, such as a smile, a cry, an anger, or a pain, the corresponding expression event can be obtained. The road environment image can be analyzed for behavioral events using the front collision warning model. If the driver's current vehicle may have a front collision, a front collision event can be obtained. The road environment image can be analyzed for behavioral events using the lane departure model. If the driver's current vehicle may have a lane change, a lane change event can be obtained. For vehicle driving data, a speed analysis model can be used to perform behavioral event analysis on the speed data to obtain acceleration events, deceleration events, sudden acceleration events, or sudden deceleration events. A turning analysis model can be used to perform behavioral event analysis on turn signal data to obtain turning events or sharp turning events. After obtaining various types of behavioral events through behavioral event analysis, multiple behavioral events can be combined together in a time sequence to construct a combined event. In some specific embodiments, behavioral events occurring within a set time length can be combined together to form a combined event.

[0031] In some specific embodiments, facial video data, environmental video data, and vehicle driving data are all time-series data, collected continuously and in real time by cameras and sensors, forming a data queue. In this embodiment, based on the data queue, a driving risk analysis of the driver can be performed according to the first-in, first-out principle.

[0032] S12: Perform driving risk analysis on the combined event to determine whether the combined event is a negative event.

[0033] In this embodiment, a driving risk analysis model is pre-trained using a neural network model. This driving risk analysis model is used to calculate scores based on the different attribute information of various behavioral events within a combination of events, deriving a risk score for the combination of events. This risk score is then used to determine whether the combination of events is a poor or good event. Specifically, a score threshold can be preset to distinguish between poor and good events. After obtaining the risk score for a combination of events, the risk score is compared with the preset score threshold. Combination events with risk scores that reach the threshold are judged as poor or good events, while combination events with risk scores that do not reach the threshold are judged as good events.

[0034] In some specific embodiments, a deep learning model with multimodal classification capabilities can also be trained to detect the collected images and information, classify the limited combination events and evaluation scores of the events predetermined by the model, and determine whether each combination event is a low-quality event based on the evaluation score.

[0035] S13: Counting the number of adverse events. If the number of adverse events reaches a preset threshold, it is determined that the driver's current driving state has a driving risk.

[0036] In this embodiment, by continuously collecting the driver's facial video data, the vehicle's environmental video data, and the vehicle's driving data in the driving scene, and continuously constructing combined events in a time sequence, multiple combined events can be obtained. By performing a driving risk analysis on each combined event one by one, it is determined whether each combined event is a bad event, and then the number of bad events is counted in real time. The quality of the driver's driving habits is measured according to the number of bad events. The more bad events there are, the worse the driver's driving habits are and the higher the driving risk. In this embodiment, a preset quantity threshold can be used as a standard for determining whether the driver has a driving risk. After counting the number of bad events, the number of bad events is compared with the preset quantity threshold. If the number of bad events reaches the preset quantity threshold, the judgment result is that the driver's current driving risk is a driving risk.

[0037] From the above, it can be seen that the driver's driving risk analysis method provided in the embodiment of the present application collects facial video data, environmental video data, and vehicle driving data, and performs behavioral event analysis on the facial video data, environmental video data, and vehicle driving data to construct a combined event; performs driving risk analysis on the combined event to determine whether the combined event is a bad event; and counts the number of bad events. If the number of bad events reaches a preset threshold, it is determined that the driver's current driving state has a driving risk. The generation of the combined event contains time series information, which can not only characterize the information of the driving behavior itself, but also characterize the motivation for the driving behavior and the driving environment at the time, thereby increasing the receptive field, making accurate judgments on the driver's driving behavior, improving the accuracy of the driver's driving risk analysis, and solving the problems of single data, strong subjectivity, and low accuracy when analyzing the driver's driving risk.

[0038] In some embodiments of the present application, please refer to FIG2 , which is a flow chart of a method for constructing a combined event in the driver's driving risk analysis method provided by an embodiment of the present application, which may specifically include steps S21 to S24 .

[0039] S21: Perform driver status detection on the face video data and / or the environment video data to obtain a first behavior event representing the driver status, where the first behavior event includes an event type, a start time of the event, an end time of the event, and a probability score sequence of the event between the start time and the end time;

[0040] S22: Performing vehicle state detection on the vehicle driving data to obtain a second behavior event representing the vehicle state, where the second behavior event includes an event type, a start time of the event, an end time of the event, and a sequence of driving parameter values ​​of the event between the start time and the end time;

[0041] S23: sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence;

[0042] S24: According to the target behavior event, obtaining a combination event corresponding to the target behavior event from the behavior event sequence.

[0043] In this embodiment, different detection models can be trained for different types of behavioral events. Each detection model includes an algorithm for identifying the corresponding type of behavioral event. The input data for each detection model can be derived from the acquired facial video data, environmental video data, and vehicle driving data, based on the input requirements of each event confirmation algorithm. The event confirmation algorithm within each detection model performs behavioral analysis based on its input data to determine whether the driver has, in the current state, executed the behavioral event corresponding to the event confirmation algorithm. If the event confirmation algorithm confirms that the driver has executed the corresponding behavioral event, information about the behavioral event is obtained and a behavioral event is generated.

[0044] Specifically, facial video data and environmental video data can be used to detect the driver's state and obtain a first behavioral event that characterizes the driver's state. Specifically, the first behavioral event may include information such as the type of event, the start time of the event, the end time of the event, and a probability score sequence of the event between the start time and the end time. For example, for an eye-closing event, an eye-closing detection model can be pre-trained for this type of event, and a confirmation algorithm for determining whether the driver has closed his eyes can be configured in the eye-closing detection model. The input data of the eye-closing detection model can be the eye feature data of each frame of the facial image in the facial video data, and the event confirmation algorithm can confirm whether the driver has closed his eyes based on the eye feature data. If it is determined that the driver has closed his eyes, an eye-closing behavior event can be generated. Among them, the type of the eyes-closing behavior event is eyes-closing, the start time of the eyes-closing behavior event is the time when the first frame of the detected face image frame contains eye features that meet the eyes-closing behavior, the end time of the eyes-closing behavior event is the time when the last frame of the detected face image frame contains eye features that meet the eyes-closing behavior, and the probability score sequence of the event between the start time and the end time is the probability score sequence obtained by sorting the probability scores of each image frame between the start time and the end time as the driver having eyes-closing behavior in time sequence.

[0045] Specifically, the vehicle driving data can be used to detect the vehicle state and obtain a second behavior event that represents the vehicle state. Specifically, the second behavior event may include information such as the type of event, the start time of the event, the end time of the event, and the sequence of driving parameter values ​​between the start time and the end time. For example, for an acceleration event, a corresponding event confirmation algorithm can be pre-configured for this type of event. The event confirmation algorithm is used to analyze the vehicle driving data to determine whether the vehicle driven by the driver is currently accelerating. If it is determined that the vehicle driven by the driver is currently accelerating, an acceleration behavior event can be generated. The type of the acceleration behavior event is acceleration, the start time of the acceleration behavior event is the time when the vehicle starts to accelerate, the end time of the acceleration behavior event is the time when the vehicle stops accelerating, and the sequence of driving parameter values ​​between the start time and the end time is the speed parameter value sequence obtained by arranging the speed parameter values ​​corresponding to each time point between the start time and the end time in time sequence.

[0046] Different types of detection models and event confirmation algorithms can perform detection synchronously, in a timely and efficient manner. After obtaining the first behavior event and the second behavior event, all the first behavior events and all the second behavior events obtained can be sorted on the timeline based on the time when each behavior event occurs to obtain a behavior event sequence. A combination event can be a target behavior event and other behavior events on the timeline that are within the same set time period as the target behavior event. In this embodiment, each detected behavior event can be used as a target behavior event one by one, and based on the target behavior event, a combination event corresponding to the target behavior event can be obtained from the behavior event sequence. For example, assuming that the set time period for constructing the combination event is 5 seconds, and the target behavior event is the acceleration event that occurs at 13:33:07, then all behavior events that occurred within 5 seconds before 13:33:07 can be obtained in the behavior event sequence, that is, all behavior events that occurred to the driver and the vehicle between 13:33:02 and 13:33:07 can constitute a combination event. Through this embodiment, all possible event combinations can be included as much as possible without limiting the number of types of combined events. More potential driving habit data can be mined, the driver's driving habits can be judged more accurately, and the driver's driving risks can be analyzed more accurately.

[0047] In some embodiments of the present application, please refer to FIG3 , which is a flowchart of a method for implementing driver status detection in the driver risk analysis method provided by the embodiment of the present application, which may specifically include steps S31 to S33 .

[0048] S31: performing frame processing on the face video data and / or the environment video data to obtain a video image frame sequence;

[0049] S32: performing feature extraction processing on each frame of the video image frame sequence to obtain a feature image sequence;

[0050] S33: Inputting the feature image sequence into a preset driver state detection model to perform driver state detection, and obtaining a first behavior event representing the driver state.

[0051] In this embodiment, by performing frame processing on the video data, the video data can be segmented into image frames one by one, and the image frames obtained by segmenting the video data are sorted in time sequence to obtain a corresponding video image frame sequence. In this embodiment, corresponding driver state detection models can be pre-trained for different types of behavioral events. Furthermore, according to the input requirements of each driver state detection model, feature extraction processing is performed on each frame of the video image frame sequence to obtain a feature image sequence that serves as input data for various driver state detection models. Exemplary driver state detection models include but are not limited to eye closure detection models, downcast detection models, yawn detection models, expression detection models, front collision warning models, lane departure models, etc. It is understandable that each driver state detection model corresponds to a feature image sequence. For example, the input data of the eye closure detection model can be a feature image sequence of the eye position of the face, and the input data of the yawn detection model can be a feature image sequence of the mouth position of the face, etc. In this embodiment, a feature image sequence is input into a driver state detection model. The driver state detection model performs pixel value operations on the input feature images to calculate a probability score indicating that each feature image may contain a corresponding driving behavior. The calculated probability scores are then compared with a preset score threshold one by one to determine whether the probability scores exceed the preset score threshold. If the probability scores corresponding to multiple consecutive frames (e.g., 10 consecutive frames) of feature images exceed the preset score threshold, the probability scores corresponding to the consecutive frames of feature images are sorted in time to generate a probability score sequence. The timestamp of the first frame of the consecutive frames of feature images is used as the start time of the event, and the timestamp of the last frame of the consecutive frames of feature images is used as the end time of the event. The probability score sequence, the start time, the end time, and the type of event detected by the driver state detection model are then packaged together to form a behavior event for output, thereby obtaining a first behavior event representing the driver state.

[0052] In some embodiments of the present application, for the feature images used for driver status analysis, the pixel values ​​in the feature images can be first regularized using statistical mean and standard deviation, so as to prevent data overfitting when the feature image sequence is input into the driver status detection model for driver status detection.

[0053] In some embodiments of the present application, to consider the impact of weather conditions on the analysis of driver driving risk, weather status information for the driving scenario may also be collected, such as rainy, snowy, foggy, or sunny days. After obtaining the weather status information, the weather status information is fused with time to obtain weather events. All obtained first and second behavioral events, along with the weather events, are then sorted to obtain a behavioral event sequence. This incorporates weather status information as an influencing factor into the driving risk analysis, thereby improving the accuracy of the driving risk analysis.

[0054] In some embodiments of the present application, to consider the impact of road conditions on driver risk analysis, road information within the driving scenario may also be collected, such as information about traffic lights, intersections, highways, corridors, and provincial roads. After obtaining the road information, the road information is fused with time to generate road condition events. All first and second behavioral events, along with the road condition events, are then sorted to generate a behavioral event sequence. This incorporates road information as an influencing factor into the driving risk analysis, improving the accuracy of the driving risk analysis.

[0055] In some embodiments of the present application, please refer to FIG4 , which is a flowchart of a method for determining adverse events in the driver's driving risk analysis method provided by an embodiment of the present application, which may specifically include steps S41 to S43 .

[0056] S41: numerically encoding the attribute data of each behavior event in the combined event in time sequence to generate a high-dimensional matrix for representing the combined event;

[0057] S42: Inputting the high-dimensional matrix into a preset driving risk calculation model for numerical calculation, and outputting a risk score corresponding to the combined event;

[0058] S43: Compare the risk score with a preset score threshold. If the risk score reaches the preset score threshold, determine the combined event as a poor quality event.

[0059] In this embodiment, a combination event can include multiple behavioral events, and each behavioral event has attribute data of multiple different information dimensions, such as the type of behavior, the probability score of the behavior, the time of occurrence, the duration, the vehicle's speed, the vehicle's direction, and other attribute data. In this embodiment, these attribute data can be numerically encoded in a time series to obtain a high-dimensional matrix used to represent the combination event. The high-dimensional matrix contains the attribute data of all behavioral events in the combination event. By inputting this high-dimensional matrix into a preset driving risk calculation model, the driving risk calculation model performs numerical operations on various different attributes to obtain a score value, which is output as the risk score corresponding to the combination event. It can be understood that the driving risk calculation model is a convolutional neural network model trained to a convergence state. The convolutional neural network model is trained to calculate the risk score of the combination event based on the high-dimensional matrix. The convolutional neural network model can be a CNN or VGG convolutional neural network model. By using a large amount of sample data (e.g., a combination of various behavioral events), the convolutional neural network model is trained to convergence, enabling the convolutional neural network model to calculate the risk score of the combination based on a high-dimensional matrix. In this embodiment, a score threshold is preset to measure adverse and benign events. After the driving risk calculation model calculates the risk score of the combination, it compares the risk score with the score threshold. If the risk score reaches the preset score threshold, the combination is judged as an adverse event; if the risk score reaches the preset score threshold, the combination is judged as a benign event. In this embodiment, the attribute data of each behavioral event in the combination is numerically encoded in a time series to generate a high-dimensional matrix. The high-dimensional matrix contains time series information and can represent information about the behavior itself, as well as the motivation for the behavior and the environmental information at the time of the behavior. This can increase the receptive field of the driver's driving risk analysis and improve the accuracy of the driver's driving risk analysis.

[0060] It can be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0061] In some embodiments of the present application, please refer to Figure 5, which is a system diagram of a driver's driving risk analysis system provided in an embodiment of the present application. As shown in Figure 5, the driver's driving risk analysis system includes a device side and a platform side, wherein the device side can be a vehicle-mounted device terminal, and the platform side can be a cloud server. Different detection models can be trained or different event confirmation algorithms can be configured on the device side for different types of events to confirm whether there are corresponding events of each type. Specifically, the vehicle-mounted device terminal can obtain facial video data from the vehicle's DMS camera, environmental video data from the vehicle's ADAS camera, vehicle driving data from the vehicle's six-axis sensor, and road condition data from the vehicle's GPS sensor. Furthermore, according to different data collection channels, different detection models are used to perform detection to obtain corresponding events, and then all obtained events are uploaded to the platform side for further driving risk analysis. After receiving the events uploaded by the device, the platform side sorts the events in chronological order to obtain a behavioral event sequence, constructs a combined event based on the behavioral event sequence, performs a driving risk analysis on the combined event, determines whether the combined event is a bad event, and counts the number of bad events. If the number of bad events reaches a preset threshold, it is determined that the driver's current driving state has a driving risk, and the driver's driving habits are evaluated. Through this embodiment, the detection algorithm and the analysis algorithm can be deployed separately on the device side and the platform side, which can reduce the hardware requirements of the algorithm deployment environment. The present application performs detection on the device side, which is actually an information extraction of video evidence, which can greatly reduce the traffic loss of uploading videos to the platform, protect user privacy, and facilitate the statistical analysis of these behaviors of the driver in the later stage. Moreover, on the platform side, these combined events that derive the driver's driving habits can be saved as a chain of evidence to facilitate traffic supervision.

[0062] In some specific embodiments, please refer to Figure 6, which is a basic structural block diagram of a driver driving risk analysis device provided in an embodiment of the present application. The various units included in the device in this embodiment are used to perform the various steps in the above-mentioned method embodiment. Please refer to the relevant description in the above-mentioned method embodiment for details. For ease of explanation, only the portions relevant to this embodiment are shown. As shown in Figure 6, the driver driving risk analysis device includes: a construction module 61, an analysis module 62, and a judgment module 63. Construction module 61 is used to collect facial video data, environmental video data, and vehicle travel data in a driving scene, and perform behavioral event analysis on the facial video data, environmental video data, and vehicle travel data to construct a combined event. Analysis module 62 is used to perform driving risk analysis on the combined event and determine whether the combined event is a negative event. Judgment module 63 is used to count the number of negative events. If the number of negative events reaches a preset threshold, it is determined that the driver's current driving state presents a driving risk.

[0063] It should be understood that the driver's driving risk analysis device of this embodiment corresponds one-to-one with the driver's driving risk analysis method applied in the first vehicle-mounted terminal, and will not be described in detail here.

[0064] In some embodiments of the present application, please refer to Figure 7, which is a basic structural block diagram of an electronic device provided in an embodiment of the present application. As shown in Figure 7, the electronic device 7 of this embodiment includes: a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71, such as the program of the driving risk analysis method for each driver mentioned above. When the processor 71 executes the computer program 73, the steps in each embodiment of the driving risk analysis method for each driver mentioned above are implemented. Alternatively, when the processor 71 executes the computer program 73, the functions of each module in the embodiment corresponding to the driving risk analysis device for the above-mentioned driver are implemented. Please refer to the relevant description in the embodiment for details, which will not be repeated here.

[0065] Exemplarily, the computer program 73 may be divided into one or more modules (units), each of which is stored in the memory 72 and executed by the processor 71 to implement the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 73 in the electronic device 7. For example, the computer program 73 may be divided into a construction module, an analysis module, and a judgment module, with the functions of each module being as described above.

[0066] The electronic device may include, but is not limited to, a processor 71 and a memory 72. Those skilled in the art will appreciate that FIG7 is merely an example of the electronic device 7 and does not limit the electronic device 7. The electronic device 7 may include more or fewer components than shown in the figure, or may combine certain components or different components. For example, the electronic device may further include input and output devices, network access devices, buses, and the like.

[0067] The processor 71 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0068] The memory 72 can be an internal storage unit of the electronic device 7, such as a hard drive or memory of the electronic device 7. The memory 72 can also be an external storage device of the electronic device 7, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 7. Furthermore, the memory 72 can include both an internal storage unit of the electronic device 7 and an external storage device. The memory 72 is used to store the computer program and other programs and data required by the electronic device. The memory 72 can also be used to temporarily store data that has been output or is about to be output.

[0069] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0070] The present application also provides a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the computer-readable storage medium can implement the steps of the above-mentioned method embodiments. In this embodiment, the computer-readable storage medium can be non-volatile or volatile.

[0071] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0072] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0073] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0074] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0075] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for analyzing a driver's driving risk, characterized in that: The method comprises: Collecting face video data, environmental video data, and vehicle driving data in a driving scene, and performing behavioral event analysis on the face video data, environmental video data, and vehicle driving data to construct a combined event; Performing a driving risk analysis on the combined event to determine whether the combined event is a negative event; The number of adverse events is counted, and if the number of adverse events reaches a preset number threshold, it is determined that the driver's current driving state has a driving risk.

2. The method for analyzing the driver's driving risk according to claim 1, characterized in that: The step of performing behavioral event analysis on the face video data, the environment video data and the vehicle driving data to construct a combined event includes: Performing driver status detection on the face video data and / or the environment video data to obtain a first behavior event representing the driver status, wherein the first behavior event includes an event type, a start time of the event, an end time of the event, and a probability score sequence of the event between the start time and the end time; Performing vehicle state detection on the vehicle driving data to obtain a second behavior event representing the vehicle state, a start time of the event, an end time of the event, and a driving parameter value sequence of the event between the start time and the end time; Sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence; According to the target behavior event, a combined event corresponding to the target behavior event is obtained from the behavior event sequence.

3. The method for analyzing the driver's driving risk according to claim 2, characterized in that: The step of performing driver status detection on the face video data and / or the environment video data to obtain a first behavior event representing the driver status includes: Performing frame processing on the face video data and / or the environment video data to obtain a video image frame sequence; Performing feature extraction processing on each frame of the video image in the video image frame sequence to obtain a feature image sequence; The characteristic image sequence is input into a preset driver state detection model to perform driver state detection, and a first behavior event representing the driver state is obtained.

4. The method for analyzing the driver's driving risk according to claim 3, characterized in that: Before the step of inputting the feature image sequence into a preset driver state detection model to detect the driver state and obtain a first behavior event representing the driver state, the method further includes: A pixel regularization process is performed on each frame of the feature image in the feature image sequence.

5. The method for analyzing the driver's driving risk according to claim 2, characterized in that: Before the step of sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence, the step further includes: Collect weather status information in driving scenarios, fuse the weather status information with time, and obtain weather events; The step of sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence includes: All the first behavior events, all the second behavior events, and the weather event are sorted together on the timeline to obtain a behavior event sequence.

6. The method for analyzing the driver's driving risk according to claim 2, characterized in that: Before the step of sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence, the step further includes: Collecting road information in a driving scenario, fusing the road information with time, and obtaining road condition events; The step of sorting all the first behavior events and all the second behavior events obtained on the time axis to obtain a behavior event sequence includes: All the first behavior events, all the second behavior events, and the road condition events obtained are sorted together on the time axis to obtain a behavior event sequence.

7. The method for analyzing a driver's driving risk according to any one of claims 1 to 6, characterized in that: The step of performing driving risk analysis on the combined event to determine whether the combined event is a bad event includes: Numerical encoding is performed on the attribute data of each behavior event in the combined event in time sequence to generate a high-dimensional matrix for representing the combined event; Inputting the high-dimensional matrix into a preset driving risk calculation model for numerical calculation, and outputting the risk score corresponding to the combined event; The risk score is compared with a preset score threshold, and if the risk score reaches the preset score threshold, the combined event is determined to be an adverse event.

8. A driver's driving risk analysis system, characterized in that: The driver's driving risk analysis system includes a device side and a platform side, wherein: The device side is used to collect face video data, environmental video data and vehicle driving data in driving scenarios, perform behavior event analysis on the face video data, environmental video data and vehicle driving data, generate behavior events, and upload the behavior events to the platform side; The platform end is used to sort the behavior events uploaded by the device end in time sequence, obtain a behavior event sequence, construct a combined event based on the behavior event sequence, perform driving risk analysis on the combined event, determine whether the combined event is a bad event, and count the number of bad events. If the number of bad events reaches a preset number threshold, it is determined that the driver's current driving state has a driving risk.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Identification alarm method and system for dangerous driving

    CN108230616A

  • Assisted driving early warning method and system

    CN108423006A

  • Driving behavior analysis method and device, equipment and storage medium

    CN110733509A

  • Driver state detection method, device and equipment and storage medium

    CN114834457A

  • Driver Risk Assessment System and Method Employing Automated Driver Log

    US20100238009A1

Cited By

  • Intelligent transverse and longitudinal prediction control system and method applied to electric forklift

    CN119590406A

  • User behavior analysis method, system and equipment and storage medium

    CN121051157A

  • Data processing method and device for VR interactive training of cross-country container forklift

    CN121640580A

  • Data processing method and device for VR training of cross-country container forklift

    CN121921835A

  • A data processing method and device for off-road container fork truck VR training

    CN121921835B