Automobile data recorder for AI intelligent analysis of driving behaviors

Abstract

The invention relates to the technical field of automobile data recorders, in particular to an AI intelligent driving behavior analysis automobile data recorder, which is used for acquiring the real-time vehicle condition in front of a driving vehicle and the driving behavior of a driver in the vehicle and acquiring the driving behavior data of the driver based on the acquisition of the driving behavior data of the driver. The feature extraction module is used for extracting hand actions and eye actions in the driving behaviors of the driver, the hand actions are used as judgment criteria for standardizing the driving behaviors, and the eye actions are used as compensation basis for judging the concentration of the driver; the driving behavior is scored and predicted through a driving behavior scoring model pre-constructed by a risk prediction module, a scoring threshold value of dangerous driving and a scoring threshold value of distracted driving are set, risk-based judgment is carried out according to the score of the driving behavior, and warning is carried out through a warning module according to the judged driving behavior risk-based judgment. Therefore, the driver is reminded to correct the driving behavior in time, and traffic accidents are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of driving recorders, and more specifically, it relates to a driving recorder with AI intelligent analysis of driving behavior. Background Art

[0002] A driving recorder can record video images and sounds during vehicle driving, which is equivalent to the black box of the vehicle and can provide evidence for traffic accidents. Generally, in order to make the captured images clearer, a driving recorder mostly adopts a high-pixel camera and improves the resolution of the image sensing chip.

[0003] A large number of statistical analysis results show that more than 70% of road traffic accidents are directly related to the driver and their driving behavior, because these driving accidents are caused by the driver's perception, judgment or operation errors during driving.

[0004] Moreover, with the increase in the number of motor vehicles, safe driving has become an important issue affecting social stability factors. Among them, the driver's standardized driving behavior is crucial for ensuring safe driving. Analyzing the driver's driving behavior is beneficial to reducing the probability of risks during driving and ensuring the property and life safety of the driver.

[0005] At present, most driving recorders are only used for driving record, which can only trace back after a traffic accident, facilitating the division of responsibilities in the traffic accident and the analysis of the causes of the event. However, it cannot assist the driver in safe driving during driving to prevent or reduce the occurrence of traffic safety accidents. Summary of the Invention

[0006] Aiming at the deficiencies of the above-mentioned existing technologies, the purpose of the present invention is to provide a driving recorder with AI intelligent analysis of driving behavior, which has the advantages of reminding the driver to correct driving behavior and avoiding the occurrence of traffic accidents in advance.

[0007] The above technical purpose of the present invention is achieved through the following technical solutions: A driving recorder with AI intelligent analysis of driving behavior, comprising:

[0008] A data acquisition module for collecting real-time vehicle conditions in front of the vehicle and the driving behavior of the driver in the vehicle;

[0009] A feature extraction module, based on the recorder recording the driver's driving behavior, extracts the hand movements and eye activities in the driver's driving behavior, uses the hand movements as the evaluation criterion for standard driving behavior, and uses the eye activities as the compensation basis for judging the driver's concentration;

[0010] A risk prediction module, taking the driver's hand movements and eye activities as weights, constructs a scoring model for driving behavior: S = W1*H + W2*(1 - E);

[0011] Where S is the final score, H is the score of hand movements, E is the score of eye movements, W1 and W2 are the weights of hand movements and eye movements, and W1 + W2 = 1;

[0012] According to the driving behavior score, by setting the score thresholds for dangerous driving and distracted driving, the driving behavior of the driver is predicted for risks;

[0013] A warning module issues different warning signals according to the risk level predicted by the risk prediction module to warn the driver to correct the driving behavior.

[0014] Preferably, the data acquisition module includes a recorder carrier body, a front camera module installed on one side of the recorder carrier body, and a rear camera module installed on the other side of the recorder carrier body. The front camera module records the real-time situation in front of the vehicle, and the rear camera module records the driving behavior of the driver.

[0015] Preferably, the feature extraction module includes synchronous monitoring of hand movements and eye movements in the driver's driving behavior, frame-by-frame conversion of the video taken by the rear camera module, saving the converted pictures, and analyzing hand movements and eye movements based on the saved pictures by setting the interval threshold for saving pictures.

[0016] Preferably, the hand movements include driving with both hands holding the steering wheel, driving with one hand holding the steering wheel, and driving with hands off the steering wheel; the eye movement analysis includes analysis of eye opening and closing degree and analysis of the direction of eye concentration;

[0017] Based on the hand movement of driving with both hands holding the steering wheel, by correspondingly identifying and analyzing the eye opening and closing degree of the driver's eyes in the same period, and then using the risk prediction module to calculate the driving risk of dangerous driving;

[0018] Based on the hand movement of driving with one hand holding the steering wheel, by correspondingly identifying and analyzing the direction of eye concentration of the driver's eyes in the same period, and using the risk prediction module to calculate the driving risk of distracted driving;

[0019] Based on the hand movement of driving with hands off the steering wheel, directly issue a driving warning for dangerous driving through the warning module.

[0020] Preferably, the warning module includes an information warning light or a voice warning device, and the information warning light or the voice warning device is installed on the recorder;

[0021] Based on the risk prediction of dangerous driving and distracted driving by the risk prediction module, use different signal colors of the information warning light to give warning reminders;

[0022] Based on the risk prediction module's prediction of risky driving such as dangerous driving and distracted driving, a voice alarm is used to give warning reminders with alarm sounds of different frequencies.

[0023] Preferably, the warning module includes a mobile terminal electrically connected to the recorder. The mobile terminal includes a mobile phone, a car machine, and a smartwatch, and the driver is warned by triggering the vibration and prompt sound of the mobile terminal.

[0024] Preferably, a deep learning network model is pre-constructed in the feature extraction module. According to the pre-collected eye movement training samples and hand movement training samples, the deep learning network model is trained to make the deep learning network model reach a preset convergence condition, which is used to assist the feature extraction module in extracting features of hand movements and eye movements.

[0025] In summary, the beneficial effects of the present invention are as follows: The real-time driving conditions in front of the vehicle and the driving behavior of the driver in the vehicle are collected by the recorder. Based on the collection of the driver's driving behavior data, the feature extraction module is used to extract the hand movements and eye movements in the driver's driving behavior. The hand movements are used as the judgment criterion for standard driving behavior, and the eye movements are used as the compensation basis for judging the driver's concentration. Furthermore, the driving behavior is scored and predicted by the scoring model of the driving behavior pre-constructed by the risk prediction module. By setting the scoring thresholds for dangerous driving and distracted driving, and then making a risk determination according to the size of the score of the driving behavior, a warning is given through the warning module according to the determined driving behavior risk, so as to remind the driver to correct the driving behavior and avoid traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic diagram of the electrical signal connection of an embodiment of the present invention.

[0027] Reference numerals: 1, data acquisition module; 11, feature extraction module; 12, risk prediction module; 13, warning module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0029] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When a component is referred to as being "connected to" another component, it can be directly or indirectly connected to the other component.

[0030] It should be understood that the orientation or positional relationship indicated by terms such as "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on the present invention.

[0031] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0032] A driving recorder for AI intelligent analysis of driving behavior, see Figure 1 , including:

[0033] A data acquisition module 1 for acquiring the real-time vehicle conditions in front of the vehicle and the driving behavior of the driver in the vehicle;

[0034] A feature extraction module 11, based on the recorder recording the driving behavior of the driver, extracts the hand movements and eye activities in the driver's driving behavior, uses the hand movements as the evaluation criteria for standard driving behavior, and uses the eye activities as the compensation basis for judging the driver's concentration;

[0035] A risk prediction module 12, taking the hand movements and eye activities of the driver as weights, constructs a scoring model for driving behavior: S = W1*H + W2*(1 - E);

[0036] Where S is the final score, H is the score of hand movements, E is the score of insufficient concentration of eye activities, W1 and W2 are the weights of hand movements and eye activities, and W1 + W2 = 1;

[0037] According to the size of the driving behavior score, by setting the score thresholds for dangerous driving and distracted driving, the driving behavior of the driver is predicted for risks;

[0038] A warning module 13 issues different warning signals according to the size of the risks predicted by the risk prediction module 12 to warn the driver to correct the driving behavior.

[0039] In this embodiment, a recorder is used to collect the real-time driving conditions in front of the vehicle and the driving behavior of the driver in the vehicle. Based on the collection of the driver's driving behavior data, a feature extraction module 11 extracts the hand movements and eye activities in the driver's driving behavior. The hand movements are used as the criterion for judging standard driving behavior, and the eye activities are used as the compensation basis for judging the driver's concentration. Furthermore, a scoring model of driving behavior pre-constructed by a risk prediction module 12 is used to score and predict the driving behavior. By setting the scoring thresholds for dangerous driving and distracted driving, and then making a risk determination according to the size of the score of the driving behavior, a warning is given through a warning module 13 based on the determined driving behavior risk, so as to remind the driver to correct the driving behavior in time and avoid traffic accidents.

[0040] A deep learning network model is pre-constructed in the feature extraction module 11. According to the pre-collected eye activity training samples, hand movement training samples, and training the deep learning network model, the deep learning network model is made to reach the preset convergence condition to assist the feature extraction module 11 in extracting features of hand movements and eye activities.

[0041] The eye activity training samples and hand movement training samples are first recorded and collected through a recorder to form a training sample set. The training sample set includes collecting the normal driving videos of the driver. The resolution of the collected videos is 1920×1080, and the frame rate is 30fps. By converting the normal driving videos into pictures frame by frame and screening the pictures, deleting the unclear or blurred and uncertain pictures, and finally constructing the training sample set.

[0042] This deep learning network model uses a convolutional neural network to process image-like inputs, such as images of hand movements, and a recurrent neural network or a time series model to process dynamic time series data, such as time series data of eye activities.

[0043] The model architecture design includes:

[0044] I. Input layer:

[0045] Hand movement input: The hand movement data is an image sequence, and operations such as size normalization, grayscale conversion, or color enhancement can be performed using an image preprocessing module.

[0046] Eye activity input: The eye activity is a video sequence, which is converted into a format suitable for time series modeling.

[0047] II. Hand movement feature extraction module 11:

[0048] Convolutional Layers: Extract the spatial features of hand movements through multiple convolutional layers, and use small convolutional kernels to capture details.

[0049] Pooling Layers: Used to reduce the dimension of the feature map, reduce the computational amount, and retain key information at the same time.

[0050] Activation function: The ReLU activation function can help the network learn non-linear features.

[0051] III. Eye movement feature extraction module 11:

[0052] LSTM cell (Long Short-Term Memory): The LSTM cell can handle the time dependence in eye movement data and learn the long-term dependencies in the eye movement sequence. Multiple layers of LSTM can be used for more complex time series modeling.

[0053] Dropout layer: Avoid overfitting and improve the generalization ability of the model.

[0054] IV. Fusion layer (multi-modal feature fusion):

[0055] Fuse the features of the eye movement module and the hand movement module. The weighted fusion method is used to combine the features of the two. After fusion, a comprehensive feature representation containing multi-modal information can be obtained.

[0056] V. Fully Connected Layer:

[0057] Send the fused features into the fully connected layer (Dense Layer) for further high-dimensional feature transformation.

[0058] VI. Output layer:

[0059] According to the task requirements, the design of the output layer can be different. For classification tasks, the softmax activation function is used; for regression tasks, the linear activation function is used. The size and type of the output layer will vary according to the results expected after feature extraction.

[0060] VII. Loss function:

[0061] Cross-Entropy Loss: Suitable for classification tasks.

[0062] Mean Squared Error (MSE): Suitable for regression tasks.

[0063] Model training based on the model architecture:

[0064] I. Data preparation:

[0065] Both eye movement training samples and hand movement training samples are collected by a recorder.

[0066] The data needs to be preprocessed, such as normalization, data augmentation, etc., to enhance the generalization ability of the model.

[0067] II. Training process:

[0068] Use the training set to train the model and optimize the parameters in the network through the backpropagation algorithm.

[0069] Adopt an appropriate optimizer (such as Adam, SGD, etc.) for training and set an appropriate learning rate.

[0070] Monitor the loss and accuracy during the training process to prevent overfitting. Cross-validation can be performed using the validation set.

[0071] III. Convergence condition:

[0072] By monitoring the training and validation losses, set a threshold or the maximum number of iterations to ensure that the model reaches the preset convergence condition.

[0073] The sign of convergence is usually that the loss value no longer decreases significantly, or the training accuracy and validation accuracy tend to be stable.

[0074] Based on the collected training sample set, which contains a large number of hand movement scores, eye movement scores of drivers, and the corresponding accident risk levels. Calculate the weights through the following steps:

[0075] Data collection: Collect a large amount of driving behavior data, including hand movement scores H, eye movement inattentiveness scores E, and real risk scores.

[0076] Build a regression model: We use a simple linear regression model, set the final score S as the target variable, and H and E as input features:

[0077] S = W1 * H + W2 * (1 - E)

[0078] Through regression analysis, we can obtain the optimal values of W1 and W2 to make the score S predicted by the model best match the actual risk score.

[0079] Optimize the weights: Use optimization algorithms such as the least squares method of regression analysis to obtain the optimal W1 and W2.

[0080] For example, the analysis result of the weights is: W1 = 0.65; W2 = 0.35.

[0081] It can be seen that the score of hand movement contributes more to the final score.

[0082] Based on the known weights, the hand movements and eye activities are recognized in real time and scored accordingly, including the hand movement score H and the lack of eye activity concentration score E.

[0083] Among them, the hand movement score H: Evaluates the standard degree of driving behavior according to the driver's hand movements, and the scoring range is from 0 (non-standard) to 1 (fully standard).

[0084] The lack of eye activity concentration score E: Evaluates the concentration according to the driver's eye activities, and the scoring range is from 0 (fully concentrated) to 1 (highly distracted). It should be noted that the smaller the value of E, the more concentrated the driver is, and the larger the value, the more easily the driver's attention is dispersed.

[0085] The hand movement weight W1: The weight of hand movements, with a value between 0 and 1.

[0086] The eye activity weight W2: The weight of eye activities, with a value range opposite to that of the hand movement weight. When setting the weights, W1 + W2 = 1, that is, the sum of these two weights is 1.

[0087] Risk level: Judging the risk level of the driver's driving behavior according to the final score. The following rules can be set according to the preset threshold:

[0088] Normal driving: S > 0.7; Distracted driving: 0.6 ≤ S ≤ 0.7; Dangerous driving: S < 0.6.

[0089] Based on the above known conditions, the risk data display of driving behavior prediction is as follows:

[0090]

[0091]

[0092] Among them, the data acquisition module 1 includes a recorder carrier body, a front camera module installed on one side of the recorder carrier body, and a rear camera module installed on the other side of the recorder carrier body. The front camera module records the real-time situation in front of the vehicle, and the rear camera module records the driver's driving behavior.

[0093] On the basis of pre-constructing a deep learning network model, the feature extraction module 11 extracts features from the collected driver's driving behavior. Feature extraction includes synchronous monitoring of the hand movements and eye activities in the driver's driving behavior. By converting the video taken by the rear camera module frame by frame, the converted pictures are saved. By setting the interval threshold for saving pictures, hand movement analysis and eye activity analysis are performed based on the saved pictures.

[0094] The analysis of the hand movements includes driving with both hands holding the steering wheel, driving with one hand holding the steering wheel, and driving with hands off the steering wheel.

[0095] The analysis of the eye activities includes the analysis of the degree of eye opening and closing and the analysis of the direction of eye fixation.

[0096] Under various driving actions based on the analysis of hand movements, the corresponding activities of eye movements are analyzed.

[0097] Among them, under the hand movement of driving with both hands holding the steering wheel, by correspondingly identifying and analyzing the degree of eye opening and closing of the driver's eyes in the same period, and then using the risk prediction module 12 to calculate the driving risk of dangerous driving.

[0098] Since when driving with both hands holding the steering wheel, the driver's driving attention is concentrated, but it is possible that due to excessive concentration of attention, the driver may experience driving fatigue. Therefore, by correspondingly identifying and analyzing the degree of eye opening and closing of the driver's eyes in the same period, the risk prediction module 12 is used to calculate the driving risk of dangerous driving.

[0099] Under the hand movement of driving with one hand holding the steering wheel, at this time, the driver may be eating, swiping the phone, holding something, etc. It can be seen that the driver has been distracted. Therefore, by correspondingly identifying and analyzing the direction of eye fixation of the driver's eyes in the same period, the risk prediction module 12 calculates the driving risk of distracted driving.

[0100] Under the hand movement of driving with hands off the steering wheel, the driving warning of dangerous driving can be directly given through the warning module 13.

[0101] After predicting the driving risk, it is necessary to further warn the driver. Therefore, by setting the warning module 13, where the warning module 13 includes an information warning light or a voice warning device, and the information warning light or the voice warning device is installed on the recorder;

[0102] Based on the risk prediction of dangerous driving and distracted driving by the risk prediction module 12, different signal colors are used for warning and reminding through the information warning light;

[0103] Based on the risk prediction of dangerous driving and distracted driving by the risk prediction module 12, different frequencies of alarm sounds are used for warning and reminding through the voice warning device.

[0104] Or the warning module 13 is set as a mobile terminal electrically connected to the recorder. The mobile terminal includes a mobile phone, a car machine, and a smartwatch, and the driver is warned by triggering the vibration and prompt sound of the mobile terminal.

[0105] The above embodiments are merely explanations of the present invention and are not limitations thereof. After reading this specification, those skilled in the art may make modifications to these embodiments without creative contributions as needed, but as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A driving recorder with AI intelligent analysis of driving behavior, characterized by: include: Data collection module, used to collect real-time vehicle conditions ahead and driving behavior of the driver in the car; A feature extraction module, based on the driving behavior of the driver recorded by the recorder, extracts the hand movements and eye activities of the driver in the driving behavior, uses the hand movements as a criterion for judging the driving behavior, and uses the eye activities as a compensation basis for judging the driver's concentration; The risk prediction module uses the driver’s hand movements and eye activities as weights to construct a scoring model for driving behavior: S = W1*H+W2*(1-E); Where S is the final score, H is the score of hand movements, E is the score of insufficient eye concentration, W1 and W2 are the weights of hand movements and eye movements, and W1+W2=1; According to the driving behavior score, the driver's driving behavior risk is predicted by setting the scoring threshold of dangerous driving and the scoring threshold of distracted driving; The warning module sends out different warning signals according to the risk size predicted by the risk prediction module to warn the driver to correct his driving behavior.

2. The driving recorder with AI intelligent analysis of driving behavior according to claim 1 is characterized by: The data acquisition module includes a recorder carrying body, a front camera module installed on one side of the recorder carrying body, and a rear camera module installed on the other side of the recorder carrying body. The front camera module records the actual situation in front of the vehicle in real time, and the rear camera module records and shoots the driver's driving behavior.

3. The driving recorder with AI intelligent analysis of driving behavior according to claim 2 is characterized by: The feature extraction module includes synchronous monitoring of the driver's hand movements and eye activities during driving behavior, converting the video captured by the rear camera module frame by frame, saving the converted pictures, setting an interval threshold for saving pictures, and performing hand movement analysis and eye activity analysis based on the saved pictures.

4. The driving recorder with AI intelligent analysis of driving behavior according to claim 3 is characterized by: The hand movements include two-handed driving, one-handed driving and hands-free driving; the eye activity analysis includes eye opening degree analysis and eye focus direction analysis; Based on the hand movements of driving with both hands, the driver's eye opening degree is analyzed through corresponding recognition, and then the risk prediction module is used to calculate the driving risk of dangerous driving; Based on the hand movements of driving with one hand, the driver's eye focus direction is analyzed through corresponding recognition and analysis during the same period, and the risk prediction module is used to calculate the driving risk of distracted driving; Based on the hand movements in the hands-free driving direction, the warning module directly issues a driving warning for dangerous driving.

5. The driving recorder with AI intelligent analysis of driving behavior according to claim 1 is characterized by: The warning module includes an information warning light or a voice warning device, and the information warning light or the voice warning device is installed on the recorder; Based on the risk prediction module's prediction of dangerous driving and distracted driving, the information warning light uses different signal light colors to give warnings; Based on the risk prediction module's prediction of dangerous driving and distracted driving, the voice alarm uses alarm sounds of different frequencies to give warnings.

6. The driving recorder with AI intelligent analysis of driving behavior according to claim 1 is characterized by: The warning module includes a mobile terminal electrically connected to the recorder, and the mobile terminal includes a mobile phone, a car computer and a phone watch, and the driver is warned by triggering vibration and prompt sound of the mobile terminal.

7. The driving recorder with AI intelligent analysis of driving behavior according to claim 1 is characterized by: A deep learning network model is pre-constructed in the feature extraction module. The deep learning network model is trained based on pre-collected eye activity training samples and hand movement training samples so that the deep learning network model reaches a preset convergence condition to assist the feature extraction module in extracting features of hand movements and eye activities.

Images