Intelligent driving monitoring method and system based on automobile data recorder

Abstract

The invention discloses an intelligent driving monitoring method and system based on an automobile data recorder, and relates to the technical field of automobile data recorders. The system continuously collects video data inside and outside an automobile through a double-lens Tandem; constructing a deep learning model through a convolutional neural network CNN, a recurrent neural network RNN and a long-short term memory network LSTM, and extracting key features from the driver face image and the vehicle surrounding environment; the data analysis module processes the feature vectors by using a machine learning and statistical method, calculates a vehicle driving coefficient Clx, a traffic density coefficient Jtm and a driver fatigue coefficient Jsp, and comprehensively generates a driving early warning index ZHY; and the evaluation and early warning module performs preliminary evaluation on the fatigue state of the driver based on a preset threshold value, performs secondary evaluation according to the comprehensive driving early warning index, provides real-time safety prompts and suggestions, and effectively improves the driving safety and the driving efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of driving recorders, and in particular to an intelligent driving monitoring method and system based on a driving recorder. Background Art

[0002] In the technical field of driving recorders, most current systems mainly focus on video recording and post-event playback functions, lacking real-time analysis and warning capabilities. The limitation of such systems is that they cannot predict and prevent potential safety hazards in advance. Especially in complex traffic environments, drivers may not be able to react in time due to fatigue, sudden road condition changes, or traffic congestion, resulting in an increased safety risk. Therefore, a more intelligent system is needed to monitor and analyze the driving environment and driver status in real time and provide timely warning information.

[0003] The deficiencies of existing driving recorder technologies are mainly reflected in the lack of real-time data processing and comprehensive analysis capabilities. When a driver is fatigued or encounters complex road conditions, traditional recorders can only record the process of events occurring, and it is not easy to take effective preventive measures and real-time warnings; in the event of an emergency, such a system may cause the driver to be unable to adjust driving behavior in time, thus leading to traffic accidents. Summary of the Invention

[0004] In view of the deficiencies of the prior art, the present invention provides an intelligent driving monitoring method and system based on a driving recorder, which solves the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: An intelligent driving monitoring method and system based on a driving recorder, including a data acquisition module, a feature extraction module, a data analysis module, a comprehensive analysis module, and an evaluation and warning module;

[0006] The data acquisition module is used to continuously capture video data of the driver inside the vehicle and the driving outside the vehicle through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data;

[0007] The feature extraction module is constructed by a deep learning model based on convolutional neural network CNN, recurrent neural network RNN, and long short-term memory network LSTM as the basic framework, to identify relevant features in the collected video data and perform feature extraction;

[0008] The data analysis module is used to process the feature vectors output by the feature extraction module, perform multi-level data parsing and correlation analysis, and use statistical and machine learning models to perform normalization processing and comprehensive calculation on the extracted data set, respectively obtaining the vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp;

[0009] The comprehensive analysis module is used to perform associated calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain a comprehensive driving warning index ZHY;

[0010] The evaluation and warning module is used to preliminarily compare and evaluate a preset driver fatigue threshold A with the obtained driver fatigue coefficient Jsp, and generate a warning message according to the evaluation result to activate a second evaluation mechanism. The second evaluation mechanism is used to perform a secondary comparison and evaluation on a preset first driving safety threshold M and a preset second driving safety threshold N with the obtained comprehensive driving warning index ZHY, and generate a warning message according to the evaluation result.

[0011] Preferably, the data acquisition module includes a video acquisition unit and a video preprocessing unit;

[0012] The video acquisition unit is used to install a dual-lens driving recorder Tandem inside the vehicle to collect real-time facial video data of the driver and video data of vehicles, pedestrians, and the road surface during driving outside the vehicle;

[0013] The video preprocessing unit is used to preprocess the collected video data of vehicles, pedestrians, and the road surface. The preprocessing methods include video denoising, image enhancement, and target detection preprocessing.

[0014] Preferably, the feature extraction module includes a feature recognition unit and a feature extraction unit;

[0015] The feature recognition unit includes a data annotation unit and a deep learning unit;

[0016] The data annotation unit is used to collect a large amount of historical video data of drivers inside the vehicle, vehicles driving outside the vehicle, and pedestrians, import it into the CVAT annotation tool, and through the automatic frame segmentation and annotation functions of the CVAT annotation tool, split the video data into individual frame pictures, and annotate the frame pictures with drivers inside the vehicle and outside the vehicle;

[0017] By annotating the facial features of the frame pictures with drivers inside the vehicle, the facial features include eye closure features, eye blinking features, and mouth opening and closing features;

[0018] By annotating the vehicle driving features and pedestrian movement features of the frame pictures outside the vehicle, the vehicle driving features include vehicle speed features, vehicle distance features, vehicle deviation features, road width features, and lane number features, and the pedestrian movement features include pedestrian flow features; the deep learning unit includes a model construction unit and a model training unit;

[0019] The model construction unit is used to construct a driver fatigue state recognition model and a vehicle external situation recognition model by using a convolutional neural network (CNN), a recurrent neural network (RNN), and a long short-term memory network (LSTM) as the model framework; extract driver face features from low level to high level from the driver's face image through the CNN, process the face image sequence data through the RNN to capture the driver's time-related features, and further improve the recognition ability of the time-related features based on the memory unit of the LSTM to perform feature recognition on the obtained driver's face image; at the same time, perform object detection and semantic segmentation in the real-time object detection system SSD and the semantic segmentation model DeepLab to construct a vehicle external situation recognition model, and then integrate the driver fatigue state recognition model and the vehicle external situation recognition model to obtain a deep learning model; perform object detection of vehicles and pedestrians through the real-time object detection system SSD, which can quickly identify and locate vehicles and pedestrians outside the vehicle in the real-time video stream, and use the semantic segmentation model DeepLab to perform semantic segmentation of the external vehicle scene to accurately segment roads, lane lines, pedestrians, and external traffic elements;

[0020] The model training unit is used to divide the labeled historical in-vehicle driver and out-of-vehicle driving video data into a training set and a validation set, import the training set into the constructed deep learning model, perform iterative training on the deep learning model, and then verify the accuracy of the deep learning model in recognizing the driver's fatigue characteristics and out-of-vehicle driving characteristics through the validation set, and continuously perform iterative learning to optimize the deep learning model;

[0021] The feature extraction unit is used to import the real-time collected face picture set and out-of-vehicle picture set into the deep learning model for feature extraction, and real-time extract a fatigue driving data set, a traffic environment data set, and a vehicle driving data set;

[0022] The fatigue driving data set includes the eye closure degree yb, the mouth opening and closing frequency zz, the eye closure time ys, and the eye blinking frequency yz;

[0023] The traffic environment data set includes the vehicle flow cl, the pedestrian flow xl, the road width dk, and the number of lanes cs;

[0024] The vehicle driving data set includes the vehicle acceleration js, the vehicle speed sd, the vehicle deviation value pl, and the distance to the vehicle ahead qj.

[0025] Preferably, the data analysis module includes a vehicle behavior analysis unit, a road condition analysis unit, and a driver behavior analysis unit;

[0026] The vehicle behavior analysis unit is used to calculate and obtain the vehicle driving coefficient Clx by summarizing after normalization based on the extracted vehicle driving data set;

[0027] The vehicle driving coefficient Clx is obtained through the following formula;

[0028] ;

[0029] In the formula, ln represents the logarithmic function, and e represents the exponential function. represents the natural logarithm of the distance qj to the vehicle in front, represents the negative exponent of the vehicle speed sd.

[0030] Preferably, the road condition analysis unit is used to calculate and obtain the traffic density coefficient Jtm by summarizing after normalization based on the extracted traffic environment data set;

[0031] The traffic density coefficient Jtm is obtained through the following formula;

[0032] ;

[0033] In the formula, ln represents the logarithmic function, and ln(cl + 1) represents the logarithmic term of the vehicle flow cl.

[0034] Preferably, the driver behavior analysis unit is used to calculate and obtain the driver fatigue coefficient Jsp by summarizing after normalization based on the extracted driver fatigue data set;

[0035] The driver fatigue coefficient Jsp is obtained through the following formula;

[0036] ;

[0037] In the formula, ln represents the logarithmic function, and e represents the exponential function. represents the maximum value of the eye closing time. The higher the closing degree, the faster the fatigue coefficient increases, but it gradually slows down with ln(1 + yb); Combined with the mouth opening and closing frequency zz and the logistic function, it smoothly reflects the influence of frequency on fatigue. The higher the frequency, the faster the fatigue coefficient increases, and the increase speed is controlled by the square root; The influence of the eye blinking frequency yz is smoothed through the logistic function. The higher the blinking frequency, the increase of the fatigue coefficient, and the influence gradually slows down when the frequency is very high.

[0038] Preferably, the comprehensive analysis module is used to perform normalization processing on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp, and then calculate and obtain the comprehensive driving warning index ZHY by summarizing;

[0039] The comprehensive driving warning index ZHY is calculated through the following formula;

[0040] ;

[0041] In the formula, ln represents the logarithmic function, e represents the exponential function, and sin represents the sine function. represents the logarithmic term of the vehicle driving coefficient Clx, which is used to smoothly process the change of the vehicle driving coefficient Clx. represents the exponential term of the traffic density coefficient Jtm. represents the negative exponential term of the vehicle driving coefficient Clx. reflects the periodic interaction between driver fatigue and traffic density, and Z represents a correction constant.

[0042] Preferably, the evaluation and warning module includes a first evaluation unit and a second evaluation unit;

[0043] The first evaluation unit presets a vehicle behavior threshold A based on the historical safe driving speed and deviation interval of vehicle driving, and then conducts a preliminary evaluation with the obtained driver fatigue coefficient Jsp to analyze the driving process information of the current vehicle during driving. The specific evaluation scheme is as follows;

[0044] Evaluate the result obtained by comparing the driver fatigue coefficient Jsp with the preset driver fatigue threshold A to obtain a first prediction result;

[0045] When the driver fatigue coefficient Jsp ≤ the preset driver fatigue threshold A, it means that the driver is in a normal driving state;

[0046] When the driver fatigue coefficient Jsp > the preset driver fatigue threshold A, it means that the driver is in a fatigued driving state. At this time, the driving recorder reminds the driver every 5 minutes by voice: You are driving fatigued. Please take a break.

[0047] Preferably, the second evaluation unit presets a first driving safety threshold M and a second driving safety threshold N based on the comprehensive safety value of vehicle driving, and then conducts a secondary evaluation with the obtained comprehensive driving warning index ZHY to analyze the road conditions and the driver's driving conditions of the current vehicle during driving. The specific evaluation scheme is as follows;

[0048] When the comprehensive driving warning index ZHY ≤ the preset first driving safety threshold M, it means that the vehicle is driving safely and the driver's behavior and road conditions are normal;

[0049] When the preset first driving safety threshold M < comprehensive driving warning index ZHY < preset second driving safety threshold N, it indicates that the driving state of the vehicle is abnormal. At this time, the driving recorder uses voice to remind the driver that the road ahead is congested, and please pay attention to reducing the speed;

[0050] When the comprehensive driving warning index ZHY ≥ preset second driving safety threshold N, it indicates that the vehicle is in a dangerous driving state. At this time, the driving recorder uses voice to remind the driver that the road ahead is congested, please immediately reduce the speed, turn on the vehicle's emergency double flash indicator, and control the driving recorder to increase the data collection frequency by 90%.

[0051] An intelligent driving monitoring method based on a driving recorder includes the following steps:

[0052] S1. Continuously capture video data of the driver inside the vehicle and the driving outside the vehicle through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data;

[0053] S2. Based on the convolutional neural network CNN, recurrent neural network RNN, and long short-term memory network LSTM as the basic framework of the deep learning model, construct a deep learning model to identify relevant features in the collected video data and perform feature extraction;

[0054] S3. Process the output feature vectors, perform multi-level data analysis and correlation analysis, use statistical and machine learning models to normalize and comprehensively calculate the preprocessed data set, and respectively obtain the vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp;

[0055] S4. Perform correlation calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain the comprehensive driving warning index ZHY;

[0056] S5. Preset the driver fatigue threshold A and perform a preliminary comparison and evaluation with the obtained driver fatigue coefficient Jsp, and generate a warning message according to the evaluation result to activate the second evaluation mechanism. The second evaluation mechanism is used to preset the first driving safety threshold M and the preset second driving safety threshold N and the obtained comprehensive driving warning index ZHY for a secondary comparison and evaluation, and generate a warning message according to the evaluation result.

[0057] The present invention provides an intelligent driving monitoring method and system based on a driving recorder. It has the following beneficial effects:

[0058] (1) A driving intelligent monitoring method and system based on a driving recorder, by integrating a variety of advanced technologies and modules, realizes comprehensive monitoring and real-time warning of the vehicle driving state; the system uses a dual-lens driving recorder in the data acquisition module to capture video data inside and outside the vehicle, and constructs a deep learning model through the feature extraction module to identify and extract relevant features; the data analysis module calculates the vehicle driving coefficient Clx, traffic density coefficient Jtm and driver fatigue coefficient Jsp respectively based on the extracted features, and further integrates these coefficients through the comprehensive analysis module to calculate the comprehensive driving warning index ZHY. This system can monitor and analyze the driver's state and driving environment in real time, and provide effective safety warnings.

[0059] (2) A driving intelligent monitoring method and system based on a driving recorder, through a deep learning model, the system can efficiently extract complex features from video data and use these features to accurately analyze driving behavior, road conditions and driver fatigue status; the combination of the data analysis module and the comprehensive analysis module enables the system to comprehensively evaluate all aspects of vehicle driving, and generate warning information according to the evaluation results to timely remind the driver to take necessary safety measures; this multi-level and multi-dimensional analysis method significantly improves the monitoring accuracy and warning ability of the system, and effectively reduces traffic accidents caused by driver fatigue or poor road conditions.

[0060] (3) A driving intelligent monitoring method and system based on a driving recorder, by setting a dual evaluation mechanism of the driver fatigue threshold and the comprehensive driving warning index, the system further enhances the reliability of safety monitoring; the first evaluation unit can timely remind the driver to rest by the preliminary evaluation of the driver fatigue coefficient Jsp to prevent fatigue driving; while the second evaluation unit further evaluates the vehicle driving state according to the comprehensive driving warning index ZHY to provide warning information of road congestion and dangerous states, and guide the driver to make safe driving decisions; this multi-level evaluation and warning mechanism can effectively reduce the risk of traffic accidents, improve driving safety, and ensure the safety of drivers and road users. Brief Description of the Drawings

[0061] Figure 1 It is a schematic flow chart of a driving intelligent monitoring system based on a driving recorder of the present invention;

[0062] Figure 2 It is a schematic diagram of the steps of a driving intelligent monitoring method based on a driving recorder of the present invention. Detailed Embodiments

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0064] Embodiment 1

[0065] Please refer to Figure 1 , the present invention provides a driving intelligent monitoring method and system based on a driving recorder. To achieve the above objectives, the present invention is implemented through the following technical solutions: including a data acquisition module, a feature extraction module, a data analysis module, a comprehensive analysis module, and an evaluation and warning module;

[0066] The data acquisition module is used to continuously capture the video data of the driver inside the vehicle and the driving outside the vehicle through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data;

[0067] The feature extraction module is constructed by a deep learning model based on the convolutional neural network CNN, the recurrent neural network RNN, and the long short-term memory network LSTM to identify the relevant features in the collected video data and perform feature extraction;

[0068] The data analysis module is used to process the feature vectors output by the feature extraction module, perform multi-level data parsing and correlation analysis, and use statistical and machine learning models to perform normalization processing and comprehensive calculation on the extracted data set to respectively obtain the vehicle driving coefficient Clx, the traffic density coefficient Jtm, and the driver fatigue coefficient Jsp;

[0069] The comprehensive analysis module is used to perform correlation calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain the comprehensive driving warning index ZHY;

[0070] The evaluation and warning module is used to preliminarily compare and evaluate the preset driver fatigue threshold A with the obtained driver fatigue coefficient Jsp, and generate a warning message according to the evaluation result to activate the second evaluation mechanism. The second evaluation mechanism is used to preset the first driving safety threshold M and the second driving safety threshold N and perform a secondary comparison and evaluation with the obtained comprehensive driving warning index ZHY, and generate a warning message according to the evaluation result.

[0071] In this embodiment, through five key modules, namely the data acquisition module, the feature extraction module, the data analysis module, the comprehensive analysis module, and the evaluation and early warning module, a comprehensive monitoring of vehicle driving and driver status is achieved; the data acquisition module uses the dual-lens driving recorder Tandem to continuously capture video data inside and outside the vehicle and perform preprocessing to ensure the quality and effectiveness of the data; the feature extraction module, through a deep learning model, efficiently identifies and extracts key features in the video data, providing a solid foundation for subsequent data analysis; the data analysis module further processes these feature vectors, and through multi-level analysis and correlation analysis, obtains the vehicle driving coefficient Clx, the traffic density coefficient Jtm, and the driver fatigue coefficient Jsp, providing an accurate assessment of the driving status and driver fatigue; the comprehensive analysis module correlates and calculates these coefficients to obtain the comprehensive driving early warning index ZHY, thus providing a comprehensive driving safety assessment index; the evaluation and early warning module, through preset thresholds, conducts primary and secondary evaluations on the driver fatigue coefficient and the comprehensive driving early warning index, generates real-time early warning information, helps the driver adjust driving behavior in a timely manner, and avoids potential safety hazards.

[0072] Embodiment 2

[0073] This embodiment is an explanatory description based on Embodiment 1. Please refer to Figure 1 , specifically: the data acquisition module includes a video acquisition unit and a video preprocessing unit;

[0074] The video acquisition unit is used to install the dual-lens driving recorder Tandem inside the vehicle to real-time collect the facial video data of the driver and the video data of the vehicles, pedestrians, and road surfaces driving outside the vehicle;

[0075] The video preprocessing unit is used to perform preprocessing on the collected video data of the vehicles, pedestrians, and road surfaces. The preprocessing methods include video denoising, image enhancement, and target detection preprocessing.

[0076] In this embodiment, the video acquisition unit uses the dual-lens driving recorder Tandem installed inside the vehicle to real-time collect the facial video data of the driver and the video data of the vehicles, pedestrians, and road surfaces outside the vehicle, ensuring the timeliness and comprehensiveness of the data; the video preprocessing unit performs preprocessing on the collected video data of the vehicles, pedestrians, and road surfaces. The preprocessing methods include video denoising, image enhancement, and target detection preprocessing, effectively improving the quality of the video data; this processing method eliminates noise interference, optimizes the image contrast and brightness, and accurately identifies important objects in the video, providing high-quality input data for subsequent feature extraction and analysis.

[0077] Embodiment 3

[0078] This embodiment is an explanatory description carried out in Embodiment 2. Please refer to Figure 1 , specifically: The feature extraction module includes a feature recognition unit and a feature extraction unit;

[0079] The feature recognition unit includes a data annotation unit and a deep learning unit;

[0080] The data annotation unit is used to collect a large amount of historical video data of in-vehicle drivers, out-of-vehicle moving vehicles and pedestrians, import it into the CVAT annotation tool, and through the automatic frame segmentation and annotation function of the CVAT annotation tool, segment the video data into separate frame pictures, and annotate the frame pictures with drivers inside the vehicle and outside the vehicle;

[0081] By annotating the facial features of the frame pictures with drivers inside the vehicle, the facial features include eye closure features, eye blinking features and mouth opening and closing features;

[0082] By annotating the vehicle driving features and pedestrian movement features of the out-of-vehicle frame pictures, the vehicle driving features include vehicle speed features, vehicle distance features, vehicle deviation features, road width features and lane number features, and the pedestrian movement features include pedestrian flow features; The deep learning unit includes a model construction unit and a model training unit;

[0083] The model construction unit is used to use the convolutional neural network CNN, the recurrent neural network RNN and the long short-term memory network LSTM as the model framework to construct a driver fatigue state recognition model and a vehicle external situation recognition model; Extract low-level to high-level driver facial features from the driver's facial image through the convolutional neural network CNN, process the facial image sequence data through the recurrent neural network RNN to capture the time-related features of the driver, and further improve the recognition ability of the time-related features based on the memory unit of the long short-term memory network LSTM; At the same time, in the real-time object detection system SSD and the semantic segmentation model DeepLab, perform object detection and semantic segmentation to construct a vehicle external situation recognition model, and then integrate the driver fatigue state recognition model and the vehicle external situation recognition model to obtain a deep learning model; Through the real-time object detection system SSD, perform object detection on vehicles and pedestrians, and can quickly identify and locate vehicles and pedestrians outside the vehicle in the real-time video stream. Use the semantic segmentation model DeepLab to perform semantic segmentation on the out-of-vehicle scene, and accurately segment roads, lane lines, pedestrians and out-of-vehicle traffic elements;

[0084] The model training unit is used to divide the labeled historical in-vehicle driver and out-of-vehicle driving video data into a training set and a validation set, import the training set into the constructed deep learning model, perform iterative training on the deep learning model, and then verify the accuracy of the deep learning model in identifying the fatigue characteristics of the driver and the driving characteristics outside the vehicle through the validation set, and continuously perform iterative learning to optimize the deep learning model;

[0085] The feature extraction unit is used to import the real-time collected facial image set and out-of-vehicle image set into the deep learning model for feature extraction, and extract the fatigue driving data set, traffic environment data set and vehicle driving data set in real time;

[0086] The fatigue driving data set includes the eye closure degree yb, the mouth opening and closing frequency zz, the eye closure time ys, and the eye blinking frequency yz;

[0087] The traffic environment data set includes the vehicle flow cl, the pedestrian flow xl, the road width dk, and the number of lanes cs;

[0088] The vehicle driving data set includes the vehicle acceleration js, the vehicle speed sd, the vehicle deviation value pl, and the distance to the vehicle ahead qj.

[0089] In this embodiment, the feature extraction module significantly improves the efficiency of the intelligent driving monitoring system through the combination of the feature recognition unit and the feature extraction unit; the feature recognition unit uses the CVAT annotation tool to automatically segment the video data into frames and perform precise annotation, constructs a deep learning model through the convolutional neural network CNN, the recurrent neural network RNN, and the long short-term memory network LSTM, and respectively identifies the facial fatigue characteristics of the driver and the out-of-vehicle situation; this method not only improves the accuracy of driver fatigue state recognition, but also can monitor the out-of-vehicle environment in real time, ensuring driving safety; the deep learning unit combines the real-time object detection system SSD and the semantic segmentation model DeepLab to achieve efficient detection of out-of-vehicle vehicles and pedestrians and precise segmentation of road scenes, further improving the system's understanding ability of the external environment; through the annotation and training of a large amount of historical data, the recognition ability of the deep learning model is continuously optimized, making it more adaptable to different scenarios and having higher recognition accuracy; the feature extraction unit imports the real-time collected driver facial images and out-of-vehicle images into the deep learning model, extracts the fatigue driving data set, traffic environment data set and vehicle driving data set, and through precise feature extraction and data analysis, the system can monitor the driver's state and driving environment in real time, provide timely and effective warning information, and greatly improve driving safety.

[0090] Embodiment 4

[0091] This embodiment is an explanatory description based on Embodiment 3. Please refer to Figure 1, specifically: the data analysis module includes a vehicle behavior analysis unit, a road condition analysis unit, and a driver behavior analysis unit;

[0092] The vehicle behavior analysis unit is used to calculate the vehicle driving coefficient Clx by summarizing the extracted vehicle driving data set after normalization;

[0093] The vehicle driving coefficient Clx is obtained through the following formula;

[0094] ;

[0095] In the formula, ln represents the logarithmic function, and e represents the exponential function.

[0096] The road condition analysis unit is used to calculate the traffic density coefficient Jtm by summarizing the extracted traffic environment data set after normalization;

[0097] The traffic density coefficient Jtm is obtained through the following formula;

[0098] ;

[0099] In the formula, ln represents the logarithmic function.

[0100] The driver behavior analysis unit is used to calculate the driver fatigue coefficient Jsp by summarizing the extracted driver fatigue data set after normalization;

[0101] The driver fatigue coefficient Jsp is obtained through the following formula;

[0102] ;

[0103] In the formula, ln represents the logarithmic function, e represents the exponential function, represents the maximum value of the eye closing time.

[0104] The comprehensive analysis module is used to calculate the comprehensive driving warning index ZHY by summarizing the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp after normalization;

[0105] The comprehensive driving warning index ZHY is calculated through the following formula;

[0106] ;

[0107] In the formula, ln represents the logarithmic function, e represents the exponential function, sin represents the sine function, and Z represents the correction constant.

[0108] In this embodiment, the data analysis module effectively improves the overall efficiency of the intelligent driving monitoring system through the integration of the vehicle behavior analysis unit, the road condition analysis unit, and the driver behavior analysis unit. The vehicle behavior analysis unit accurately calculates the vehicle driving coefficient Clx by normalizing the vehicle driving data set to ensure the accurate assessment of the vehicle driving state. The road condition analysis unit accurately calculates the traffic density coefficient Jtm by normalizing the traffic environment data set to reflect the road condition in real time. The driver behavior analysis unit accurately calculates the driver fatigue coefficient Jsp by normalizing the driver fatigue data set to effectively monitor the driver's fatigue state. The comprehensive analysis module normalizes and comprehensively calculates the above three coefficients to obtain the comprehensive driving warning index ZHY, providing the system with comprehensive and accurate warning capabilities. Compared with traditional technical means, this system significantly improves the accuracy and real-time performance of driving monitoring through multi-level data analysis and the application of deep learning models, thus greatly enhancing driving safety.

[0109] Embodiment 5

[0110] This embodiment is an explanatory note based on Embodiment 4. Please refer to Figure 1 , specifically: The evaluation and warning module includes a first evaluation unit and a second evaluation unit;

[0111] The first evaluation unit presets a vehicle behavior threshold A based on the historical safe driving speed and deviation range of vehicle driving, and then makes a preliminary evaluation with the obtained driver fatigue coefficient Jsp to analyze the driving process information of the vehicle during driving. The specific evaluation scheme is as follows;

[0112] Evaluate the result obtained by comparing the driver fatigue coefficient Jsp with the preset driver fatigue threshold A to obtain the first prediction result;

[0113] When the driver fatigue coefficient Jsp ≤ the preset driver fatigue threshold A, it means that the driver is in a normal driving state;

[0114] When the driver fatigue coefficient Jsp > the preset driver fatigue threshold A, it means that the driver is in a fatigue driving state. At this time, the driving recorder reminds the driver every 5 minutes by voice: You are driving fatigued. Please take a break.

[0115] The second evaluation unit presets a first driving safety threshold M and a second driving safety threshold N based on the comprehensive safety value of vehicle driving, and then makes a secondary evaluation with the obtained comprehensive driving warning index ZHY to analyze the road conditions and the driver's driving conditions during the driving of the current vehicle. The specific evaluation scheme is as follows;

[0116] When the comprehensive driving warning index ZHY ≤ the preset first driving safety threshold M, it indicates that the vehicle is driving safely, and the driver's behavior and road conditions are normal;

[0117] When the preset first driving safety threshold M < the comprehensive driving warning index ZHY < the preset second driving safety threshold N, it indicates that the driving state of the vehicle is abnormal. At this time, the driving recorder reminds the driver through voice that the road ahead is congested, and please pay attention to reducing the speed;

[0118] When the comprehensive driving warning index ZHY ≥ the preset second driving safety threshold N, it indicates that the vehicle is in a dangerous state. At this time, the driving recorder reminds the driver through voice that the road ahead is congested, please immediately reduce the speed, turn on the vehicle's emergency double flash indicator, and control the driving recorder to increase the data collection frequency by 90%.

[0119] In this embodiment, the first evaluation unit sets the vehicle behavior threshold A through the historical safe driving speed and the deviation range, and compares it with the driver fatigue coefficient Jsp to evaluate the driver's fatigue state in real time; when it is detected that the driver is in a fatigue driving state, the system can give a voice reminder every 5 minutes to effectively prevent driving risks caused by fatigue; the second evaluation unit sets the first driving safety threshold M and the second driving safety threshold N through the comprehensive safety value, and compares them with the comprehensive driving warning index ZHY to comprehensively evaluate the driving safety of the vehicle; when the driving state of the vehicle is abnormal or in a dangerous state, the system can give a voice reminder in time and increase the data collection frequency according to the degree of danger to ensure real-time monitoring and rapid response under complex road conditions; compared with traditional technical means, this module significantly improves the system's ability to identify and intervene in potential risks through a multi-level warning mechanism, thereby effectively improving driving safety and the driver's reaction speed.

[0120] Embodiment 6

[0121] Please refer to Figure 2 , a driving intelligent monitoring method based on a driving recorder, including the following steps:

[0122] S1. Continuously capture the video data of the driver in the vehicle and the driving outside the vehicle through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data;

[0123] S2. Based on the convolutional neural network CNN, the recurrent neural network RNN, and the long short-term memory network LSTM as the basic framework of the deep learning model, construct a deep learning model to identify the relevant features in the collected video data and perform feature extraction;

[0124] S3. Process the output feature vectors, perform multi-level data parsing and correlation analysis, and use statistical and machine learning models to normalize and comprehensively calculate the preprocessed data set to obtain the vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp respectively;

[0125] S4. Perform correlation calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain the comprehensive driving warning index ZHY;

[0126] S5. Make a preliminary comparison and evaluation between the preset driver fatigue threshold A and the obtained driver fatigue coefficient Jsp, and generate a warning message according to the evaluation result to activate the second evaluation mechanism. The second evaluation mechanism is used to make a secondary comparison and evaluation between the preset first driving safety threshold M and the preset second driving safety threshold N and the obtained comprehensive driving warning index ZHY, and generate a warning message according to the evaluation result.

[0127] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A driving intelligent monitoring system based on a driving recorder, characterized in that: It includes a data acquisition module, a feature extraction module, a data analysis module, a comprehensive analysis module, and an evaluation and warning module; The data acquisition module is used to continuously capture the video data of the driver inside the vehicle and the vehicle driving outside through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data; The feature extraction module constructs a deep learning model based on the convolutional neural network CNN, the recurrent neural network RNN, and the long short-term memory network LSTM as the basic framework of the deep learning model, identifies the relevant features in the collected video data, and performs feature extraction; The data analysis module is used to process the feature vectors output by the feature extraction module, perform multi-level data parsing and correlation analysis, and use statistical and machine learning models to perform normalization processing and comprehensive calculation on the extracted data set to obtain the vehicle driving coefficient Clx, the traffic density coefficient Jtm, and the driver fatigue coefficient Jsp respectively; The comprehensive analysis module is used to perform correlated calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain the comprehensive driving warning index ZHY; The evaluation and warning module is used to perform a preliminary comparison and evaluation between the preset driver fatigue threshold A and the obtained driver fatigue coefficient Jsp, and generate a warning message according to the evaluation result, and activate the second evaluation mechanism. The second evaluation mechanism is used to perform a secondary comparison and evaluation between the preset first driving safety threshold M and the preset second driving safety threshold N and the obtained comprehensive driving warning index ZHY, and generate a warning message according to the evaluation result.

2. The intelligent driving monitoring system based on a driving recorder according to claim 1, wherein: The data acquisition module includes a video acquisition unit and a video preprocessing unit; The video acquisition unit is used to install the dual-lens driving recorder Tandem inside the vehicle to collect the facial video data of the driver in real time and the video data of the vehicle, pedestrians, and road surface driving outside; The video preprocessing unit is used to preprocess the collected video data of the vehicle, pedestrians, and road surface. The preprocessing methods include video denoising, image enhancement, and target detection preprocessing.

3. The intelligent driving monitoring system based on a driving recorder according to claim 2, characterized in that: The feature extraction module includes a feature recognition unit and a feature extraction unit; The feature recognition unit includes a data annotation unit and a deep learning unit; The data annotation unit is used to collect a large amount of historical video data of the driver inside the vehicle and the vehicle and pedestrians driving outside, import it into the CVAT annotation tool, and use the automatic frame segmentation and annotation function of the CVAT annotation tool to split the video data into individual frame pictures and annotate the frame pictures with drivers inside the vehicle and outside; By annotating the facial features of the frame pictures with drivers inside the vehicle, the facial features include eye closure features, eye blinking features, and mouth opening and closing features; By annotating the vehicle driving features and pedestrian movement features of the frame pictures outside the vehicle, the vehicle driving features include vehicle speed features, vehicle distance features, vehicle deviation features, road width features, and lane number features, and the pedestrian movement features include pedestrian flow features; The deep learning unit includes a model construction unit and a model training unit; The model construction unit is used to construct a driver fatigue state recognition model and a vehicle external situation recognition model by using a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), and a Long Short-Term Memory Network (LSTM) as the model framework; extract driver facial features from low level to high level from the driver's facial images through the CNN, process the facial image sequence data through the RNN to capture the time-related features of the driver, and further improve the recognition ability of the time-related features based on the memory unit of the LSTM; at the same time, perform object detection and semantic segmentation in the real-time object detection system SSD and the semantic segmentation model DeepLab to construct a vehicle external situation recognition model, and then integrate the driver fatigue state recognition model and the vehicle external situation recognition model to obtain a deep learning model; The real-time object detection system SSD is used to perform object detection on vehicles and pedestrians, which can quickly identify and locate vehicles and pedestrians outside the vehicle in the real-time video stream. The semantic segmentation model DeepLab is used to perform semantic segmentation on the external vehicle scene to accurately segment roads, lane lines, pedestrians, and external traffic elements; The model training unit is used to divide the labeled historical in-vehicle driver and out-of-vehicle driving video data into a training set and a validation set, import the training set into the constructed deep learning model, perform iterative training on the deep learning model, and then verify the accuracy of the deep learning model in recognizing the fatigue features of the driver and the driving features outside the vehicle through the validation set, and continuously perform iterative learning to optimize the deep learning model; The feature extraction unit is used to import the real-time collected facial picture set and external vehicle picture set into the deep learning model for feature extraction, and extract the fatigue driving data set, traffic environment data set, and vehicle driving data set in real time; The fatigue driving data set includes the eye closure degree yb, mouth opening and closing frequency zz, eye closure time ys, and eye blinking frequency yz; The traffic environment data set includes vehicle flow cl, pedestrian flow xl, road width dk, and number of lanes cs; The vehicle driving data set includes vehicle acceleration js, vehicle speed sd, vehicle deviation value pl, and distance to the vehicle ahead qj.

4. The intelligent driving monitoring system based on a driving recorder according to claim 3, characterized in that: The data analysis module includes a vehicle behavior analysis unit, a road condition analysis unit, and a driver behavior analysis unit; The vehicle behavior analysis unit is used to calculate and obtain the vehicle driving coefficient Clx by summarizing and normalizing the extracted vehicle driving data set; The vehicle driving coefficient Clx is obtained through the following formula; ; In the formula, ln represents the logarithmic function, and e represents the exponential function.

5. The intelligent driving monitoring system based on a driving recorder according to claim 4, wherein: The road condition analysis unit is used to calculate and obtain the traffic density coefficient Jtm by summarizing and normalizing the extracted traffic environment data set; The traffic density coefficient Jtm is obtained through the following formula; ; In the formula, ln represents the logarithmic function.

6. The intelligent driving monitoring system based on a driving recorder according to claim 4, characterized in that: The driver behavior analysis unit is used to calculate and obtain the driver fatigue coefficient Jsp by summarizing and normalizing the extracted driver fatigue data set; The driver fatigue coefficient Jsp is obtained through the following formula; ; where ln represents the logarithmic function and e represents the exponential function, represents the maximum value of the eye closure time.

7. The intelligent driving monitoring system based on a driving recorder according to claim 1, characterized in that: The comprehensive analysis module is used to normalize the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp, and then perform a summary calculation to obtain the comprehensive driving warning index ZHY; The comprehensive driving warning index ZHY is calculated through the following formula; ; In the formula, ln represents the logarithmic function, e represents the exponential function, sin represents the sine function, and Z represents the correction constant.

8. An intelligent driving monitoring system based on a driving recorder according to claim 4, characterized in that: The evaluation and warning module includes a first evaluation unit and a second evaluation unit; The first evaluation unit presets a vehicle behavior threshold A based on the historical safe driving speed and deviation range of the vehicle during driving, and then performs a preliminary evaluation with the obtained driver fatigue coefficient Jsp to analyze the driving process information of the vehicle during driving. The specific evaluation scheme is as follows; Evaluate the result obtained by comparing the driver fatigue coefficient Jsp with the preset driver fatigue threshold A to obtain the first prediction result; When the driver fatigue coefficient Jsp ≤ the preset driver fatigue threshold A, it means that the driver is in a normal driving state; When the driver fatigue coefficient Jsp > the preset driver fatigue threshold A, it means that the driver is in a fatigued driving state. At this time, the driving recorder reminds the driver once every 5 minutes by voice: You are driving fatigued. Please take a break.

9. The intelligent driving monitoring system based on a driving recorder according to claim 8, characterized in that: The second evaluation unit presets a first driving safety threshold M and a second driving safety threshold N based on the comprehensive safety value of the vehicle during driving, and then performs a secondary evaluation with the obtained comprehensive driving warning index ZHY to analyze the road conditions and the driver's driving conditions of the vehicle during driving. The specific evaluation scheme is as follows; When the comprehensive driving warning index ZHY ≤ the preset first driving safety threshold M, it means that the vehicle is driving safely and the driver's behavior and road conditions are normal; When the preset first driving safety threshold M < the comprehensive driving warning index ZHY < the preset second driving safety threshold N, it means that the driving state of the vehicle is abnormal. At this time, the driving recorder reminds the driver by voice that the road ahead is congested. Please pay attention to reducing the speed; When the comprehensive driving warning index ZHY ≥ the preset second driving safety threshold N, it means that the vehicle is in a dangerous driving state. At this time, the driving recorder reminds the driver by voice that the road ahead is congested. Please immediately reduce the speed, turn on the vehicle's emergency double flash indicator, and control the driving recorder to increase the data collection frequency by 90%.

10. A driving intelligent monitoring method based on a driving recorder, comprising the driving intelligent monitoring system based on a driving recorder according to any one of the above claims 1 to 9, characterized in that: It includes the following steps: S1. Continuously capture the video data of the driver inside the vehicle and the driving outside the vehicle through the dual-lens driving recorder Tandem installed on the vehicle, and preprocess the collected video data; S2. Based on the convolutional neural network CNN, recurrent neural network RNN, and long short-term memory network LSTM as the basic framework of the deep learning model, construct a deep learning model to identify the relevant features in the collected video data and perform feature extraction; S3. Process the output feature vectors, conduct multi-level data parsing and correlation analysis, and use statistical and machine learning models to perform normalization processing and comprehensive calculations on the preprocessed data set to obtain the vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp respectively; S4. Conduct correlation calculations on the obtained vehicle driving coefficient Clx, traffic density coefficient Jtm, and driver fatigue coefficient Jsp to obtain the comprehensive driving warning index ZHY; S5. Preset the driver fatigue threshold A and conduct a preliminary comparison and evaluation with the obtained driver fatigue coefficient Jsp, and generate warning information according to the evaluation result, and activate the second evaluation mechanism. The second evaluation mechanism is used to preset the first driving safety threshold M and the preset second driving safety threshold N and conduct a secondary comparison and evaluation with the obtained comprehensive driving warning index ZHY, and generate warning information according to the evaluation result.

Images