Driving data recording method and device, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of intelligent vehicles, and provides a driving data recording method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining real-time driving data through employing a vehicle-mounted sensor and a vehicle-mounted camera, and the real-time driving data comprises vehicle feature data, environment feature data and user feature data; preprocessing the real-time driving data to form candidate driving data, the preprocessing including at least one of data cleaning, data compression and data fusion; analyzing the candidate driving data, and determining user state data including emotional state data and heart rate state data; fusing the candidate driving data and the user state data to generate target driving data; and performing clustering processing on the target driving data, and storing the target driving data in a classified manner. According to the technical scheme provided by one or more embodiments of the invention, rich driving data can be acquired in real time, and the driving data can be effectively mined and sorted.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent vehicle technology, and in particular to a driving data recording method, device, electronic device and readable storage medium. Background Art

[0002] In the context of the rapid development of modern automobile technology, the issues of driving safety and efficiency have become increasingly prominent and have become the top priority of research and innovation in the automotive industry. With the continuous advancement of science and technology, especially the rise of intelligent transportation systems, automobile driving data recording systems, as an emerging technical means, have received widespread attention.

[0003] However, traditional driving data recording systems can only simply record and store driving data, and lack the ability to deeply analyze and process the data. This results in the system being unable to effectively identify potential risks in driving behavior, and unable to provide timely feedback and warnings. Summary of the invention

[0004] In view of this, one or more embodiments of the present disclosure provide a driving data recording method, device, electronic device and readable storage medium, which can not only obtain rich driving data in real time, but also effectively mine and organize driving data.

[0005] On the one hand, the present disclosure provides a driving data recording method, the method comprising: using a vehicle-mounted sensor and a vehicle-mounted camera to obtain real-time driving data, the real-time driving data including vehicle feature data, environmental feature data and user feature data; preprocessing the real-time driving data to form candidate driving data, the preprocessing including at least one of data cleaning, data compression and data fusion; analyzing the candidate driving data to determine user state data, the user state data including emotional state data and heart rate state data; fusing the candidate driving data with the user state data to generate target driving data; clustering the target driving data, and classifying and storing the target driving data.

[0006] On the other hand, the present disclosure further provides a driving data recording device, the device comprising: a data acquisition unit, used to acquire real-time driving data using vehicle-mounted sensors and vehicle-mounted cameras, the real-time driving data comprising vehicle characteristic data, environmental characteristic data and user characteristic data; a preprocessing unit, used to preprocess the real-time driving data to form candidate driving data, the preprocessing comprising at least one of data cleaning, data compression and data fusion; a data analysis unit, used to analyze the candidate driving data and determine user status data, the user status data comprising emotional status data and heart rate status data; a data fusion unit, used to fuse the candidate driving data with the user status data to generate target driving data; a data storage unit, used to cluster the target driving data and classify and store the target driving data.

[0007] On the other hand, the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned driving data recording method is implemented.

[0008] On the other hand, the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned driving data recording method is implemented.

[0009] The technical solution provided by one or more embodiments of the present disclosure not only obtains a large amount of real-time driving data, but also ensures that the real-time driving data covers vehicle characteristics, environmental characteristics and user characteristics, thereby ensuring the mineability of driving data. Through data preprocessing, useful information can be quickly extracted from the original data, noise and outliers can be removed, and a solid foundation can be laid for subsequent in-depth analysis. By analyzing the driving data, the user's emotional state and heart rate state can be determined, so that when the user's state is abnormal, a timely warning can be issued, thereby improving driving safety and road safety. The generated target driving data retains both the candidate driving data after preprocessing and the user status data after in-depth analysis, which is conducive to further analysis and data return. Clustering is performed on the target driving data to make the stored target driving data more orderly, which is convenient for viewing statistical information and later calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The features and advantages of the embodiments of the present disclosure may be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0011] Figure 1 A schematic diagram showing the steps of a driving data recording method in one embodiment of the present disclosure is shown;

[0012] Figure 2A schematic diagram of the structure of a driving data recording system in one embodiment of the present disclosure is shown;

[0013] Figure 3 A schematic diagram of functional modules of a driving data recording device in one embodiment of the present disclosure is shown;

[0014] Figure 4 A schematic structural diagram of an electronic device in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0016] See also Figure 1 , a driving data recording method provided by an embodiment of the present disclosure may include the following steps.

[0017] S1: Using vehicle-mounted sensors and vehicle-mounted cameras to obtain real-time driving data, the real-time driving data includes vehicle characteristic data, environment characteristic data and user characteristic data.

[0018] In this embodiment, the vehicle-mounted sensors may include: an inertial measurement unit, such as an accelerometer and a gyroscope, for obtaining the real-time motion status of the vehicle; a vehicle speed sensor, for monitoring the vehicle speed in real time; a steering angle sensor, for monitoring the steering wheel angle and determining the direction of the vehicle; an accelerator and brake pedal sensor, for monitoring the pedal position and assisting in determining the user's driving intention; an airbag sensor, for recording collision data in the event of a collision; vehicle bus data acquisition, through the OBD interface, for obtaining vehicle fault codes, engine status and other data; a GPS module, for obtaining the vehicle's real-time location information; a radar module, for identifying pedestrians, obstacles and other vehicles around the vehicle; and a heart rate detection device, for monitoring the user's heart rate in a non-contact manner.

[0019] In this embodiment, the vehicle-mounted camera may include: a driving recorder for recording the road conditions ahead; and an in-vehicle monitoring camera for acquiring user videos and images and identifying user actions and expressions.

[0020] In this embodiment, the real-time driving data may cover vehicle characteristics, environmental characteristics, and user characteristics, thereby ensuring the mineability of the real-time driving data.

[0021] S2: Preprocessing the real-time driving data to form candidate driving data, wherein the preprocessing includes at least one of data cleaning, data compression and data fusion.

[0022] In this embodiment, the original real-time driving data needs to undergo a standardized preprocessing process. Through the data cleaning process, the real-time driving data can be subjected to operations such as denoising, deduplication, missing value filling, error correction, and format conversion to ensure data quality. Through the data compression process, the pressure of data transmission and storage can be reduced. Through the data fusion process, data collected by different sensors can be fused to improve data accuracy.

[0023] S3: Analyze the candidate driving data to determine user state data, where the user state data includes emotional state data and heart rate state data.

[0024] In this embodiment, by mining candidate driving data, user status data can be determined, thereby focusing on the user's emotional state and driving behavior, such as fatigue driving, distraction, improper operation, etc. In this way, potential risks in the user's driving behavior can be effectively identified, and timely feedback and warnings can be provided.

[0025] In some embodiments, analyzing the candidate driving data and determining the user state data includes: extracting user facial image data based on the candidate driving data; inputting the user facial image data into an emotion classification model, so that the emotion classification model determines the emotional state data based on a facial feature point positioning algorithm and a local binary pattern histogram algorithm.

[0026] Specifically, the emotion classification model can be a pre-trained deep learning model. Using the emotion classification model, facial recognition technology can be implemented to process the user's facial micro-expressions in the user video acquired in real time by the vehicle-mounted camera. In this way, real-time detection of user emotions can be achieved, real-time monitoring of the user's dangerous driving state can be achieved, and driving suggestions can be provided to the user, thereby reducing the probability of accidents and improving driving safety.

[0027] In an actual application example, the process of determining the emotional state data can be implemented through steps S311 - S317 .

[0028] S311 dataset preparation: Collect and label facial images of users in different emotional states. Ensure that the images in the dataset are diverse, including different lighting, angles, and expression intensities.

[0029] S312 Face Detection: Use a cascade classifier based on Haar features and AdaBoost algorithm to detect facial areas in images.

[0030] S313 Facial feature point positioning: Facial key points usually include the positions of eyes, nose, mouth, and chin. The facial feature point locator based on local binary features is used to locate the key feature points of the face, so as to improve the accuracy of emotion recognition.

[0031] S314 Feature Extraction: For each detected face, features are extracted based on the LBPH algorithm; it is based on local binary pattern features extracted from the facial image, which are then converted into histogram form for subsequent classification or recognition tasks.

[0032] S315 training model: train the recognizer model using the extracted features.

[0033] S316 Emotion Classification: Train a classifier based on a self-made dataset and use the trained features to identify different emotions.

[0034] S317 Testing and Evaluation: Use images that have not been used in training to test the performance of the model; use the trained emotion recognizer to perform real-time result detection on real-time camera shooting data.

[0035] In some embodiments, analyzing the candidate driving data to determine the user state data includes: extracting user facial video data based on the candidate driving data; analyzing facial optical change information based on the user facial video data; and calculating the heart rate state data using the facial optical change information. It can be seen that the heart rate state data is achieved by analyzing the optical absorption characteristics of hemoglobin in the facial area.

[0036] In a practical application example, the process of determining the heart rate status data can be implemented through steps S321-S326.

[0037] S321 Video Capture: Capture continuous frames of user images through the camera.

[0038] S322 Face localization: Use the pre-trained face detection model to find the face area, ensuring that subsequent processing is focused on the face part.

[0039] S323 Blood flow change analysis: In the face area, especially the cheek area, changes in blood flow will cause slight color changes; use color space conversion to enhance these changes and use differential algorithms to track these changes.

[0040] S324 Signal Processing: Heart rate information is separated from the raw optical signal through filtering, noise reduction and feature extraction signal processing techniques.

[0041] S325 Heart rate calculation: Use Fourier transform time domain or frequency domain analysis method to obtain the number of heart beats per minute, that is, heart rate.

[0042] S326 outputs heart rate: displays the estimated heart rate in digital and graphical form.

[0043] S4: Fusion the candidate driving data and the user state data to generate target driving data.

[0044] In this embodiment, the target driving data retains both the pre-processed candidate driving data and the user status data after in-depth analysis, which is conducive to further data analysis and data return to the source.

[0045] In some embodiments, the fusing of the candidate driving data with the user status data to generate the target driving data includes: extracting and merging features based on the candidate driving data and the user status data to generate multimodal data, wherein the multimodal data includes user expression modal data, user heart rate modal data, vehicle motion modal data, and environmental modal data; inputting the multimodal data into an information fusion model to generate the target driving data, wherein the target driving data includes user safety information and vehicle safety information.

[0046] Specifically, by fusing user modal information such as user expressions, user heart rate, and user actions, the user's own safety information can be determined, such as sudden illness of the user, user fatigue driving, etc. By fusing vehicle modal information such as vehicle motion status and vehicle equipment status, the vehicle's own safety information can be determined, such as power system failure, brake failure, etc. By fusing comprehensive modal information such as environmental status, traffic status, vehicle motion status, and user status, the vehicle's traffic safety information, such as potential collision, can be determined.

[0047] In a practical application example, a fusion process of multimodal driving data can be implemented through steps S331-S336.

[0048] S331 Data preprocessing: First, data from different modalities (such as facial expressions, monitored heart rate) need to be preprocessed (data cleaning, noise removal, normalization operations, etc.) to ensure the quality and consistency of the data.

[0049] S332 Feature Extraction: Next, useful features are extracted from the preprocessed data. For facial expression recognition, this includes extracting the motion features of the facial feature points of the eyes, mouth, and eyebrows. For heart rate monitoring, this includes extracting the frequency, amplitude, and heart rate variability features of the heart rate.

[0050] S333 Feature Selection: Since data from different modalities may have redundant or complementary features, feature selection is required to select the features that are most useful for identification. For example, the best features can be selected by using a random forest machine learning algorithm.

[0051] An example of feature selection may include steps a to d.

[0052] aData preprocessing: Preprocess the data from different modalities.

[0053] b Feature extraction: Extract useful features from the preprocessed data. For facial expression recognition, extract the motion features of the facial feature points of eyes, mouth, and eyebrows. For heart rate monitoring, extract the frequency, amplitude, and heart rate variability features of the heart rate.

[0054] cFeature importance evaluation: Use the random forest algorithm to evaluate the importance of each feature to the model prediction results. The random forest algorithm can provide the contribution of each feature to the prediction during the training process, thereby helping to determine which features are more important.

[0055] d Feature selection: Based on the feature importance scores provided by the random forest algorithm, the top-ranked features are selected as the optimal feature subset.

[0056] S334 Model Training and Fusion: Use the selected features to train the fusion recognition model. Using the random forest machine learning algorithm, features from different modalities are fused during the training process to build a fusion recognition model that can integrate different information.

[0057] S335 Model Evaluation and Optimization: Evaluate the performance of the fusion recognition model to ensure that it can accurately identify the driver's emotions and heart rate status. Establish cross-validation evaluation indicators, and optimize and adjust the fusion recognition model based on the indicator evaluation results to improve its performance.

[0058] S336 Real-time monitoring and recognition: The trained fusion recognition model is applied to the real-time monitoring system. By continuously inputting data from facial expression recognition and heart rate monitoring, the fusion recognition model can recognize the driver's emotions and heart rate status in real time, and trigger a warning mechanism when negative emotions and abnormal heart rate are detected.

[0059] S5: performing clustering processing on the target driving data, and storing the target driving data in categories.

[0060] In this embodiment, a document-type database structure can be designed according to the characteristics of the target driving data to store the target driving data. For example, each driving trip is taken as a document, and the document can contain information such as timestamp, speed, acceleration, geographic location, camera image, etc.

[0061] In this embodiment, for the stored target driving data, data backup operations can be performed regularly to ensure that the target driving data can be quickly restored when a storage device fails. For the stored target driving data, it can be updated regularly to reflect the latest user activities.

[0062] In this implementation, the target driving data may be clustered and then stored in a classified manner, which not only improves the storage efficiency of the target driving data but also ensures that the deposited target driving data is more reliable and useful.

[0063] In some embodiments, clustering the target driving data includes: running a first clustering algorithm on the target driving data to determine a first clustering result; determining a plurality of clustering centers based on the first clustering result; determining the clustering centers as initial seed points of a second clustering algorithm; and using the second clustering algorithm and the initial seed points to process the target driving data to determine a second clustering result.

[0064] In some embodiments, the clustering processing of the target driving data further includes: analyzing the first clustering result to determine clustering difference points; analyzing the second clustering result to determine clustering noise points; and quantifying the abnormal scores of the clustering difference points and the clustering noise points.

[0065] Specifically, the first clustering algorithm may be a K-means algorithm, and the second clustering algorithm may be a DBSCAN algorithm. The K-means algorithm can quickly identify the main user groups and behavior patterns by clustering car driving data points into a predetermined number of clusters, but it is sensitive to noise and outliers, and assumes that the clusters are convex. The DBSCAN algorithm can identify clusters of any shape and exclude noise points, and does not depend on the preset number of clusters. The combination of the two enables anomaly detection to not only identify the main user behavior patterns, but also to discover abnormal patterns that K-means may ignore, such as small clusters or non-convex clusters, thereby improving the overall detection accuracy. In addition, the addition of DBSCAN makes the clustering process more flexible, is not affected by the initial center selection, reduces the risk of local optimal solutions, and enhances the robustness of clustering.

[0066] Combining the K-means and DBSCAN algorithms can optimize resource utilization and computational efficiency, while improving the storage quality of target driving data. The high computational efficiency of the K-means algorithm makes it suitable for processing large-scale automobile driving data sets, and although DBSCAN has a high computational complexity, its ability to recognize complex shape clusters is unparalleled. By first using K-means for preliminary clustering and then using DBSCAN to optimize the preliminary results, the quality of clustering can be improved while ensuring computational efficiency. This optimization not only improves the processing efficiency of target driving data, but also ensures that the deposited target driving data is more reliable and useful. High-quality storage of target driving data provides a solid foundation for subsequent automobile driving data analysis and business decision-making, and also provides a more effective and comprehensive technical means for protecting user information, which helps to identify and isolate potential malicious or abnormal behaviors, thereby ensuring the data security of target driving data.

[0067] In a practical application example, the process of clustering target driving data by combining the K-means algorithm and the DBSCAN algorithm can be as follows.

[0068] Step 1: Preprocess the car driving data to remove invalid or erroneous car driving data. Then, convert the string type car driving data into numerical features and extract key features.

[0069] Step 2: Use the K-means algorithm to initialize the cluster center. The formula is:

[0070]

[0071] Among them, C i is the selected i-th cluster center, X is the driving data set, d(x, C i ) is the driving data point x to the cluster center C i , P(x) is the probability of selecting point x as the cluster center.

[0072] Step 3: Use the cluster center obtained by K-means as the initial seed point of DBSCAN, run the DBSCAN algorithm, and automatically identify dense areas (clusters) and noise points for DBSCAN clustering. If there are at least MinPts points in the ε neighborhood of a point, the point is a core point. If the number of points in the ε neighborhood of a point is less than MinPts, but it is in the ε neighborhood of another core point, the point is a boundary point. Points that are neither core points nor boundary points are defined as noise points. Among them, ε represents the neighborhood radius, which represents the neighborhood size of a point in space; MinPts represents the minimum number of samples, which represents the minimum number of neighborhood points required for a core point.

[0073] The domain calculation formula can be as follows:

[0074] N ε (p) = {q∈X|d(p,q)≤v}

[0075] Where X is the driving data set, p and q are two driving data points in X, and N ε (p) is the epsilon neighborhood of p, and (p, q) is the distance between point p and point q.

[0076] Step 4: Merge clustering results. For the points that DBSCAN failed to cluster (noise points), calculate their distance to the K-means cluster center, and redistribute the DBSCAN noise points to the nearest K-means cluster according to their distance to the K-means cluster center. The calculation formula is:

[0077]

[0078] Among them, d(x,c) is the distance from point x to cluster center c, and n is the feature dimension.

[0079] For DBSCAN noise points, the probability of redistributing them to K-means clustering can be calculated:

[0080]

[0081] Among them, P(p→C i ) is the point p redistributed to the cluster center C i The probability of d(p, C i ) is the distance from point p to cluster center C i distance.

[0082] Step 5: Anomaly detection, analyze the noise points of DBSCAN and the K-means clustering results, and identify anomalies. Anomalies may be noise points of DBSCAN or points that are significantly different from other points in K-means clustering. For each point, an anomaly score S(p) can be calculated to quantify its degree of anomaly:

[0083]

[0084] Among them, S(p) is the abnormal score of point p, avg(d(p, N ε (p))) is the average distance from point p to all points in its ε neighborhood. The higher the anomaly score, the more likely point p is an anomaly.

[0085] Step 6: Intelligent audit and feedback, perform intelligent audit on the clustering results and generate a report containing abnormal patterns.

[0086] In some implementations, the driving data recording method further includes: transmitting the target driving data to a data sharing platform; and receiving a data analysis model pushed by the data sharing platform.

[0087] Specifically, through a simple and easy-to-use data upload interface, the target driving data can be anonymously uploaded to the user-authorized driving data sharing platform. The driving data sharing platform adopts an efficient data storage system to ensure the reliable storage and management of massive driving data. The driving data sharing platform implements a strict data access control policy to ensure that only authorized researchers and institutions can access shared data. The driving data sharing platform can be set up with a cross-vehicle learning mechanism to push data analysis models for different vehicles. These data analysis models can be generated by exchanging and learning driving data from different vehicles. These data analysis models can be personalized for each vehicle. The data analysis models generated by shared learning can quickly identify and disseminate abnormal driving patterns and improve the early warning capabilities of all vehicles.

[0088] In a practical application example, the data sharing platform can use a distributed learning framework to establish a distributed learning network, collect driving data from different vehicles, and optimize the data analysis model. The data sharing platform can regularly synchronize the data analysis models of each vehicle through an encrypted communication protocol to ensure the consistency and update of the model. The data sharing platform can use the driving data collected from many vehicles to train a more accurate driving behavior prediction model. The data sharing platform can combine the driving data of individual vehicles to personalize the general model to adapt to the driving style of different drivers.

[0089] In some implementations, the driving data recording method further includes: transmitting the target driving data to a target vehicle or a target server.

[0090] In a practical application example, the HTTPS protocol can be used to achieve communication between the vehicle and the server or other vehicles to ensure the security of data during transmission. Through the SSL / TLS handshake process, a secure connection can be established. During the data transmission process, the MQTT protocol can be used to implement message push, which is suitable for low-bandwidth, high-latency network environments. The MQTT publish / subscribe model allows a message to be obtained by multiple recipients at the same time, which is suitable for real-time distribution of target driving data.

[0091] In some implementations, the driving data recording method further includes: graphically displaying the target driving data; and receiving a user's interactive instruction regarding the target driving data.

[0092] In a practical application example, the target driving data can be converted into a chart using JavaScript libraries such as D3.js or Highcharts. The display and interaction of the target driving data can be implemented based on a responsive user interface based on HTML5, CSS3, and JavaScript to ensure the consistency and availability of the target driving data on different devices. The data display page is dynamically updated through Ajax technology, allowing users to view the target driving data of different time periods through operations such as sliding and clicking, and providing interactive functions such as search and filtering.

[0093] In some embodiments, the driving data recording method further includes: encrypting the target driving data. Before the target driving data is transmitted and stored, the target driving data may be encrypted and protected.

[0094] In a practical application example, the decentralization and immutability of blockchain technology can be used to store the hash value of the target driving data to ensure that the target driving data is not tampered with.

[0095] Before target driving data is shared, the K-Anonymity algorithm can be used to desensitize sensitive information to protect personal privacy. Role-based access control can assign different permissions to different users to limit access to sensitive data. Using the TLS protocol to encrypt target driving data, combined with the AES symmetric encryption algorithm and the RSA asymmetric encryption algorithm, can ensure the security of target driving data during data transmission.

[0096] See also Figure 2 A driving data recording system provided in one embodiment of the present disclosure may include a data acquisition module, an intelligent analysis and processing module, a data storage module, a data transmission module, a data display and interaction module, and a security and privacy protection module.

[0097] In this embodiment, the input end of the data acquisition module is connected to a sensor unit module and a camera unit module for acquiring real-time driving data, which includes vehicle feature data, environment feature data and user feature data.

[0098] In this embodiment, the intelligent analysis and processing module is internally configured with: a data preprocessing module, an emotional behavior analysis module, a data sharing and collaboration module, and a multimodal fusion module.

[0099] The raw data is transmitted from the data acquisition module to the data preprocessing module of the intelligent analysis and processing module. The preprocessed data and analysis results are stored in the data storage module. The stored data is sent to a remote server or mobile device through the data transmission module. The transmitted data is displayed in the data display and interaction module for the driver or vehicle management personnel to view. During the data transmission and storage process, the intelligent analysis and processing module sends the data to the security and privacy protection module for encryption and protection. Before storing the data, the storage module sends the data to the security and privacy protection module for encryption. Before sending the data, the transmission module sends the data to the security and privacy protection module for encryption.

[0100] The data preprocessing module fuses the data collected by different sensors to improve data accuracy; the data preprocessing module removes noise from the data to ensure data quality; the data preprocessing module reduces the pressure of data transmission and storage through data compression.

[0101] The emotional behavior analysis module is used to determine the user status data, which includes emotional status data and heart rate status data. For the specific process, please refer to the relevant introduction of step S3 above, which will not be repeated here.

[0102] The data sharing and collaboration module has an internal data upload interface, which provides drivers with a simple and easy-to-use data upload interface, allowing the vehicle's driving data to be uploaded anonymously to the platform. The platform backend uses an efficient data storage system to ensure the reliable storage and management of massive driving data. The platform implements a strict data access control policy to ensure that only authorized researchers and institutions can access shared data.

[0103] The multimodal fusion module is used to fuse the pre-processed driving data and the user state data determined by the emotional behavior analysis module to generate target driving data. For the specific process, please refer to the relevant introduction of step S4 above, which will not be repeated here.

[0104] In this embodiment, the data storage module uses the NoSQL database MongoDB. According to the characteristics of driving data, a document-type database structure is designed to treat each driving trip as a document. The data storage module regularly performs data backup operations and uses the replica set function of MongoDB to achieve redundant data storage to ensure that data can be quickly restored in the event of hardware failure. The data storage module regularly updates the information in the data warehouse to reflect the latest user activities.

[0105] The data storage module can also perform clustering processing on the target driving data and store the target driving data in a classified manner. For the specific process, refer to the relevant introduction of step S5 above, which will not be repeated here.

[0106] In this embodiment, the data transmission module uses the HTTPS protocol to encrypt the communication between the client (vehicle) and the server to ensure the security of the data during transmission. A secure connection is established through the SSL / TLS handshake process. The data transmission module uses the MQTT protocol to implement message push between servers, which is suitable for low-bandwidth, high-latency network environments. The MQTT publish / subscribe model allows a message to be obtained by multiple recipients at the same time, which is suitable for real-time distribution of driving data.

[0107] In this embodiment, the data display and interaction module uses JavaScript libraries such as D3.js or Highcharts to convert driving data into charts; the data display and interaction module builds a responsive user interface based on HTML5, CSS3 and JavaScript to ensure the consistency and availability of data display on different devices: the page is dynamically updated through Ajax technology, allowing users to view driving data of different time periods through operations such as sliding and clicking, and provides interactive functions such as search and filtering.

[0108] In this embodiment, the security and privacy protection module utilizes the decentralization and immutability of blockchain technology to store the hash value of the driving data to ensure that the data is not tampered with.

[0109] Before data sharing, the security and privacy protection module uses the K-Anonymity algorithm to desensitize sensitive information to ensure that the published data contains at least K indistinguishable records, thereby protecting personal privacy. The security and privacy protection module implements role-based access control, assigns different permissions to different users, and limits access to sensitive data. The security and privacy protection module uses the TLS protocol to encrypt data transmission, combined with the AES symmetric encryption algorithm and the RSA asymmetric encryption algorithm to ensure the security of data during transmission.

[0110] In this embodiment, the primary beneficial effect of the intelligent analysis and processing module on the entire system is reflected in the significant improvement of driving safety and efficiency. Through the data preprocessing module, the system can quickly extract useful information from the raw data, remove noise and outliers, and lay a solid foundation for subsequent in-depth analysis. The emotional behavior analysis module can monitor the driver's emotional state and driving behavior in real time, such as fatigue driving, distraction or improper operation, and issue warnings in time. This instant feedback helps the driver adjust his behavior and avoid potential dangerous situations. At the same time, the driver's physiological state is monitored by combining the facial recognition technology of the convolutional neural network and the long-distance heart rate detection technology. The driver's anger, panic and panic state, as well as heart rate, are detected. By using facial recognition technology based on convolutional neural networks, the facial micro-expressions in the driver's driving video obtained in real time by the on-board camera are processed to realize real-time detection of the driver's emotions, realize real-time monitoring of the driver's dangerous driving state, provide driving suggestions for the driver, and enable the system to provide early warning before an accident occurs, thereby greatly reducing the probability of an accident and improving driving safety.

[0111] In this embodiment, another important beneficial effect of the intelligent analysis and processing module is that it promotes data-driven decision support and the provision of personalized services. The data sharing and collaboration module enables the driver's anonymous data to be used for larger-scale data analysis, which not only helps researchers and vehicle manufacturers understand driving behavior patterns, but also promotes broader traffic safety research. Through these analyses, more accurate driving assistance systems can be developed to provide drivers with personalized driving advice and training. For example, the system can adjust vehicle settings such as seat position, steering wheel sensitivity, etc. according to the driver's habits and preferences to improve driving comfort and ease of operation. The intelligent analysis and processing module can also provide valuable data insights for vehicle management units. Through the analysis of a large amount of driving data, vehicle management units can better understand vehicle performance, wear and tear conditions, and maintenance needs, thereby optimizing vehicle maintenance plans, extending vehicle service life, and reducing operating costs. This data-driven decision support makes vehicle management more scientific and refined, and ultimately improves the operating efficiency and safety of the entire transportation system.

[0112] In this embodiment, the K-means algorithm can quickly identify the main user groups and behavior patterns by clustering car driving data points into a predetermined number of clusters, but it is sensitive to noise and outliers and assumes that the clusters are convex. The DBSCAN algorithm can identify clusters of any shape and exclude noise points, independent of the preset number of clusters. The combination of the two enables anomaly detection to not only identify the main user behavior patterns, but also to find abnormal patterns that K-means may ignore, such as small clusters or non-convex clusters, thereby improving the overall detection accuracy. In addition, the addition of DBSCAN makes the clustering process more flexible, unaffected by the initial center selection, reduces the risk of local optimal solutions, and enhances the robustness of clustering.

[0113] In this embodiment, the K-means and DBSCAN algorithms are combined to optimize resource utilization and computational efficiency, while improving the quality of secure sedimentation of automobile driving data. The high computational efficiency of the K-means algorithm makes it suitable for processing large-scale automobile driving data sets, and although DBSCAN has a high computational complexity, its ability to recognize complex shape clusters is unparalleled. By first using K-means for preliminary clustering and then using DBSCAN to optimize the preliminary results, the quality of clustering can be improved while ensuring computational efficiency. This optimization not only improves the efficiency of automobile driving data processing, but also ensures that the sedimented automobile driving data is more reliable and useful. High-quality automobile driving data sedimentation provides a solid foundation for subsequent automobile driving data analysis and business decision-making, and also provides a more effective and comprehensive technical means for protecting user UA information, which helps to identify and isolate potential malicious or abnormal behaviors, thereby ensuring the security of user automobile driving data.

[0114] In summary, the technical solution provided by one or more embodiments of the present disclosure not only obtains a large amount of real-time driving data, but also ensures that the real-time driving data covers vehicle characteristics, environmental characteristics and user characteristics, thereby ensuring the mineability of driving data. Through data preprocessing, useful information can be quickly extracted from the original data, noise and outliers can be removed, and a solid foundation can be laid for subsequent in-depth analysis. By analyzing the driving data, the user's emotional state and heart rate state can be determined, so that when the user's state is abnormal, a timely warning can be issued, thereby improving driving safety and road safety. The generated target driving data retains both the candidate driving data after preprocessing and the user status data after in-depth analysis, which is conducive to further analysis and data return. Clustering is performed on the target driving data to make the stored target driving data more orderly, which is convenient for viewing statistical information and later calls.

[0115] See also Figure 3 The present disclosure also provides a driving data recording device, the device comprising:

[0116] The data acquisition unit 100 is used to acquire real-time driving data using vehicle-mounted sensors and vehicle-mounted cameras, wherein the real-time driving data includes vehicle characteristic data, environment characteristic data, and user characteristic data;

[0117] A preprocessing unit 200, configured to preprocess the real-time driving data to form candidate driving data, wherein the preprocessing includes at least one of data cleaning, data compression and data fusion;

[0118] A data analysis unit 300, configured to analyze the candidate driving data and determine user state data, wherein the user state data includes emotional state data and heart rate state data;

[0119] A data fusion unit 400, configured to fuse the candidate driving data with the user state data to generate target driving data;

[0120] The data storage unit 500 is used to perform clustering processing on the target driving data and store the target driving data in a classified manner.

[0121] In one embodiment, the data analysis unit 300 includes a sentiment analysis subunit 301. The sentiment analysis subunit 301 is specifically used to extract user facial image data based on the candidate driving data; and input the user facial image data into a sentiment classification model so that the sentiment classification model determines the sentiment state data based on a facial feature point location algorithm and a local binary pattern histogram algorithm.

[0122] In one embodiment, the data analysis unit 300 includes a heart rate analysis subunit 302. The heart rate analysis subunit 302 is specifically used to extract user facial video data based on the candidate driving data; analyze facial optical change information according to the user facial video data; and calculate the heart rate state data using the facial optical change information.

[0123] In one embodiment, the data fusion unit 400 is specifically used to extract and merge features based on the candidate driving data and the user status data to generate multimodal data, wherein the multimodal data includes user expression modal data, user heart rate modal data, vehicle motion modal data, and environmental modal data; input the multimodal data into an information fusion model to generate the target driving data, wherein the target driving data includes user safety information and vehicle safety information.

[0124] In one embodiment, the data storage unit 500 is specifically used to run a first clustering algorithm for the target driving data to determine a first clustering result; determine a plurality of clustering centers based on the first clustering result; determine the clustering centers as initial seed points of a second clustering algorithm; and use the second clustering algorithm and the initial seed points to process the target driving data to determine a second clustering result.

[0125] In one embodiment, the data storage unit 500 is further used to analyze the first clustering result to determine clustering difference points; analyze the second clustering result to determine clustering noise points; and quantify the abnormal scores of the clustering difference points and the clustering noise points.

[0126] In one embodiment, the driving data recording device further includes a transmission and sharing unit 600. The transmission and sharing unit 600 is specifically used to transmit the target driving data to a data sharing platform; receive a data analysis model pushed by the data sharing platform; and transmit the target driving data to a target vehicle or a target server.

[0127] In one embodiment, the driving data recording device further includes a display and interaction unit 700. The display and interaction unit 700 is specifically used to graphically display the target driving data and receive a user's interactive instruction for the target driving data.

[0128] In one embodiment, the driving data recording device further includes a security protection unit 800. The security protection unit 800 is specifically used to encrypt the target driving data.

[0129] Each unit described in the above embodiments may be implemented by a computer chip or a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0130] For the convenience of description, the above devices are described in terms of functions and are described separately in various units. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0131] See also Figure 4 The present disclosure also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned driving data recording method is implemented.

[0132] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned driving data recording method is implemented.

[0133] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0134] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0135] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0137] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0138] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0139] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A driving data recording method, characterized in that: The method comprises: Using vehicle-mounted sensors and vehicle-mounted cameras to obtain real-time driving data, the real-time driving data including vehicle characteristic data, environment characteristic data and user characteristic data; Preprocessing the real-time driving data to form candidate driving data, wherein the preprocessing includes at least one of data cleaning, data compression and data fusion; Analyze the candidate driving data to determine user state data, wherein the user state data includes emotional state data and heart rate state data; fusing the candidate driving data with the user state data to generate target driving data; Clustering is performed on the target driving data, and the target driving data is stored in a classified manner.

2. The method according to claim 1, characterized in that: The analyzing the candidate driving data to determine the user status data includes: Extracting user facial image data based on the candidate driving data; The user facial image data is input into an emotion classification model so that the emotion classification model determines the emotion state data based on a facial feature point location algorithm and a local binary pattern histogram algorithm.

3. The method according to claim 1, characterized in that The analyzing the candidate driving data to determine the user status data includes: Extracting user facial video data based on the candidate driving data; Analyzing facial optical change information based on the user's facial video data; The heart rate status data is calculated using the facial optical change information.

4. The method according to claim 1, characterized in that: The fusing the candidate driving data and the user state data to generate target driving data includes: Based on the candidate driving data and the user state data, feature extraction and merging are performed to generate multimodal data, wherein the multimodal data includes user expression modal data, user heart rate modal data, vehicle motion modal data, and environmental modal data; The multimodal data is input into an information fusion model to generate the target driving data, where the target driving data includes user safety information and vehicle safety information.

5. The method according to claim 1, characterized in that The clustering process of the target driving data includes: Running a first clustering algorithm on the target driving data to determine a first clustering result; Determining a plurality of cluster centers according to the first clustering result; Determining the cluster center as an initial seed point of a second clustering algorithm; The target driving data is processed using the second clustering algorithm and the initial seed point to determine a second clustering result.

6. The method according to claim 5, characterized in that The clustering process of the target driving data further includes: Analyze the first clustering results to determine clustering differences; Analyze the second clustering result to determine clustering noise points; Anomaly scores of the cluster difference points and the cluster noise points are quantified.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises at least one of the following: transmitting the target driving data to a data sharing platform; Receiving the data analysis model pushed by the data sharing platform; transmitting the target driving data to a target vehicle or a target server; encrypting the target driving data; graphically displaying the target driving data; Receive a user's interactive instruction for the target driving data.

8. A driving data recording device, characterized in that: The device comprises: A data acquisition unit, used to acquire real-time driving data using vehicle-mounted sensors and vehicle-mounted cameras, wherein the real-time driving data includes vehicle characteristic data, environment characteristic data, and user characteristic data; A preprocessing unit, configured to preprocess the real-time driving data to form candidate driving data, wherein the preprocessing includes at least one of data cleaning, data compression and data fusion; a data analysis unit, configured to analyze the candidate driving data and determine user state data, wherein the user state data includes emotional state data and heart rate state data; a data fusion unit, configured to fuse the candidate driving data with the user state data to generate target driving data; The data storage unit is used to perform clustering processing on the target driving data and store the target driving data in a classified manner.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images