Driving behavior recognition methods, devices, storage media and electronic equipment

Abstract

This disclosure relates to a driving behavior recognition method, device, storage medium, and electronic device, belonging to the field of data processing technology. The method includes: acquiring motion data of the driver's hands; processing first feature data based on a first classification rule to obtain a first category result; the first feature data is feature data corresponding to a first feature type obtained by extracting features from the motion data; processing second feature data based on a second classification rule corresponding to the first category result to obtain the driver's driving behavior; the second feature data is feature data corresponding to a second feature type obtained by extracting features from the motion data. This method can improve the accuracy of driving behavior recognition.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to a driving behavior recognition method, device, storage medium, and electronic device. Background Technology

[0002] During driving, there are many driving behaviors, some of which pose a threat to road safety. Therefore, effectively identifying these bad driving behaviors and reminding drivers can improve driving safety and reduce traffic accidents. Summary of the Invention

[0003] The purpose of this disclosure is to provide a driving behavior recognition method, device, storage medium, and electronic device to improve the accuracy of driving behavior recognition.

[0004] To achieve the above objectives, in a first aspect, this disclosure provides a driving behavior recognition method, comprising: Acquire driver's hand movement data; The first feature data is processed based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from the motion data. The second feature data is processed based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from the motion data.

[0005] Optionally, the first classification rule includes multiple first sub-classification models, and the training method for the first sub-classification models includes: Obtain a first training dataset, which includes multiple training samples. Each training sample includes sample feature data corresponding to the first feature type and a first category label. The first training dataset is used to perform supervised training on multiple different types of initial neural network models to obtain the multiple first sub-classification models. Obtain the prediction error dataset of the first training dataset relative to each of the first sub-classification models, wherein the sample feature data in the prediction error dataset are the training samples that were misclassified by the first sub-classification models.

[0006] Optionally, the step of processing the first feature data based on the first classification rule to obtain the first category result includes: Based on the correlation between the first feature data and each of the sample feature data, target sample feature data associated with the first feature data is determined; Based on the subordinate relationship between the target sample feature data and each of the prediction error datasets, the prediction classification result of each first sub-classification model is determined; Based on each of the predicted classification results, a target classification rule is determined from the first classification rule; The first feature data is processed based on the target classification rule to obtain the first category result.

[0007] Optionally, the predicted classification result includes a first result, the first result indicating that the target sample feature data does not belong to the prediction error dataset of at least one first sub-classification model, and the step of determining the target classification rule from the first classification rules based on each of the predicted classification results includes: Determine each candidate first sub-classification model corresponding to the first result; Obtain the classification accuracy of each candidate first subclassification model on the test dataset; The candidate first sub-classification model with the highest classification accuracy is determined as the target classification rule.

[0008] Optionally, the first classification rule further includes a second sub-classification rule, the predicted classification result includes a second result, the second result representing the prediction error dataset of the target sample feature data belonging to each of the first sub-classification models, and the step of determining the target classification rule from the first classification rule based on each of the predicted classification results includes: If all predicted classification results are the second result, the second sub-classification rule is determined as the target classification rule.

[0009] The step of processing the first feature data based on the target classification rule to obtain the first category result includes: When the target classification rule is the second sub-classification rule, the first category label corresponding to the target sample feature data is determined as the first category result.

[0010] Optionally, the first category result includes a first subclass and a second subclass, where the amplitude of the driving behavior in the first subclass is greater than the amplitude of the driving behavior in the second subclass. The second feature includes the frequency domain feature and mean feature corresponding to the motion data. The step of processing the second feature data based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior includes: If the result of the first category is the first subclass, the frequency domain features are processed based on a pre-trained decision tree classifier to obtain the driver's driving behavior. If the result of the first category is the second subclass, the mean feature is processed based on a pre-trained supervised classifier neural network to obtain the driver's driving behavior.

[0011] Optionally, the motion data includes sub-motion data corresponding to different sampling times within a preset time period. The sub-motion data corresponding to one sampling time includes the 3-axis acceleration and 3-axis angular velocity of the driver's left hand, and the 3-axis acceleration and 3-axis angular velocity of the right hand. The first feature type includes the maximum absolute value of the acceleration of each axis sampled within the preset time period, and the variance of the acceleration of each axis sampled within the preset time period. The frequency domain feature corresponding to the motion data includes the energy in the discrete Fourier transform spectrum of the acceleration of each axis sampled within the preset time period and a preset number of maximum peak values. The mean feature corresponding to the motion data includes the mean of the acceleration of each axis sampled within the preset time period, and the mean of the angular velocity of each axis sampled within the preset time period. The driving behaviors mentioned include driving while talking on the phone, driving while eating, driving while smoking, driving with your hands on the window, and driving with both hands normally.

[0012] Optionally, acquiring the driver's hand movement data includes: The movement data of the driver's hands are obtained through intelligent sensing devices worn on the driver's left and right hands; The method further includes: If the driver's driving behavior is determined to be bad driving behavior, the intelligent sensing device will output a driving warning message in the form of vibration or voice.

[0013] Secondly, embodiments of this disclosure provide a driving behavior recognition device, comprising: The first acquisition module is used to acquire the driver's hand movement data; The first processing module is used to process the first feature data based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from the motion data. The second processing module is used to process the second feature data based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from the motion data.

[0014] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0015] Fourthly, embodiments of this disclosure provide an electronic device, including: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspects.

[0016] The above technical solution first acquires the driver's hand movement data. Then, based on a first classification rule, the first feature data corresponding to the first feature type is processed to obtain a first category result. Next, based on a second classification rule corresponding to the first category result, the second feature data corresponding to the second feature type is processed to obtain the driver's driving behavior. Since the second classification rule requires the processing result of the first classification rule, it is equivalent to using two different levels of classification rules. Furthermore, because the first and second classification rules respectively use the first and second feature data, each level of classification can selectively choose the most suitable features, thereby effectively reducing the complexity of the decision boundary and improving the accuracy of driving behavior recognition.

[0017] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a driving behavior recognition method according to an exemplary embodiment of this disclosure.

[0019] Figure 2 This is a flowchart illustrating another driving behavior recognition method as shown in an exemplary embodiment of this disclosure.

[0020] Figure 3 This is a schematic diagram illustrating the classification result of a first sub-classification model as shown in an exemplary embodiment of this disclosure.

[0021] Figure 4 This is a block diagram of a driving behavior recognition device shown in an exemplary embodiment of the present disclosure.

[0022] Figure 5 This is a block diagram of an electronic device shown in an exemplary embodiment of the present disclosure. Detailed Implementation

[0023] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0024] Figure 1 This is a flowchart illustrating a driving behavior recognition method according to an exemplary embodiment of the present disclosure. The driving behavior recognition method can be executed by an electronic device, specifically a driving behavior recognition device. This device can be implemented in software and / or hardware and configured within an electronic device, which can be an in-vehicle terminal in a vehicle, or an external device such as a mobile phone, computer, or wearable device. (Refer to...) Figure 1 The driving behavior recognition method includes the following steps: S101, acquire the driver's hand movement data.

[0025] In some implementations, the motion data may include sub-motion data corresponding to different sampling times within a preset time period. The sub-motion data corresponding to each sampling time includes the 3-axis acceleration and 3-axis angular velocity of the driver's left hand, and the 3-axis acceleration and 3-axis angular velocity of the right hand.

[0026] In some implementations, the driver can collect 3-axis acceleration and 3-axis angular velocity data from the left hand and the right hand using smart sensing devices worn on both hands, for a total of 12 axes of data. Optionally, the smart sensing devices can be smart bracelets, smart wristbands, smartwatches, etc., equipped with accelerometers and gyroscopes.

[0027] S102, the first feature data is processed based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from motion data.

[0028] In this embodiment of the disclosure, the first classification rule refers to a classification rule used to process the first feature data to obtain a first category result. Optionally, the first classification rule may include a first sub-classification rule and a second sub-classification rule, wherein the first sub-classification rule may be a classification rule based on a neural network model, and the second sub-classification rule may be a classification rule based on a preset correspondence.

[0029] In some implementations, the first classification rule may include multiple first sub-classification rules trained on a first training dataset. Here, the first sub-classification rule may also be referred to as the first sub-classification model. The first training dataset includes multiple sample feature data corresponding to the first feature type and a first category label.

[0030] Here, the type of the first category label can be the same as the type of the first category result. For example, the first category label can include a first subclass and a second subclass, where the magnitude of the driving behavior in the first subclass is greater than the magnitude of the driving behavior in the second subclass.

[0031] In some implementations, the training method for the first sub-classification model includes: Obtain a first training dataset, which includes multiple training samples. Each training sample includes sample feature data corresponding to the first feature type and a first category label. Supervised training is performed on multiple initial neural network models of different types using the first training dataset to obtain multiple first sub-classification models. The prediction error dataset of the first training dataset relative to each first sub-classification model is obtained. The sample feature data in the prediction error dataset are the training samples that the first sub-classification models misclassified.

[0032] In this embodiment of the disclosure, supervised training can be performed on multiple different types of initial neural network models using the same set of data (i.e., the first training dataset) to obtain multiple different types of first sub-classification models.

[0033] For example, the BP neural network, support vector machine, and Naive Bayes classifiers to be trained can be trained separately using the first training dataset to obtain the trained BP neural network, support vector machine, and Naive Bayes classifiers. Optionally, the top-performing trained classifiers in the first training dataset can be selected as the first sub-classification model.

[0034] In addition, it is possible to obtain the training samples that are classified incorrectly when a certain first sub-classification model classifies the training samples in the first training dataset, thus obtaining the prediction error dataset relative to the first sub-classification model.

[0035] The process of supervising the training of the initial neural network model using a training dataset can be found in relevant technologies and will not be elaborated upon in this embodiment. For example, the initial neural network model can be used to predict sample feature data to obtain a predicted classification result. Then, based on the difference between the predicted classification result and the first category label, the parameters of the initial neural network model are updated to achieve the process of supervising the training of the initial neural network model.

[0036] In some implementations, the first category of results includes a first subclass and a second subclass, where the magnitude of the driving behavior in the first subclass is greater than the magnitude of the driving behavior in the second subclass.

[0037] For example, taking driving behaviors including driving while talking on the phone, driving while eating, driving while smoking, driving while holding onto the window, and driving with both hands normally as examples, the first subclass can include behaviors with a large range of motion, such as driving while eating and driving while smoking, while the second subclass can include behaviors with a small range of motion, such as driving while talking on the phone, driving while holding onto the window, and driving with both hands normally.

[0038] Considering that behaviors with large amplitude of motion have large variance and maximum absolute value of acceleration, and behaviors with small amplitude of motion have small variance and maximum absolute value of acceleration, in some implementations, the first feature type includes the maximum absolute value of acceleration of each axis sampled within a preset time period, and the variance of acceleration of each axis sampled within the preset time period.

[0039] S103, based on the second classification rule corresponding to the first category result, the second feature data is processed to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from motion data.

[0040] In this embodiment of the disclosure, the second classification rule refers to the classification rule used to process the second feature data to obtain the driver's driving behavior.

[0041] In some implementations, different first category results correspond to different second classification rules, and each second classification rule has corresponding second feature data.

[0042] In some implementations, the second feature data includes frequency domain features and mean features corresponding to the motion data.

[0043] Furthermore, in some implementations, the second feature type includes the energy in the discrete Fourier transform spectrum of acceleration sampled within a preset time period and a preset number of maximum peak values. In this case, the second classification rule is a second classification model trained on a second training dataset. The second classification model is a decision tree classifier. The second training dataset includes multiple sample feature data corresponding to the second feature type and driving behavior labels.

[0044] In some implementations, the second feature type includes the mean of acceleration of each axis sampled within a preset time period and the mean of angular velocity of each axis sampled within a preset time period. In this case, the second classification rule is a third classification model trained on a third training dataset. The third classification model is a supervised classifier neural network. The third training dataset includes multiple sample feature data corresponding to the second feature type and driving behavior labels.

[0045] Using the above method, the driver's hand movement data is first acquired. Then, the first feature data corresponding to the first feature type can be processed based on the first classification rule to obtain the first category result. Next, the second feature data corresponding to the second feature type can be processed based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. Since the second classification rule requires the processing result of the first classification rule, it is equivalent to using two different levels of classification rules. Furthermore, because the first and second classification rules respectively use the first and second feature data, each level of classification can selectively choose the most suitable features, thereby effectively reducing the complexity of the decision boundary and improving the accuracy of driving behavior recognition.

[0046] As can be seen from the foregoing, in some implementations, the first classification rule includes multiple first sub-classification models trained on a first training dataset. The first training dataset includes multiple sample feature data corresponding to the first feature type and a first category label. In this case, processing the first feature data based on the first classification rule to obtain the first category result includes: Based on the correlation between the first feature data and the feature data of each sample, the target sample feature data associated with the first feature data is determined. Based on the relationship between the target sample feature data and each prediction error dataset, determine the prediction classification result of each first sub-classification model; Based on the predicted classification results, the target classification rule is determined from the first classification rule; The first feature data is processed based on the target classification rules to obtain the first category result.

[0047] In this embodiment of the disclosure, both the sample feature data and the first feature data are data under the first feature type. Therefore, in some implementations, the correlation between the first feature data and each sample feature data can be calculated by the Pearson correlation coefficient. Furthermore, the sample feature data with the highest correlation can be determined as the target sample feature data associated with the first feature data.

[0048] In some implementations, the predicted classification results include two types: a first result and a second result. The first result indicates that the target sample feature data does not belong to the prediction error dataset of at least one first sub-classification model. That is, the first category result of the target sample feature data processed by the first sub-classification model is the same as the first category label corresponding to the target sample feature data, meaning that the first sub-classification model can accurately predict the target sample feature data. The second result indicates that the target sample feature data belongs to the prediction error dataset of each first sub-classification model. That is, the result of the target sample feature data processed by the first sub-classification model is different from the first category label corresponding to the target sample feature data, meaning that the first sub-classification model cannot accurately predict the target sample feature data.

[0049] In some implementations, after training multiple first sub-classification models using the first training dataset, the target sample feature data can be input again into the trained first sub-classification model for prediction processing to obtain the first category result corresponding to the target sample feature data. Then, the first category result is compared with the first category label carried by the target sample feature data to obtain the predicted classification result.

[0050] As can be seen from the foregoing, in some implementations, the predicted classification result includes a first result, which indicates that the target sample feature data does not belong to the prediction error dataset of at least one first sub-classification model. In this case, based on each predicted classification result, the target classification rule is determined from the first classification rules, including the following steps: Determine the candidate first subclassification models corresponding to the first result; Obtain the classification accuracy of each candidate first subclassification model on the test dataset; The candidate first sub-classification model with the highest classification accuracy is determined as the target classification rule.

[0051] In this embodiment of the disclosure, after obtaining the predicted classification results of the target sample feature data in each of the first sub-classification models, each candidate first sub-classification model corresponding to the first result can be determined. That is, the target sample feature data does not belong to the prediction error dataset corresponding to at least one candidate first sub-classification model, thereby determining the candidate first sub-classification model that can accurately classify the target sample feature data. Then, in order to ensure the accuracy of processing the first feature data as much as possible, the classification accuracy of each candidate first sub-classification model in the test dataset can be obtained, and then the candidate first sub-classification model with the highest classification accuracy is determined as the target classification rule.

[0052] As can be seen from the foregoing, in some implementations, the first classification rule further includes a second sub-classification rule, and the predicted classification result includes a second result. The second result represents the prediction error dataset of the target sample feature data belonging to each of the first sub-classification models. In this case, the target classification rule is determined from the first classification rule based on each predicted classification result, including: If all predicted classification results are the second result, the second sub-classification rule is determined as the target classification rule.

[0053] In this embodiment of the disclosure, if the predicted classification results of the target sample feature data in each of the first sub-classification models are all the second results, that is, no first sub-classification model can accurately classify the target sample feature data, then using the first sub-classification model for processing will result in an inaccurate first classification result. Therefore, it is advisable to determine the second sub-classification rule as the target classification rule, that is, to use the second sub-classification rule to process the first feature data.

[0054] In some implementations, when the target classification rule is a second sub-classification rule, processing the first feature data based on the target classification rule to obtain the first category result may include the following steps: When the target classification rule is the second sub-classification rule, the first category label corresponding to the target sample feature data is determined as the first category result.

[0055] In this embodiment of the disclosure, considering that the first feature data and the target sample feature data are strongly correlated, and their corresponding first category results are likely to be the same, in the absence of a first sub-classification model that can accurately classify the target sample feature data, the first category label corresponding to the target sample feature data can be directly determined as the first category result.

[0056] Based on the foregoing, in some implementations, the first category result includes a first subclass and a second subclass, where the amplitude of the driving behavior in the first subclass is greater than that in the second subclass. The second feature includes the frequency domain feature and mean feature corresponding to the motion data. In this case, the second feature data is processed based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior, including: If the first category result is the first subclass, the frequency domain features are processed based on a pre-trained decision tree classifier to obtain the driver's driving behavior; if the first category result is the second subclass, the mean features are processed based on a pre-trained supervised classifier neural network to obtain the driver's driving behavior.

[0057] In this embodiment, for behaviors with large amplitude movements, i.e., driving behaviors initially classified into the first subcategory, such as eating while driving and smoking while driving, considering the frequency difference in acceleration between eating while driving and smoking while driving, the frequency domain features corresponding to the motion data can effectively distinguish these driving behaviors. Simultaneously, the multi-level decision-making of the decision tree classifier can effectively handle high-dimensional features, improve the accuracy of driving behavior recognition, and has low computational complexity and fast recognition speed.

[0058] For driving behaviors with smaller amplitude of motion, which are initially classified as driving behaviors in the second subcategory, such as driving while talking on the phone, driving with hands on the window, and driving with both hands normally, considering the significant differences in acceleration or angular velocity values ​​between these behaviors, the mean features corresponding to the motion data can effectively distinguish these driving behaviors. Furthermore, employing a supervised classifier neural network can further improve the accuracy of driving behavior recognition.

[0059] The training methods for the pre-trained decision tree classifier and the pre-trained supervised classifier neural network can be found in relevant technologies and will not be elaborated here. It should be noted that the pre-trained decision tree classifier can be trained using the frequency domain features corresponding to the sample motion data and the driving behavior labels, while the pre-trained supervised classifier neural network can be trained using the mean features corresponding to the sample motion data and the driving behavior labels.

[0060] In some implementations, the motion data includes sub-motion data corresponding to different sampling times within a preset time period. The sub-motion data corresponding to one sampling time includes the 3-axis acceleration and 3-axis angular velocity of the driver's left hand, and the 3-axis acceleration and 3-axis angular velocity of the driver's right hand. The first feature type includes the maximum absolute value of the acceleration of each axis sampled within the preset time period, and the variance of the acceleration of each axis sampled within the preset time period. The frequency domain feature corresponding to the motion data includes the energy in the discrete Fourier transform spectrum of the acceleration of each axis sampled within the preset time period and a preset number of maximum peak values. The mean feature corresponding to the motion data includes the mean of the acceleration of each axis sampled within the preset time period, and the mean of the angular velocity of each axis sampled within the preset time period. The driving behavior includes driving while talking on the phone, driving while eating, driving while smoking, driving with the window open, and driving with both hands normally.

[0061] As can be seen from the foregoing, in some implementations, the driver's hand movement data can be obtained through smart sensing devices worn on the driver's left and right hands. In this case, the method of this disclosure embodiment may further include the following steps: If the driver's driving behavior is determined to be unsafe, a driving warning message will be output via vibration or voice through intelligent sensing devices.

[0062] In this embodiment of the disclosure, if it is determined that the driver's driving behavior is bad driving behavior, a driving warning message can be output in the form of vibration or voice through a smart sensing device to remind the user to correct the bad driving behavior.

[0063] Figure 2 This is a flowchart illustrating another driving behavior recognition method according to an exemplary embodiment of the present disclosure. This driving behavior recognition method can be executed by an electronic device, specifically by a driving behavior recognition device. This device can be implemented in software and / or hardware and configured within an electronic device, which can be an in-vehicle terminal in a vehicle, or an external device such as a mobile phone, computer, or wearable device. (Refer to...) Figure 2 The driving behavior recognition method includes the following steps: S201, acquire the driver's hand movement data.

[0064] S202, extract features from the motion data to obtain the first feature data.

[0065] S203, based on the correlation between the first feature data and the feature data of each sample used to train the first sub-classification model, determine the target sample feature data associated with the first feature data.

[0066] S204, obtain the predicted classification results of the target sample feature data in each first sub-classification model.

[0067] S205, if a first result exists in the predicted classification results, determine each candidate first sub-classification model corresponding to the first result.

[0068] S206, obtain the classification accuracy of each candidate first subclassification model on the test dataset.

[0069] S207, the candidate first sub-classification model with the highest classification accuracy is determined as the target classification rule.

[0070] S208, if all predicted classification results are the second result, the second sub-classification rule is determined as the target classification rule.

[0071] S209, Process the first feature data based on the target classification rule to obtain the first category result.

[0072] S210, extract features from the motion data to obtain the second feature corresponding to the first category result.

[0073] S211, based on the second classification rule corresponding to the first category result, the extracted second feature is processed to obtain the driver's driving behavior.

[0074] The detailed description of steps S201-S211 can be found in the foregoing embodiments, and will not be repeated here.

[0075] The driving behavior recognition method of this disclosure will now be described with reference to an example.

[0076] This example uses acceleration and gyroscope sensor data to identify five types of vehicle driving behavior, including driving with both hands on the ground, driving with the window open, driving while talking on the phone, driving while smoking, and driving while eating.

[0077] The above five types of vehicle driving behaviors are divided into two subcategories: the first subcategory (hereinafter referred to as Category A) is defined as dynamic behaviors with a large range of motion, including driving while smoking and driving while eating; the second subcategory (hereinafter referred to as Category B) is defined as dynamic behaviors with a small range of motion, including driving with both hands normally, driving while holding the window, and driving while talking on the phone.

[0078] First, the model training process before starting driving behavior recognition is as follows: The driver wears miniature sensors containing three-axis accelerometers and three-axis gyroscopes on both hands to... The second window length serves as the action window, collecting acceleration and angular velocity signals generated during the five driving behaviors described above. Each sampling yields a total of 12 axes of data (the left hand includes 3-axis acceleration and 3-axis angular velocity, and the right hand includes 3-axis acceleration and 3-axis angular velocity). Data sampled within a one-second window is used as data on the driver's hand movements.

[0079] For each type of driving behavior, multiple sets of data are collected and stored. A 7th-order moving average filtering algorithm is used to perform low-pass filtering on the collected motion data. For a given set of data, the following steps are taken: A sequence of data ,in, This represents the data from 12 axes obtained in a single sampling. The formula for the moving average filtering algorithm, representing the number of samples, is as follows:

[0080] The filtered data sequence is as follows: .

[0081] For each group The maximum absolute value of acceleration along each axis and the variance of acceleration along each axis are extracted to obtain sample feature data. Simultaneously, this data is combined with the first category label to which the driving behavior belongs to obtain the first training dataset for training the first sub-classification model. and through the first training dataset The first sub-classification model was trained as follows: A supervised classifier is used on the first training dataset. Training can be performed using, for example, BP neural networks, support vector machines, Naive Bayes classifiers, etc. After the above training steps are completed, find the first training dataset. The two classifiers with the highest recognition accuracy are selected as the first sub-classification model, and are respectively set as _____. and ( higher accuracy ); Get and The classification results of the samples in the first training dataset will be Samples that are misclassified are denoted as ,Will Samples that are misclassified are denoted as ,Will and Samples with average misclassification are denoted as (Not shown in the figure), schematic diagram as follows Figure 3 As shown; It should be noted that the classification results obtained here can be obtained from [the relevant documentation / source]. and Then, the samples from the first training dataset are input again. and It can be obtained through processing, or it can be directly based on the first training dataset from the previous training step. The data obtained during training.

[0082] For each group The energy and five maximum peak values ​​of the discrete Fourier transform spectrum of acceleration for each axis are extracted from it. At the same time, combined with driving behavior labels, a second training dataset is obtained for training the second classification model. The decision tree classifier is trained using the second training dataset to obtain the second classification model in the second classification rule. For each group The mean acceleration of each axis and the mean angular velocity of each axis are extracted from the data. At the same time, the driving behavior labels are combined to obtain the third training dataset for training the third classification model. The supervised classifier is trained using the third training dataset to obtain the third classification model in the second classification rule. After completing the above model training process, the driving behavior recognition process can be performed. The specific recognition process is as follows: by The second window length is used as the action window to obtain... Acceleration and angular velocity signals are collected at each sampling time point per second. Each sampling yields a total of 12 axes of data (left-handed data includes 3-axis acceleration and 3-axis angular velocity, right-handed data includes 3-axis acceleration and 3-axis angular velocity). Data sampled within a one-second window is used as data on the driver's hand movements.

[0083] The collected motion data were low-pass filtered using a 7th-order moving average filtering algorithm to obtain the driver's hand motion data; The filtered data sequence is as follows: .

[0084] from Extract the maximum absolute value of the acceleration along each axis and the variance of the acceleration along each axis to obtain the first feature data; The correlation between the first feature data and the feature data of each group of samples is calculated using the Pearson correlation coefficient, and the feature data of the sample with the highest correlation is determined as the target sample feature data. ,in, ; if Then directly use the classification with the highest accuracy. The classifier classifies it and obtains the first category result; if ,and ,illustrate The classifier may misclassify it, while The classifier performs well in recognizing it, therefore we choose it. The classifier classifies it and obtains the first category result; if ,and (Right now ),illustrate and The classifier may misclassify it, therefore a second sub-classification rule is used for classification, that is... The corresponding first category label is used as the first category result; Based on the first category result in the above steps, which is either category A or category B, select the corresponding second classification rule to further classify it. Regarding the two behaviors in class A, from In the process, the energy and five maximum peaks in the discrete Fourier transform spectrum of acceleration for each axis are extracted, totaling 36 dimensions (6 axes * 6 features), which are used for recognition by the second classification model. Regarding the three behaviors in category B, from In the process, the mean value of acceleration and the mean value of angular velocity of each axis are extracted (12-dimensional, 12 axes * 1 feature) for recognition by the third classification model; If the behavior is ultimately identified as unsafe driving, such as driving while talking on the phone, eating while driving, smoking while driving, or driving with the window open, the driver will be alerted via vibration or voice prompts. The relevant data and driving behavior type will be uploaded to the relevant management platform for further processing.

[0085] The aforementioned driving behavior recognition method employs a hierarchical structure, offering excellent flexibility and scalability. Each layer can selectively choose the most suitable features, effectively reducing the complexity of the decision boundary and improving recognition accuracy. Furthermore, the top-level system (first classification rule) design utilizes a combination of unsupervised (second sub-classification rule) and supervised training and recognition methods, providing accurate classification results for the bottom-level system (second classification rule).

[0086] Based on the same inventive concept, this disclosure provides a driving behavior recognition device. Figure 4 This is a block diagram of a driving behavior recognition device 400 shown in an exemplary embodiment of this disclosure, with reference to... Figure 4 The driving behavior recognition device 400 includes: The first acquisition module 401 is used to acquire the movement data of the driver's hands; The first processing module 402 is used to process the first feature data based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from the motion data. The second processing module 403 is used to process the second feature data based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from the motion data.

[0087] In some implementations, the first classification rule includes multiple first sub-classification models, and the driving behavior recognition device 400 includes: The second acquisition module is used to acquire the first training dataset, which includes multiple training samples. Each training sample includes sample feature data corresponding to the first feature type and a first category label. The training module is used to perform supervised training on multiple initial neural network models of different types using the first training dataset to obtain the multiple first sub-classification models. The third acquisition module is used to acquire the prediction error dataset of the first training dataset relative to each of the first sub-classification models, wherein the sample feature data in the prediction error dataset are the training samples that were classified incorrectly by the first sub-classification models.

[0088] In some embodiments, the first processing module 402 includes: The first determining submodule is used to determine target sample feature data associated with the first feature data based on the correlation between the first feature data and each of the sample feature data; The prediction classification result determination submodule is used to determine the prediction classification result of each of the first sub-classification models based on the subordinate relationship between the target sample feature data and each of the prediction error datasets; The second determining submodule is used to determine the target classification rule from the first classification rule based on each of the predicted classification results; The first processing submodule is used to process the first feature data based on the target classification rule to obtain the first category result.

[0089] In some implementations, the predicted classification result includes a first result, which characterizes that the target sample feature data does not belong to the prediction error dataset of at least one first sub-classification model, and the second determining sub-module includes: The first determining unit is used to determine each candidate first sub-classification model corresponding to the first result; The acquisition unit is used to obtain the classification accuracy of each candidate first sub-classification model on the test dataset; The second determining unit is used to determine the candidate first sub-classification model with the highest classification accuracy as the target classification rule.

[0090] In some implementations, the first classification rule further includes a second sub-classification rule, the predicted classification result includes a second result, the second result characterizing the target sample feature data belonging to the prediction error dataset of each of the first sub-classification models, and the second determining sub-module includes: The third determining unit is used to determine the second sub-classification rule as the target classification rule when all predicted classification results are the second result.

[0091] In some implementations, the first processing submodule includes: The fourth determining unit is used to determine the first category label corresponding to the target sample feature data as the first category result when the target classification rule is the second sub-classification rule.

[0092] In some implementations, the first category result includes a first subclass and a second subclass, wherein the amplitude of the driving behavior in the first subclass is greater than the amplitude of the driving behavior in the second subclass, the second feature includes the frequency domain feature and the mean feature corresponding to the motion data, and the second processing module 403 includes: The second processing submodule is used to process the frequency domain features based on a pre-trained decision tree classifier when the first category result is the first subclass, in order to obtain the driver's driving behavior. The third processing submodule is used to process the mean features based on a pre-trained supervised classifier neural network when the first category result is the second subclass, in order to obtain the driver's driving behavior.

[0093] In some implementations, the motion data includes sub-motion data corresponding to different sampling times within a preset time period. The sub-motion data corresponding to one sampling time includes the 3-axis acceleration and 3-axis angular velocity of the driver's left hand, and the 3-axis acceleration and 3-axis angular velocity of the driver's right hand. The first feature type includes the maximum absolute value of each axis acceleration sampled within the preset time period, and the variance of each axis acceleration sampled within the preset time period. The frequency domain feature corresponding to the motion data includes the energy and a preset number of maximum peak values ​​in the discrete Fourier transform spectrum of each axis acceleration sampled within the preset time period. The mean feature corresponding to the motion data includes the mean of each axis acceleration sampled within the preset time period, and the mean of each axis angular velocity sampled within the preset time period. The driving behavior includes driving while talking on the phone, driving while eating, driving while smoking, driving with the driver's hand on the window, and driving with both hands normally.

[0094] In some implementations, the first acquisition module 401 includes: The second acquisition submodule is used to acquire the motion data of the driver's hands through the intelligent sensing devices worn on the driver's left and right hands; The driving behavior recognition device 400 also includes: The output module is used to output driving warning information in the form of vibration or voice through the intelligent sensing device when it is determined that the driver's driving behavior is bad driving behavior.

[0095] Regarding the driving behavior recognition device 400 in the above embodiments, the specific methods by which each module performs its operation have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0096] Based on the same inventive concept, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the driving behavior recognition method described in any embodiment of this disclosure.

[0097] Based on the same inventive concept, this disclosure also provides an electronic device, including: A memory on which computer programs are stored; A processor is configured to execute the computer program in the memory to implement the steps of the driving behavior recognition method described in any embodiment of the present disclosure.

[0098] Figure 5 This is a block diagram illustrating an electronic device 500 according to an exemplary embodiment. For example... Figure 5 As shown, the electronic device 500 may include a processor 501 and a memory 502. The electronic device 500 may also include one or more of a multimedia component 503, an input / output (I / O) interface 504, and a communication component 505.

[0099] The processor 501 controls the overall operation of the electronic device 500 to complete all or part of the steps in the driving behavior recognition method described above. The memory 502 stores various types of data to support the operation of the electronic device 500. This data may include, for example, instructions for any application or method operating on the electronic device 500, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 503 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 502 or transmitted via communication component 505. The audio component also includes at least one speaker for outputting audio signals. I / O interface 504 provides an interface between processor 501 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof; therefore, the corresponding communication component 505 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0100] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the driving behavior recognition method described above.

[0101] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the driving behavior recognition method described above. For example, the computer-readable storage medium may be the memory 502 including the program instructions described above, which may be executed by the processor 501 of the electronic device 500 to complete the driving behavior recognition method described above.

[0102] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the driving behavior recognition method described above when executed by the programmable device.

[0103] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0104] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0105] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A driving behavior recognition method, characterized in that, include: Acquire driver's hand movement data; The first feature data is processed based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from the motion data. The second feature data is processed based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from the motion data. The first category result includes a first subclass and a second subclass, where the amplitude of the driving behavior in the first subclass is greater than the amplitude of the driving behavior in the second subclass. The second feature includes the frequency domain feature and mean feature corresponding to the motion data. The process of processing the second feature data based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior includes: If the result of the first category is the first subclass, the frequency domain features are processed based on a pre-trained decision tree classifier to obtain the driver's driving behavior; If the result of the first category is the second subclass, the mean feature is processed based on a pre-trained supervised classifier neural network to obtain the driver's driving behavior.

2. The method according to claim 1, characterized in that, The first classification rule includes multiple first sub-classification models, and the training methods for the first sub-classification models include: Obtain a first training dataset, which includes multiple training samples. Each training sample includes sample feature data corresponding to the first feature type and a first category label. The first training dataset is used to perform supervised training on multiple different types of initial neural network models to obtain the multiple first sub-classification models. Obtain the prediction error dataset of the first training dataset relative to each of the first sub-classification models, wherein the sample feature data in the prediction error dataset are the training samples that were misclassified by the first sub-classification models.

3. The method according to claim 2, characterized in that, The process of processing the first feature data based on the first classification rule to obtain the first category result includes: Based on the correlation between the first feature data and each of the sample feature data, target sample feature data associated with the first feature data is determined. Based on the subordinate relationship between the target sample feature data and each of the prediction error datasets, the prediction classification result of each first sub-classification model is determined; Based on each of the predicted classification results, a target classification rule is determined from the first classification rule; The first feature data is processed based on the target classification rule to obtain the first category result.

4. The method according to claim 3, characterized in that, The predicted classification result includes a first result, which characterizes that the target sample feature data does not belong to the prediction error dataset of at least one first sub-classification model. The step of determining the target classification rule from the first classification rule based on each of the predicted classification results includes: Determine each candidate first sub-classification model corresponding to the first result; Obtain the classification accuracy of each candidate first subclassification model on the test dataset; The candidate first sub-classification model with the highest classification accuracy is determined as the target classification rule.

5. The method according to claim 3, characterized in that, The first classification rule further includes a second sub-classification rule, and the predicted classification result includes a second result, which characterizes the prediction error dataset of the target sample feature data belonging to each of the first sub-classification models. The step of determining the target classification rule from the first classification rule based on each of the predicted classification results includes: If all predicted classification results are the second result, the second sub-classification rule is determined as the target classification rule; The step of processing the first feature data based on the target classification rule to obtain the first category result includes: When the target classification rule is the second sub-classification rule, the first category label corresponding to the target sample feature data is determined as the first category result.

6. The method according to claim 1, characterized in that, The motion data includes sub-motion data corresponding to different sampling times within a preset time period. The sub-motion data corresponding to one sampling time includes the 3-axis acceleration and 3-axis angular velocity of the driver's left hand, and the 3-axis acceleration and 3-axis angular velocity of the right hand. The first feature type includes the maximum absolute value of the acceleration of each axis sampled within the preset time period, and the variance of the acceleration of each axis sampled within the preset time period. The frequency domain feature corresponding to the motion data includes the energy in the discrete Fourier transform spectrum of the acceleration of each axis sampled within the preset time period and a preset number of maximum peak values. The mean feature corresponding to the motion data includes the mean of the acceleration of each axis sampled within the preset time period, and the mean of the angular velocity of each axis sampled within the preset time period. The driving behaviors mentioned include driving while talking on the phone, driving while eating, driving while smoking, driving with your hands on the window, and driving with both hands normally.

7. A driving behavior recognition device, characterized in that, The device includes: The first acquisition module is used to acquire the driver's hand movement data; The first processing module is used to process the first feature data based on the first classification rule to obtain the first category result. The first feature data is the feature data corresponding to the first feature type obtained by extracting features from the motion data. The second processing module is used to process the second feature data based on the second classification rule corresponding to the first category result to obtain the driver's driving behavior. The second feature data is the feature data corresponding to the second feature type obtained by extracting features from the motion data. The first category result includes a first subclass and a second subclass, wherein the amplitude of the driving behavior in the first subclass is greater than the amplitude of the driving behavior in the second subclass. The second feature includes the frequency domain feature and the mean feature corresponding to the motion data. The second processing module is used to process the frequency domain feature based on a pre-trained decision tree classifier to obtain the driver's driving behavior when the first category result is the first subclass; and to process the mean feature based on a pre-trained supervised classifier neural network to obtain the driver's driving behavior when the first category result is the second subclass.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images