A dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm

Abstract

The present application relates to the technical field of automobile intelligent cockpit driver behavior detection, and particularly relates to a dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm. The present application firstly photographs a large amount of training data sets, uses an OODCI algorithm, and uses the calibrated data to train the model, so as to calibrate a large amount of unlabeled data. In this process, the return value is not only the label but also the credibility of the label, and the data with higher credibility is selected to expand the training data set, and the sample is further enhanced and expanded, so that a large amount of data required for training is obtained. Then, the combination of the convolutional neural network and the genetic algorithm greatly improves the speed and accuracy. Finally, the method is deployed on the domain controller of the automobile to detect the driver in real time, displays the current state of the driver through the central control touch screen, and gives a suitable reminder if there is dangerous driving behavior.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent cockpit driver behavior detection of automobiles, and in particular relates to a dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm. BACKGROUND

[0002] With the continuous improvement of people's living standards and technological level, more and more people will use small cars as a means of transportation. According to statistics, by the end of 2022, the number of motor vehicles in China was 417 million, and the number of motor vehicle drivers was about 470 million. On average, there are about 60 cars for every 100 households. Small cars have gradually become an indispensable product in people's lives, but they also have many negative effects, such as tailpipe emissions polluting the environment and high energy consumption. The most concerning issue is that as the number of small cars increases, traffic accidents are also occurring more frequently.

[0003] According to statistics, in recent years, there have been more than 200,000 traffic accidents in China every year, and on average, one person loses their life due to a traffic accident every seven to eight minutes. There are many reasons for traffic accidents, some of which are uncontrollable external factors, such as road safety hazards, vehicle performance, and weather conditions. Some are due to the driver's illegal driving operations, such as smoking, making or receiving calls, not wearing a seatbelt, fatigue driving, and even drunk driving. Therefore, it is crucial to detect the driver's driving state in real time and determine whether the driver's driving state has any illegal behavior. If there is illegal driving behavior, appropriate reminders or operations can be provided in cooperation with intelligent driving technology, which can greatly reduce the occurrence of traffic accidents.

[0004] With the continuous development of machine vision and deep learning, the penetration of the field is also more extensive. In the field of automobiles, the development is also changing rapidly, and the development of automation and intelligence in automobiles is very rapid. Intelligent cockpit technology is also one of the main development directions of the automobile industry. Deep learning technology has great advantages in detection, and through deep learning network training models for vehicle detection, the purpose of detecting driver behavior can be achieved, providing an additional guarantee for safe driving. SUMMARY

[0005] The present application addresses the problems in the background art by developing a dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm, which can detect the current driving state of the driver in real time and provide appropriate reminders to reduce the likelihood of accidents.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] A dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm, specifically comprising the following steps:

[0008] (1) Collecting dangerous data of drivers

[0009] Collecting dangerous driving behavior data of drivers through cameras installed in the vehicle;

[0010] (2) Expanding the data set

[0011] Using the OODCI algorithm, the labeled data is trained, and the trained model is used for automatic labeling. In this process, the return value is not only the label but also the confidence of the label. Higher confidence data is selected to expand the training data set. The accuracy is ensured while labeling, and the sample is further enhanced and expanded, so as to obtain a large amount of data required for training;

[0012] (3) Training model with mixed network

[0013] Using convolutional neural network and genetic algorithm combination, convolutional layer is used for feature extraction, fully connected layer is used for classification task, and genetic algorithm is used for selecting the best feature extractor and selecting dense layer parameters;

[0014] (4) Deploy to domain controller

[0015] Pruning the trained model and embedding it into the vehicle-mounted domain controller to test the speed and accuracy of the model in the real driving environment of the vehicle;

[0016] (5) Classify driver behavior

[0017] Recognize the current state of the driver and classify it. If there is a corresponding dangerous behavior, give timely reminders to reduce the probability of accidents and achieve safe driving.

[0018] Preferably, the camera is installed on the co-driver A-pillar.

[0019] Preferably, the driver's dangerous driving behavior data mainly includes: making and receiving calls, smoking, eating and drinking, playing mobile phones, both hands leaving the steering wheel, not wearing seat belts, and normal driving.

[0020] Preferably, the OODCI algorithm has the following specific steps: after obtaining the classification result, the gradient of the model is returned using the cross-entropy loss function. The smaller the loss value, the better the model. The calculation method is as follows:

[0021]

[0022] In the above formula, T is the total number of samples, N is the total number of categories, is the predicted probability of the observation sample belonging to the category, y inis the true label, here a symbolic function that takes the value 1 when the sample is in the class and 0 otherwise.

[0023] Preferably, the genetic algorithm specific steps are as follows:

[0024] (1) VGG19, ResNet50 and DenseNet121 are used as feature extractors, based on different numbers of feature extractors in each model, the genetic algorithm uses 13 VGG layers or 120 DenseNet151 or any number between 1 and 40 ResNet 50 for feature extraction;

[0025] (2) After the convolutional layer, the driver distraction classification is performed using the dense layer, 2 dense layers and 2 dropout layers are used before the last classification layer, the number of neurons in the last layer is equal to the number of classes in the dataset, the activation function is the softmax function, and other hyperparameters, after creating this structure, the genetic algorithm will select the best hyperparameter set based on the objective function, which reduces the loss while improving the accuracy, the formula of the objective function is as follows:

[0026]

[0027] Where Accuracy and Loss represent the accuracy and loss value, which is the accuracy and loss of the model created by the genetic algorithm by selecting different hyperparameters, and the loss function is the cross-entropy loss function;

[0028] (3) At the beginning of the training stage, the compiled model will be selected according to the feature extractor, and the number of neurons in each layer and the corresponding activation function will be randomly initialized, then the fitness function of the first generation is calculated, and a generation is created by using the crossover and mutation of the parents; The fitness values of the new generation are sorted in descending order, and the best values are selected for the next generation.

[0029] Preferably, the dropout factor is selected between 0.10-0.5, and the activation function is selected between ReLU, SELU and ELU.

[0030] Compared with the prior art, the present application has the following beneficial effects:

[0031] (1) The present application uses the OODCI algorithm to train the samples that have been calibrated, and then uses the model to calibrate a large number of uncalibrated sample sets, so as to achieve the purpose of expanding the data set, and the accuracy is high.

[0032] (2) The application uses a combination of convolutional neural networks and genetic algorithms, the convolutional layer is used for feature extraction, the fully connected layer is used for classification tasks, and the genetic algorithm is used to select the best feature extractor and select the dense layer parameters. Using the combination of the algorithm, the loss of the model gradually decreases, and the accuracy continuously improves. The constraint of completing the training process is 99% accuracy in order to save the best model and get better results than other work. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A flowchart of the whole process of the dangerous driving behavior recognition based on the hybrid neural network of the application.

[0034] Figure 2 A flowchart of the OODCI algorithm based on the dangerous driving behavior data enhancement of the application.

[0035] Figure 3 A flowchart of the genetic algorithm based on the dangerous driving behavior recognition of the application.

[0036] Figure 4 A flowchart of the dangerous driving behavior recognition of the application. DETAILED DESCRIPTION

[0037] In order to further illustrate the technical means and effects taken by the application to achieve the predetermined application purposes, the specific embodiments, structures, features and effects of the application are described in detail below in combination with the drawings and preferred embodiments.

[0038] The specific implementation process of the application is shown in Figure 1 and specifically includes the following schemes:

[0039] The application first photographs a large number of training data sets, uses the OODCI algorithm, and uses the already labeled data to train the model, in this way to label a large number of unlabeled data. The return value in this process is not only the label but also the credibility of the label, we select the data with higher credibility to expand the training data set, at the same time use rotation, scaling range, horizontal inversion, brightness adjustment and normalization technology to further enhance and expand the sample, so as to obtain a large amount of data required for training. Then through the combination of convolutional neural network and genetic algorithm, the speed and accuracy are greatly improved. Finally, deploy it to the domain controller of the car, detect the driver in real time, display the current state of the driver through the central control touch screen, and give appropriate reminders if there is dangerous driving behavior.

[0040] The driving behaviors involved in the application are as follows: making and receiving calls, smoking, eating and drinking, playing mobile phones, taking hands off the steering wheel, not wearing seat belts, normal driving and the like. In order to improve the reliability of the training model, the quality and quantity of the training data play a decisive role. For the above 7 driving behaviors, there is no suitable public data set, but the training of the network must be supported by a large number of data sets, and undoubtedly the labeling of the data set is a huge workload. At present, many algorithms can complete the function of labeling unlabeled data, but the accuracy is not very high. For the training model, if the information of the label itself is inaccurate, the model will learn the wrong information, which will directly lead to the decrease of the accuracy of the model.

[0041] In order to solve this problem, the application proposes an OODCI algorithm based on small sample learning, the main idea of which is to train through the labeled samples, and then use the model to label a large number of unlabeled sample sets, so as to achieve the purpose of expanding the data set. The OODCI algorithm judges the labeling at the same time, and feeds back the confidence degree while feeding back the label, we can select a batch of data with high confidence to expand the data set, and the algorithm flow of OODCI is as follows Figure 2 After obtaining the classification result, the gradient of the model is returned using the cross-entropy loss function, the smaller the loss value, the better the model, and the calculation method is as follows:

[0042]

[0043] In the above formula, T is the total number of samples, N is the total number of categories, is the predicted probability of the observation sample belonging to the category, y in is the true label, which is a symbol function at this time, when the sample is classified, its value is 1, otherwise 0. In order to expand the data set of the model, rotation, scaling range, horizontal inversion, brightness adjustment and normalization techniques are also used to enhance the sample, so as to obtain a large amount of data required for training.

[0044] The research idea of the application is to use genetic algorithm (GA) and convolutional neural network in combination, to develop a hybrid model to monitor whether the driver has dangerous behavior during driving. The main contributions are as follows: a new combination of convolutional neural network and machine learning algorithm is optimized for detecting dangerous driving behavior; a data set of dangerous behavior in this paper is made; the corresponding time of driver behavior detection and recognition is shortened.

[0045] In the study, the convolutional layers are used for feature extraction, the fully connected layers are used for classification task, and the genetic algorithm is used for selecting the best feature extractor and selecting the dense layer parameters. We all know that the input extraction is the most important aspect of the convolutional neural network, and there are many choices for feature extraction. In the past few decades, many excellent CNN models have emerged, which have performed very well in image classification. The most noteworthy ones are VGG, ResNrt, and DenseNet that have appeared in recent years. These networks are developed based on different convolutional connections. For example, the VGG variants use almost all convolutional layers with different sizes of convolutional kernels to obtain features during feature extraction, which makes the network deeper. ResNet uses a skip connection method to make the signal flow more effective in the network. In this way, we can create a deeper network. In the VGG variant, there are 16 layers, 19 layers, and 32 layers of network, and in the ResNet, the number of layers can be increased to 51 layers or even 151 layers. In the DenseNet, each layer of the network obtains additional input from all previous layers and passes its results to all subsequent layers. The structure of DenseNet is a combination of ResNet and VGG models, and its purpose is to make the network model structure deeper. After understanding the structure of the feature extractor, the genetic algorithm is used to select the best number of layers for feature extraction. Here, VGG19, ResNet50 and DenseNet121 are used as feature extractors. Based on the different number of feature extractors in each model, GA can use 13 VGG layers or 120 DenseNet151 or any number between 1 to 40 ResNet50 for feature extraction. After the convolutional layer, the dense layer is used for driver distraction classification. Two dense layers and two dropout layers are used before the last classification layer. The number of neurons in the last layer is equal to the number of classes in the dataset, and the activation function is the softmax function. Other hyperparameters, such as the number of neurons in each hidden layer, are determined by the genetic algorithm. The dropout factor can be selected between 0.10 ~ 0.5, and the activation function can be selected between ReLU, SELU and ELU. After creating this structure, the genetic algorithm will try to select the best set of hyperparameters based on the objective function to improve accuracy and reduce loss. The formula of the objective function is as follows:

[0046]

[0047] where Accuracy and Loss represent the accuracy and loss value, which is the accuracy and loss of the model created by the genetic algorithm by selecting different hyperparameters. The loss function in the invention is the cross-entropy loss function, as follows: Figure 3The training process starts with the selection of the compiled model according to the feature extractor and the random initialization of the number of neurons in each layer and the corresponding activation function. Then, the fitness function of the first generation is calculated. With the crossover and mutation of the parents, the next generation is created. The fitness values of the new generation are sorted in descending order and the best values are selected for the next generation. Using this algorithm, the loss of the model gradually decreases and the accuracy constantly improves. The constraint to complete the training process is the accuracy of 99% in order to save the best model and get better results than other works. Then the trained model is embedded in the vehicle domain controller to monitor the driver's driving state in real time through the camera and give appropriate reminders. The main workflow is as follows Figure 4 .

[0048] The above is only the preferred embodiment of the present application, not any form of limitation on the present application, although the present application has been disclosed as above with the preferred embodiment, however, not to limit the present application, any person skilled in the art, within the scope of the technical scheme of the present application, can make some changes or modifications to the equivalent embodiments with the above disclosed technical content, but as long as it does not deviate from the technical scheme of the present application, according to the technical essence of the present application, any modification, equivalent change and modification of the above embodiments, still belongs to the scope of the technical scheme of the present application.

Claims

1. A method for dangerous driving behavior recognition based on a convolutional neural network and a genetic algorithm, characterized in that: Specifically comprising the following steps: (1) Collecting the driver's data The driver's driving behavior data is collected by a camera installed in the vehicle; (2) Expanding the data set Using the OODCI algorithm, the labeled data is trained, and the trained model is used for automatic labeling. In this process, the return value is not only the label but also the confidence of the label. Select the data with higher confidence to expand the training data set. While ensuring accuracy, the sample is further enhanced and expanded to obtain a large amount of data required for training; The specific steps of the OODCI algorithm are as follows: after obtaining the classification result, the gradient of the model is returned using the cross-entropy loss function. The smaller the loss value, the better the model. The calculation method is as follows: T is the total number of samples, N is the total number of classes, is the predicted probability of a sample belonging to a class, y in is the true label, which is a sign function, taking the value 1 when the sample is in the class and 0 otherwise. (3) Training model using hybrid network Convolutional neural network and genetic algorithm are combined. Convolutional layer is used for feature extraction, and fully connected layer is used for classification task. Genetic algorithm is used to select the best feature extractor and dense layer parameters; The specific steps of the genetic algorithm are as follows: (1) VGG19, ResNet50 and DenseNet121 are used as feature extractors. Based on different number of feature extractors in each model, genetic algorithm uses 13 VGG layers or 120 DenseNet151 or any number between 1 to 40 ResNet 50 for feature extraction; (2) After the convolutional layer, dense layer is used for driver distraction classification. 2 dense layers and 2 dropout layers are used before the last classification layer. The number of neurons in the last layer is equal to the number of classes in the data set. The activation function is softmax function. Other hyperparameters, after creating this structure, genetic algorithm will select the best hyperparameter set based on the objective function to improve accuracy and reduce loss. The formula of the objective function is as follows: Where Accuracy and Loss represent accuracy and loss value, and is the accuracy and loss of the model created by genetic algorithm by selecting different hyperparameters. The loss function is cross-entropy loss function; (3) At the beginning of the training stage, the compiled model will be selected according to the feature extractor, and the number of neurons in each layer and the corresponding activation function will be randomly initialized. Then, the fitness function of the first generation is calculated, and a generation is created by using the crossover and mutation of the parent generation. The fitness values of the new generation are sorted in descending order, and the best values are selected for the next generation; (4) Deploy to domain controller Prune the trained model and embed it into the vehicle domain controller. Test the speed and accuracy of the model in the real driving environment of the vehicle; (5) Classify the driver's behavior Identify the current state of the driver and classify it. If there is a corresponding dangerous behavior, give timely reminders to reduce the probability of accidents and achieve safe driving.

2. The dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm according to claim 1, characterized in that: The camera is installed on the co-driver A-pillar. 3.The dangerous driving behavior recognition method based on convolutional neural network and genetic algorithm according to claim 1, characterized in that: The driver's driving behavior data mainly includes: making and receiving calls, smoking, eating and drinking, playing mobile phones, both hands leaving the steering wheel, not wearing seat belts, and normal driving.

4. The dangerous driving behavior recognition method based on a convolutional neural network and a genetic algorithm according to claim 1, characterized in that: The dropout factor is selected between 0.10-0.5 and the activation function is selected between ReLU, SELU, and ELU.

Images