Traffic sign recognition method based on deep learning
Through the traffic sign recognition method based on deep learning, using convolutional neural network and pre-processing technology, the problem of insufficient recognition ability of traditional methods in complex environments is solved, and efficient and accurate traffic sign recognition is achieved.
Patent Information
- Application Number
- CN202411422951.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-14
AI Technical Summary
Traditional traffic sign recognition methods are difficult to adapt to complex and changeable road environments, and due to the characteristics of manual design and the generalization ability of classifiers, it is difficult to achieve accurate and efficient traffic sign recognition.
Adopt a traffic sign recognition method based on deep learning, including building a traffic sign image data set, preprocessing and enhancing the data, using a convolutional neural network model for training and classification, and alleviating overfitting by adding a dropout layer and L1 regularization .
It achieves efficient identification and classification of traffic signs in complex road environments, improves the robustness and adaptability of the model, and improves the recognition effect of more traffic signs.
Smart Images
Figure CN119314148A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a traffic sign recognition method based on deep learning. Background Art
[0002] The development of the automobile industry has greatly facilitated people's work and life, but it has also inevitably brought about traffic safety problems. One of the ways to solve traffic safety problems is to accurately and effectively set up road traffic signs and provide drivers with a wealth of prohibitions, warnings, instructions and other information, thereby reducing traffic accidents. In order to ensure that the information of traffic signs can be conveyed in a timely and accurate manner, the automatic traffic sign recognition system (Traffic Sign Recognition, TSR) has attracted the attention of scholars from various countries. The traffic sign recognition system has a wide range of application prospects. It is mainly used in driving assistance, traffic sign maintenance, unmanned driving and other fields, involving machine vision, pattern recognition, image processing, digital signal processing, artificial intelligence, communication and information technology and other disciplines. It is a typical pattern recognition application system like face recognition and target tracking. However, although traditional methods such as SVM and SVMs have achieved certain results, they are limited by the hand-designed features and the generalization ability of the classifier, and are difficult to adapt to the complex and changing road environment. Therefore, using deep learning technology to perform traffic sign recognition has become a topic of great practical significance. Summary of the invention
[0003] The purpose of the present invention is to provide a traffic sign recognition method based on deep learning to solve the problems existing in the above-mentioned background technology.
[0004] To achieve the above object, the present invention provides a traffic sign recognition method based on deep learning, comprising the following steps: S1. Construct a traffic sign image dataset, and enhance and preprocess the data in the traffic sign image dataset; S2, detect traffic signs and extract features and target areas in the image; S3, building a convolutional neural network model, training it with the training set data in the traffic sign image dataset, extracting the features of the traffic signs, and classifying them; S4. Identify the classified traffic signs and output the names and categories of the traffic signs.
[0005] Preferably, preprocessing the data in the traffic sign image dataset specifically includes: S11, performing image cropping on the traffic sign images in the traffic sign image dataset; S12, performing standardization processing on the cropped traffic sign image; S13, performing size normalization processing on the standardized traffic sign image; S14, output the traffic sign image after preprocessing for the convolutional neural network model input image.
[0006] Preferably, in step S2, the traffic sign is detected by using HOG features to describe the traffic sign image. The HOG (Histogram of Oriented Gradients) feature is a commonly used image feature description method that can be used for target detection and classification tasks. In this project, the HOG feature is used to describe the traffic sign image, and the target area is classified by the SVM algorithm, specifically including: using the cv2.HOGDescriptor() function in OpenCV to extract the HOG features, and using the cv2.ml.SVM_create() function to create an SVM classifier; in the training phase, the extracted HOG features and corresponding labels are input into the SVM classifier for training, and in the testing phase, the input image is subjected to HOG feature extraction and SVM classification to obtain the target area and category of the traffic sign.
[0007] Preferably, convolutional neural network is one of the most important neural network structures, which is widely used in image classification, target detection and other fields. A typical convolutional neural network is usually composed of an input layer, a convolution layer, a pooling layer and a fully connected layer. The convolution layer is used to extract local features of the image. In image processing, the convolution layer performs a convolution operation by sliding a filter on the input image to extract feature information such as image contour, contrast, and brightness. The parameters in the convolutional network are filter weights, and the parameters are learned through the back propagation algorithm. As shown below: ; Among them, the convolution kernel and The bias is a learnable parameter and the gradient is calculated using the chain rule.
[0008] The pooling layer is used for feature selection, which can reduce the number of parameters and avoid overfitting. This method uses the maximum pooling sampling method to select the maximum output of neurons in each sub-region as the summary representation of the region.
[0009] The function of the fully connected layer is to map the feature representation learned by the convolution part to the sample category space and provide input for the final classification or regression.
[0010] The convolutional neural network model in step S3 includes an input layer, three custom convolutional units, a fully connected layer and an output layer, and the three custom convolutional units all include a convolutional layer, a batch normalization layer, a ReLU activation layer and a pooling layer; a batch normalization layer is inserted between each convolutional layer and the ReLU activation layer to normalize the data, which normalizes the data to a uniform interval, reduces the degree of data divergence, and retains the original data distribution to a certain extent, avoids gradient explosion or gradient disappearance, and plays a certain regularization role, thereby accelerating the convergence of the model and making the training process more stable; the fully connected layer is composed of a flattening layer and a dropout layer.
[0011] Preferably, the activation function of the ReLU activation layer adopts the Leaky ReLU function. When , a non-zero real number is output, and its mathematical expression is: ; in, For input, , output a non-zero real number.
[0012] Preferably, in order to alleviate overfitting and improve the generalization ability of the model, thereby improving its recognition effect on more traffic signs, the convolutional neural network model and loss function are optimized by adding a dropout layer and L1 regularization.
[0013] Preferably, the dropout layer is used in conjunction with a fully connected layer with more weight parameters, and specifically, a number of neurons and their corresponding connecting edges are randomly discarded during training of the convolutional neural network to avoid overfitting. In this way, by activating only a certain proportion of neurons each time, multiple different network structures can be generated and their learning effects can be comprehensively examined, thereby improving the robustness of the model; and the neurons discarded by the dropout layer each time are randomly selected, specifically, a fixed probability is set in advance p , and then for each neuron, the probability p Discard (i.e. set the neuron output to 0).
[0014] Preferably, the L1 regularization method is to introduce the L1 regularization penalty term into the loss function to sparse the model parameters, which plays a role in feature selection, model simplification and speeding up the calculation. At the same time, L1 regularization can also prevent overfitting to a certain extent. It is the sum of the absolute values of all parameters of the fully connected layer, and is weighted. The expression of the loss function is: ; in, represents the cross entropy loss, is the weight parameter of the penalty term, and the sum sign represents the sum of all weight parameters of the fully connected layer. Norm summation.
[0015] Therefore, the present invention adopts the above-mentioned traffic sign recognition method based on deep learning, which has the following beneficial effects: (1) According to the characteristics of the images in the dataset, the dataset is properly preprocessed and a suitable network structure is selected to retain the main features of the image as much as possible while controlling the computational cost; (2) By adding a dropout layer and an L1 regular penalty term, overfitting is alleviated and the robustness of the model is improved.
[0016] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A flow chart of a traffic sign recognition method based on deep learning according to the present invention; Figure 2 A schematic diagram of the structure of a convolutional neural network model of a traffic sign recognition method based on deep learning in the present invention; Figure 3 It is a schematic diagram of the convergence of the loss function on the training set and the validation set in the traffic sign classification task of an embodiment of the present invention; Figure 4 Schematic diagram of the ROC curve of the traffic sign classification result according to the embodiment of the present invention. DETAILED DESCRIPTION
[0018] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] like Figure 1 As shown, a traffic sign recognition method based on deep learning includes the following steps: S1. Construct a traffic sign image dataset, and enhance and preprocess the data in the traffic sign image dataset; S2, detect traffic signs and extract features and target areas in the image; S3, build a convolutional neural network model, train it with the training set data in the traffic sign image dataset, extract the features of the traffic signs, and classify them; Figure 2As shown in the figure, the convolutional neural network model includes an input layer, three custom convolutional units, a fully connected layer and an output layer, and the three custom convolutional units all include a convolutional layer, a batch normalization layer, a ReLU activation layer and a pooling layer; a batch normalization layer is inserted between each convolutional layer and the ReLU activation layer for data normalization. This layer normalizes the data into a uniform interval, reduces the degree of data divergence, and retains the original data distribution to a certain extent, avoids gradient explosion or gradient disappearance, and plays a certain regularization role, thereby accelerating the convergence of the model and making the training process more stable; the fully connected layer is composed of a flattening layer and a dropout layer.
[0020] In actual training, the Leaky ReLU function is used, which is a non-saturated activation function and an improvement on the ReLU function. It solves the problem of neuron deactivation in the ReLU function and does not cause the gradient to disappear.
[0021] S4. Identify the classified traffic signs and output the names and categories of the traffic signs.
[0022] The dataset used in this embodiment is CTSDB. The CTSDB dataset contains 5998 traffic sign images of 58 categories, each image is a zoomed view of a single traffic sign, and the image annotation provides basic attribute information such as the image file name, width, height, and category.
[0023] In this embodiment, the data set is randomly divided into a training set (4000 pictures), a validation set (800 pictures) and a test set (1198 pictures).
[0024] During the training process, Adam was selected as the optimizer, batch_size was set to 50, the initial learning rate was set to 0.0008, and the epoch was set to 30.
[0025] In this embodiment, the aforementioned parameter settings are used to perform training and testing on the CTSDS dataset, and the cross entropy loss is selected as the loss function.
[0026] This embodiment is a multi-classification task. Due to the imbalance between categories in the data set, when evaluating the classification effect of the model, the usual accuracy index (Acc) is not sufficient to fully characterize the prediction accuracy, and other indicators need to be introduced.
[0027] ; Note that the CTSDB dataset has a total of 58 category labels, and the prediction accuracy (p-value) corresponding to each category can also be calculated with the help of the confusion matrix and recall (r-value) (Change the category are considered as positive samples, and other categories are considered as negative samples) ; ; Further calculate the true positive rate of the classification results ( , equivalent to recall) and false positive rate ( ): ; ; Based on this, the ROC curve is drawn and the area under the curve (AUC) value is calculated. The larger the AUC, the closer the ROC curve is to the upper left corner, that is, Much greater than , the model approaches a completely accurate classifier.
[0028] In order to extend the AUC index of the binary classification task to the multi-classification case, the so-called "micro-AUC", that is, the global AUC value, can be introduced. The specific calculation method is to accumulate the confusion matrices of all categories and then calculate , , AUC. This method takes into account the characteristics of the imbalanced data set and is more reasonable as an evaluation indicator.
[0029] The total time for model training is 369.5s. The loss function value changes during training and verification are as follows: Figure 3 shown.
[0030] Obviously, the loss curve converges relatively smoothly after 30 epochs, and the convergence speed is relatively fast. It can be seen that the loss function values of the training process and the verification process continue to decline steadily, indicating that the model has reached a relatively ideal learning state. After calculation, the ACC value of the classification model is 89.65%, the AUC value is 0.99, and the ROC curve is drawn as follows Figure 4 The graph shows the relationship between the true positive rate and false positive rate of the classification results when the judgment threshold of the classifier is set to different values in [0,1]. Figure 4 As can be seen in the figure, the left half of the ROC curve is close to the vertical axis, indicating that a suitable threshold can be obtained so that the classifier can achieve the highest possible true positive rate while keeping the false positive rate very low, that is, the classifier has good performance in practical applications.
[0031] Therefore, the present invention adopts the above-mentioned traffic sign recognition method based on deep learning, which can efficiently complete the recognition and classification tasks of traffic signs and has good adaptability to complex environments.
[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A traffic sign recognition method based on deep learning, characterized in that: The following steps are involved: S1. Construct a traffic sign image dataset, and enhance and preprocess the data in the traffic sign image dataset; S2, detect traffic signs and extract features and target areas in the image; S3, building a convolutional neural network model, training it with the training set data in the traffic sign image dataset, extracting the features of the traffic signs, and classifying them; S4. Identify the classified traffic signs and output the names and categories of the traffic signs.
2. The traffic sign recognition method based on deep learning according to claim 1, characterized in that: Preprocess the data in the traffic sign image dataset, including: S11, performing image cropping on the traffic sign images in the traffic sign image dataset; S12, performing standardization processing on the cropped traffic sign image; S13, performing size normalization processing on the standardized traffic sign image; S14, output the traffic sign image after preprocessing for the convolutional neural network model input image.
3. The traffic sign recognition method based on deep learning according to claim 1, characterized in that: In step S2, traffic signs are detected by using HOG features to describe the traffic sign image and classifying the target area through the SVM algorithm, which specifically includes: using the cv2.HOGDescriptor() function in OpenCV to extract HOG features and using the cv2.ml.SVM_create() function to create an SVM classifier; in the training phase, the extracted HOG features and corresponding labels are input into the SVM classifier for training; in the testing phase, the input image is subjected to HOG feature extraction and SVM classification to obtain the target area and category of the traffic sign.
4. The traffic sign recognition method based on deep learning according to claim 1, characterized in that: The convolutional neural network model in step S3 includes an input layer, three custom convolutional units, a fully connected layer and an output layer, and the three custom convolutional units all include a convolutional layer, a batch normalization layer, a ReLU activation layer and a pooling layer; the fully connected layer consists of a flattening layer and a dropout layer.
5. The traffic sign recognition method based on deep learning according to claim 4, characterized in that: The activation function of the ReLU activation layer uses the Leaky ReLU function. When , a non-zero real number is output, and its mathematical expression is: ； in, For input, , output a non-zero real number.
6. The method for traffic sign recognition based on deep learning according to claim 5, characterized in that: The convolutional neural network model and loss function are optimized by adding dropout layer and L1 regularization.
7. The method for traffic sign recognition based on deep learning according to claim 6, characterized in that: The dropout layer is used in conjunction with the fully connected layer with many weight parameters. Specifically, when training the convolutional neural network, a number of neurons and their corresponding connection edges are randomly discarded to avoid overfitting. The neurons discarded by the dropout layer are randomly selected each time, with a fixed probability set in advance. p , and then for each neuron, the probability p Discard.
8. The method for traffic sign recognition based on deep learning according to claim 6, characterized in that: The L1 regularization method introduces the L1 regularization penalty term into the loss function, which is the sum of the absolute values of all parameters of the fully connected layer and performs weighted processing. The expression of the loss function is: ； in, represents the cross entropy loss, is the weight parameter of the penalty term, and the sum sign represents the sum of all weight parameters of the fully connected layer. Norm summation.