Traffic sign text detection and recognition method for unmanned driving system

Abstract

The invention discloses a traffic sign text detection and recognition method for an unmanned driving system. The method mainly comprises three stages of traffic sign detection, traffic sign text area detection and traffic sign text recognition. In the traffic sign detection stage, the traffic sign is extracted from a natural scene by using an improved yolov5s network; in the text area detection stage, an output image detected by the traffic sign is used as input, a text area containing a text is output through a text detection network improved based on DBNet, the text area is input into a text recognition model improved based on CRNN, and text characters on the traffic sign are recognized and obtained. According to the algorithm, the text content in the traffic sign can be recognized in a complex natural scene, and the recognition accuracy and real-time performance are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision in intelligent transportation, and in particular to a traffic sign text detection and recognition method for an unmanned driving system. Background Art

[0002] In recent years, my country's transportation infrastructure construction has achieved leapfrog development. At the same time, unmanned driving technology has also ushered in rapid progress. In this context, text traffic signs, as key information media in the transportation system, carry a large amount of accurate and rich traffic semantic information. This information is crucial for the navigation, decision-making and behavior planning of unmanned driving systems. Therefore, accurately detecting and identifying text information on text traffic signs is of great significance for improving the perception ability of unmanned driving systems, ensuring driving safety, and optimizing traffic flow management.

[0003] With the rapid development of deep learning technology, the detection and recognition technology of text information in natural scenes has made significant progress. Despite this, there are currently only a small number of works on the detection and recognition of text traffic signs, and the recognition work on Chinese traffic signs is even more limited. In addition, there is a relative lack of public Chinese traffic sign recognition datasets, which limits the development and application of related technologies. There are still many problems in existing research work, including the lack of distinction between traffic signs and other signs in complex natural road environments; and technical problems such as low text detection accuracy in variable natural environments, such as weather changes. Summary of the invention

[0004] In view of the problems of the prior art, the present invention establishes a text detection and recognition data set for Chinese traffic signs, and proposes a method for detecting and recognizing traffic sign text in complex environments for unmanned driving systems. The algorithm introduces a traffic sign detection module, which can accurately and efficiently identify traffic signs in natural scenes, and improves the text detection and recognition algorithm, which can improve the accuracy and real-time performance of text detection and recognition on traffic signs in complex natural environments, and effectively improve the robustness of the system.

[0005] The object of the present invention is to achieve the following technical solution: a method for detecting and recognizing text on traffic signs for unmanned driving systems, comprising the following steps: (1) Obtain images of traffic signs in natural scenes and construct a dataset for Chinese traffic sign text detection and recognition. Use the dataset to train the traffic sign detection network, text detection network, and text recognition network. (2) Construct a traffic sign detection network based on yolov5s, and use the traffic sign detection network based on yolov5s to process the input natural scene image and detect the area of ​​the traffic sign; (3) Construct a text detection network based on DBNet, and use the improved text detection network based on DBNet to perform text detection on the traffic sign image in the area of ​​the input traffic sign, and extract the text area in the traffic sign; (4) Construct a text recognition network based on the improved CRNN, and use the text recognition network based on the improved CRNN to input the text area after text detection into the recognition network to identify the text content in the text area.

[0006] Furthermore, in the step (1), a Chinese traffic sign text detection and recognition dataset is constructed, specifically by collecting and integrating traffic sign images with Chinese text on the Internet, annotating the traffic sign images in the dataset, including annotating the traffic signs, annotating the text areas, and annotating the text contents, and dividing the annotated images into a training set and a test set.

[0007] Furthermore, the specific construction method of the traffic sign detection network improved by yolov5s in step (2) includes the following sub-steps: (2.1) The backbone network in yolov5s is lightweighted, and the backbone network CSPDarknet53 of yolov5s is replaced with a lightweight GhostNet network; the SE attention mechanism module in the model is replaced with the SRM module to obtain an improved traffic sign detection network based on yolov5s; (2.2) Train the traffic sign detection network improved based on yolov5s. Output a large number of images of the training set in the CTSTD dataset to the improved yolov5s network for training to obtain the optimal weight of the network model. Output the images of the test set to the improved yolov5s network configured with the optimal weight for testing. The evaluation indicators of network performance are average precision AP and frame rate FPS.

[0008] Furthermore, the specific construction method of the text detection network improved by DBNet in step (3) includes the following sub-steps: (3.1) Construct a text detection network. The improved DBNet network is used as the detection model in the text detection stage. The DBNet network includes feature extraction, feature fusion and Head network. The feature extraction part uses the ResNet18 residual network as the backbone network. In the conv3, conv4 and conv5 stages of the feature extraction network, all 3x3 convolutional layers are replaced with spatial and channel reconstruction convolution SCConv; for the feature fusion part, an improved spatial and channel mixed attention mechanism is used to enhance feature expression, that is, after the last step of upsampling and feature fusion operations, an attention mechanism CBAM that mixes the spatial attention mechanism and the channel attention mechanism is added; (3.2) Train the traffic sign text detection network. Extract the traffic sign area images from the images in the CTSTD dataset through the traffic sign detection network. After the traffic sign area images are processed to a uniform size, they are input into the text detection network improved based on DBNet for training. After obtaining the optimal weight of the network model, the network performance is tested using the test set data. The network performance is evaluated using the F1 value and the frame rate FPS.

[0009] Furthermore, the implementation process of the spatial attention mechanism in the step (3.1) is as follows: first, the feature map output by the channel attention is subjected to maximum pooling and average pooling operations along the channel dimension to generate features of different context scales; then, a splicing operation is performed along the channel dimension, specifically, two feature maps are spliced ​​through a 7×7 convolutional layer with an input channel of 2 and an output channel of 1, and finally, the generated spatial attention weights are applied; finally, the channel attention weights are constrained between 0 and 1 through the Sigmoid activation function.

[0010] Furthermore, the implementation process of the channel attention mechanism in the step (3.1) is as follows: first, the input feature map is globally averaged and globally max-pooled to generate two feature vectors containing the number of channels, and the global features of each channel are obtained; then, the above two feature vectors containing the number of channels are input into a shared multi-layer perceptron to learn the attention weight of each channel. This operation is to construct a fully connected layer through 1×1 convolution to learn the weight of each channel; finally, the channel attention weight is constrained between 0 and 1 through the Sigmoid activation function.

[0011] Furthermore, the specific construction of the text recognition network improved by CRNN in step (4) includes the following sub-steps: (4.1) Construct a text recognition network. The text recognition network uses an improved CRNN network as the recognition model. The CRNN network consists of a convolutional layer for feature extraction, a recurrent layer for learning and prediction, and a transcription layer for decoding and classification. Improve the activation function of the convolutional layer and replace the original activation function with the Softplus activation function. The Softplus activation function conforms to the biological model of neuron activation and avoids the gradient being 0 during training. (4.2) Train the traffic sign text detection network. The text area extracted in step (3) is processed into a uniform size and input into the text recognition network improved by CRNN for training. After obtaining the optimal weight of the network model, the network performance is tested with the test set data. The evaluation indicator text recognition accuracy is used to evaluate the recognition accuracy.

[0012] Specifically, the Softplus activation function in step (4.1) is defined as Softplus( x )=ln(1+ e x ), which is smooth and has continuous derivatives over the entire real number domain.

[0013] Furthermore, in the step (4.1), the recurrent layer uses a bidirectional long short-term memory network Bi-LSTM to capture the contextual information in the sequence using the forward and backward networks respectively, and then fuses the information in the two directions by adding and splicing; after Bi-LSTM processing, a probability distribution will be output for each time step, which contains the predicted probabilities of all characters. These probability distributions constitute a posterior probability matrix, in which each row corresponds to a time step and each column corresponds to a character category; the transcription layer takes this posterior probability matrix as input and processes it using the CTC loss function. In CTC, one output sequence corresponds to multiple paths. The task of the transcription layer is to find the label sequence with the highest probability combination based on the prediction of each frame, that is, the maximum probability path; after CTC translation, the sequence feature information learned by the network is converted into the final recognition text.

[0014] Specifically, the evaluation index text recognition accuracy is used in step (4.2) to evaluate the recognition accuracy, and its calculation formula is: , Indicates the number of samples that are correctly identified. Represents the total number of samples in the test set.

[0015] The beneficial effects of the present invention are as follows: A method for detecting and recognizing traffic sign text for unmanned driving systems is proposed. Through the three stages of traffic sign detection, traffic sign text area detection, and traffic sign text recognition, the text information on the traffic sign can be effectively extracted from the images taken by the vehicle camera. It is of great significance for improving the perception ability of the unmanned driving system, ensuring driving safety, and optimizing traffic flow management. An improved target detection network based on yolov5s is proposed. Traffic sign detection is added before the traditional text detection and recognition algorithm, which can effectively eliminate the interference of other signs in complex natural scenes. At the same time, the network is lightweighted and the attention mechanism is updated, which can effectively improve the real-time and accuracy of traffic sign detection. An improved text detection network based on DBNet is proposed. Convolutional reconstruction is performed in the feature extraction process, and an attention mechanism module is added in the feature fusion process, so that the text detection network can still accurately extract the text area of ​​the traffic sign when facing weather changes in natural scenes. A text recognition network based on improved CRNN is proposed. The activation function of the convolutional layer is improved in the feature extraction process to make the training and learning process more stable. It can effectively deal with the problem of sharp changes in brightness of traffic sign images caused by illumination changes in complex natural scenes, thereby enhancing the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 This is the network structure diagram of GhostNet, the backbone network of the traffic sign detection network improved by yolov5s; Figure 3 This is the structure diagram of the SRM attention module; Figure 4 This is the network structure diagram of the traffic sign text detection network feature extraction network ResNet18 before and after improvement based on DBNet; Figure 5 This is a schematic diagram of the Softplus activation function replaced by the convolutional layer of the traffic sign text recognition network based on the CRNN improvement. DETAILED DESCRIPTION

[0017] The core technology of the present invention is to detect and identify the text content of the traffic sign, so as to realize accurate and rapid identification of the text content of the traffic sign.

[0018] The present invention proposes a method for detecting and recognizing traffic sign text for an unmanned driving system. Figure 1 As shown, the following steps are included: (1) Obtain images to be detected containing traffic signs in natural scenes; and construct a dataset for Chinese traffic sign text detection and recognition; (2) Input the image into the trained traffic sign detection network based on yolov5s for detection and output the traffic sign area; (3) Input the obtained traffic sign image into the trained text detection network and output the text area; (4) Input the obtained text area image into the trained text recognition network and output the text content.

[0019] For the three-stage deep learning network of traffic sign detection network, text detection network and text recognition network based on yolov5s improvement mentioned in steps (2)-(4), a data set is needed to train and test the network. Since the current public Chinese traffic sign text detection and recognition data sets are relatively scarce, a Chinese traffic sign text detection and recognition data set is constructed by ourselves. The steps to construct the data set are as follows: collect and integrate traffic sign images with Chinese text on the Internet, annotate the traffic sign images in the data set, and divide the annotated images into training set and test set. The specific construction method is as follows: a) The sources of traffic sign images are mainly the following parts: TT100K data set, China Traffic Sign Detection Dataset (CCTSDB), public data set (CTSU Dataset), and traffic sign data set collected by individuals. Images containing Chinese text signs are selected from the first two data sets and supplemented with Chinese text traffic sign images taken by individuals, and a new traffic sign text data set CTSTD (Chinese Traffic Sign Text Dataset) is constructed. The traffic sign text dataset CTSTD has a total of 2,000 original images, which are expanded to 8,000 images after data enhancement, of which 7,000 are used as training sets and 1,000 are used as test sets.

[0020] b) Annotate the CTSTD dataset, including annotation of traffic signs, annotation of text areas, and annotation of text content. The annotation of traffic signs is used for training and testing in the traffic sign detection phase, the annotation of text areas is used for training and testing in the traffic sign text detection phase, and the annotation of text content is used for training and testing in the traffic sign text recognition phase.

[0021] The step (2) is the traffic sign detection stage. The detection network uses a traffic sign detection network improved based on yolov5s to process the input natural scene image and detect the area of ​​the traffic sign therein.

[0022] The specific construction method of the traffic sign detection network improved based on yolov5s is as follows: (2.1) The yolov5s target detection network is mainly composed of three parts: backbone network, neck network and head network. The present invention replaces the yolov5s backbone network CSPDarknet53 with a lighter GhostNet network to reduce the number of network parameters and improve feature extraction efficiency. The network structure of the GhostNet network is as follows: Figure 2 As shown in the figure (Conv2d is a convolution operation using a two-dimensional 1×1 convolution kernel; G-bneck is the Ghost bottleneck residual module), the input image is unified into a 224×224×3 format and input into the GhostNet network. It is first processed by a 16-channel ordinary convolution block and then processed by a series of Ghost Bottlenecks. Each Ghost Bottleneck consists of two GhostModule stacks. The first Ghost Module is used to increase the feature dimension, while the second is used to reduce the feature dimension, and finally a 7×7×160 feature layer is obtained. Then a 1×1 convolution layer is used to adjust the number of channels to obtain a 7×7×960 feature layer, and then a global average pooling operation is performed. After the pooling is completed, a 1×1 convolution layer is used to adjust the number of channels. Finally, it is flattened and fully connected to complete the feature extraction of the image.

[0023] (2.2) At the same time, the SE attention module in the model is replaced by the SRM attention module to improve the recognition accuracy in complex natural scenes. The SRM attention module is mainly composed of Figure 3 As shown (where Figure 3The cross symbol in represents the generated weight vector multiplied and fused with the original feature map to complete feature recalibration). This module mainly consists of two parts: style pooling and style integration. Style pooling uses global average pooling and global standard deviation pooling to extract style information from each channel of the feature map, which can effectively extract image style information. After style pooling, SRM calculates the recalibration weight of each channel through style integration, and uses the style features obtained from style pooling to generate specific style weights, which eventually recalibrate the feature map to emphasize or suppress their information. The specific implementation of the style integration part is to pass the result of style pooling through a 1x1 convolution channel fully connected (CFC), and then apply batch normalization (BN) and sigmoid activation function to generate attention weights for each channel. The SRM attention module enhances the representation ability of CNN by adaptively recalibrating the intermediate feature map, only introduces a small number of parameters, has the characteristics of lightweight, and has better effect than SENet.

[0024] (2.3) The specific implementation method of the training process of the traffic sign detection network improved based on yolov5s is to output a large number of pictures of the training set of the CTSTD dataset to the improved yolov5s network for iterative training to obtain the optimal weight of the network model, and use the test set to test the improved yolov5s network configured with the optimal weight. The evaluation indicators are average precision AP and frame rate FPS.

[0025] The step (3) is traffic sign text detection. The detection network uses a text detection network improved based on DBNet to perform text detection on the input traffic sign image and extract the text area in the traffic sign.

[0026] The specific construction method of the traffic sign text detection network improved based on DBNet is as follows: (3.1) The DBNet text detection network is mainly composed of a feature extraction part, a feature fusion part, and a Head network. This invention mainly improves the feature extraction part and the feature fusion part of the network.

[0027] (3.2) The feature extraction part uses the ResNet18 residual network as the backbone network. In the conv3, conv4 and conv5 stages of the feature extraction network, all 3x3 convolutional layers are replaced with spatial and channel reconstruction convolution SCConv (Spatialand Channel reconstruction Convolution) to reduce the redundant information in the feature map, reduce the computational cost and the number of model parameters. The improved ResNet18 network structure is shown in the figure below: Figure 4As shown. The spatial and channel reconstruction convolution SCConv consists of a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU). The purpose of the SRU is to reduce spatial redundancy. It uses a separation-reconstruction method to achieve this goal. Specifically, the SRU first uses the scaling factor of group normalization to evaluate the information content in different feature maps, and then separates the feature maps with large information content from the feature maps with small information content, thereby reducing redundant features in the spatial dimension. The purpose of the CRU is to reduce channel redundancy. It uses a split-conversion-fusion method. The CRU first splits the input features into upper and lower layer features, then performs global convolution and point convolution transformations on the upper layer features, and performs similar transformations on the lower layer features. Finally, feature fusion is performed through adaptive average pooling and softmax function to obtain channel refined features.

[0028] (3.3) The feature fusion part adopts a mixed attention mechanism (CBAM) of spatial attention mechanism and channel attention mechanism to enhance feature expression. After the last upsampling and feature fusion operation of the network, a mixed attention mechanism of spatial and channel is added to improve the accuracy of the text detection network.

[0029] The specific implementation of the spatial attention mechanism is to first perform maximum pooling and average pooling operations on the feature maps output by the channel attention along the channel dimension to generate features of different context scales. Then, a splicing operation is performed along the channel dimension, specifically, two feature maps are spliced ​​through a 7×7 convolution layer with an input channel of 2 and an output channel of 1, and finally the generated spatial attention weights are generated. Finally, the channel attention weights are constrained between 0 and 1 through the Sigmoid activation function.

[0030] The specific implementation of the channel attention mechanism is to first perform global average pooling and global maximum pooling on the input feature map to generate two feature vectors containing the number of channels to obtain the global features of each channel. Then the above two feature vectors are input into a shared multi-layer perceptron to learn the attention weight of each channel. This operation mainly constructs a fully connected layer through 1×1 convolution to learn the weight of each channel. Finally, the channel attention weight is constrained between 0 and 1 through the Sigmoid activation function.

[0031] (3.4) The specific implementation method of the training process of the traffic sign text detection network improved based on DBNet is as follows: the traffic sign area image obtained by extracting the pictures in the CTSTD dataset through the traffic sign detection network is processed into a uniform size and input into the text detection network improved based on DBNet for training. After obtaining the optimal weight of the network model, the network performance is tested with the test set data. The evaluation indicators use F1 value and frame rate FPS to evaluate the detection accuracy.

[0032] The step (4) is traffic sign text recognition. The recognition network uses a text recognition network improved based on CRNN. ​​The text area after text detection is input into the recognition network to recognize the text content in the text area.

[0033] The specific construction method of the text recognition network improved based on CRNN is as follows: The improved text recognition network based on CRNN is divided into three stages: the convolution layer performs feature extraction, the recurrent layer performs learning prediction, and the transcription layer performs decoding classification. The present invention mainly improves the activation function of the convolution layer for feature extraction. The activation function in the original network uses the ReLU activation function, which makes it non-differentiable when the input is less than 0 during training, which may cause unstable gradients during training. Therefore, the present invention chooses to replace it with the Softplus activation function, which is defined as Softplus( x )=ln(1+ e x ), whose image is Figure 5 As shown. The Softplus activation function is smooth over the entire real domain, that is, its derivative is continuous. This helps to provide a more stable gradient during training, making the algorithm of the feature extraction part of the training and learning process more stable. The subsequent recurrent layer uses a bidirectional long short-term memory network (Bi-LSTM) to capture the contextual information in the sequence using two networks, the forward and backward networks, respectively, and then adds and splices the information in the two directions. After Bi-LSTM processing, a probability distribution is output for each time step. This distribution contains the predicted probabilities of all possible characters. These probability distributions constitute a posterior probability matrix, in which each row corresponds to a time step and each column corresponds to a character category. Finally, the transcription layer takes this posterior probability matrix as input and processes it using the CTC loss function. In CTC, an output sequence can correspond to multiple paths. The task of the transcription layer is to find the label sequence with the highest probability combination based on the prediction of each frame, that is, the maximum probability path. After CTC translation, the sequence feature information learned by the network is converted into the final recognition text.

[0034] The specific implementation method of the training process of the traffic sign text recognition network improved by CRNN is as follows: the traffic sign text area image extracted by the traffic sign text detection network improved by DBNet is processed to a uniform size and then input into the text recognition network improved by CRNN for training. After obtaining the optimal weight of the network model, the network performance is tested with the test set data. The evaluation index uses the text recognition accuracy to evaluate the detection. The calculation formula is: , Indicates the number of samples that are correctly identified. Represents the total number of samples in the test set.

[0035] Example 1 The implementation example of the present invention is implemented on a machine equipped with an Intel Core i9 central processor, an NVidia RTX 3060 graphics processor and 32 GB of memory, and three implementation cases are carried out for three improved networks respectively.

[0036] 1. The traffic sign detection network before improvement and the improved traffic sign detection network based on yolov5s were trained and tested. The data set used was the CTSTD data set. The learning rate used a dynamically changing learning rate, with the initial setting of 0.01, the batch size set to 32, and the epoch set to 100. The experimental results are shown in Table 1 below: Table 1 Model Original yolov5s network Improved yolov5s network AP (Average Precision) 79.5 83 FPS (frames per second) 74.8 85.3 The experimental results show that before the model was improved, the average accuracy AP was 79.5% and the frame rate FPS was 74.8. After the improvement, the average accuracy increased by 3.5 percentage points to 83%, and the inference speed was also improved, with the FPS increased to 85.3.

[0037] 2. The traffic sign text detection model before improvement and the improved traffic sign text detection model were trained and tested. The CTSTD data set was used as the data set. All traffic sign images were unified in size through preprocessing. The learning rate used a dynamically changing learning rate, with an initial setting of 0.005, a batch size of 16, and an epoch of 100. The experimental results are shown in Table 2 below: Table 2 Model Original DBNet Network Improved DBNet Network F1 value (harmonic mean of precision and recall) 83.4 86.1 FPS (frames per second) 39.5 40.7 The F1 value refers to the harmonic mean of precision and recall, and is an important comprehensive indicator for evaluating model performance in target detection tasks. It is defined as: ; Precision reflects the proportion of positive samples predicted correctly, which is defined as: ; Recall reflects the proportion of samples that are correctly detected in the actual positive samples, which is defined as: ; TP represents the number of samples that are actually positive and predicted as positive; FP represents the number of samples that are actually negative but predicted as positive; and FN represents the number of samples that are actually positive but predicted as negative. The experimental results show that the F1 value increased from 83.4% before the improvement to 86.1%, and the FPS increased from 39.5 before the improvement to 40.7. The accuracy and real-time performance of the improved network have been effectively improved.

[0038] 3. The traffic sign text recognition model before improvement and the improved traffic sign text recognition model were trained and tested. The CTSTD data set was used as the data set. All images of the traffic sign text area were unified in size through preprocessing. The learning rate was dynamically changed, with an initial setting of 0.001, a batch size of 16, and an epoch of 100. The experimental results are shown in Table 3 below: Table 3 Model Original DBNet Network Improved DBNet Network (Accuracy)Accuracy 73.7% 78% The experimental results show that the text recognition accuracy increased from 73.7% to 78%. The accuracy of the improved traffic sign text recognition network has been effectively improved.

[0039] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0040] It will be appreciated that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for detecting and recognizing traffic sign text for an unmanned driving system, characterized in that: The steps include: (1) Obtain images of traffic signs in natural scenes and construct a dataset for Chinese traffic sign text detection and recognition. Use the dataset to train the traffic sign detection network, text detection network, and text recognition network. (2) Construct a traffic sign detection network based on yolov5s, and use the traffic sign detection network based on yolov5s to process the input natural scene image and detect the area of ​​the traffic sign; (3) Construct a text detection network based on DBNet, and use the improved text detection network based on DBNet to perform text detection on the traffic sign image in the area of ​​the input traffic sign, and extract the text area in the traffic sign; (4) Construct a text recognition network based on the improved CRNN, and use the text recognition network based on the improved CRNN to input the text area after text detection into the recognition network to identify the text content in the text area.

2. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 1, characterized in that: In the step (1), a Chinese traffic sign text detection and recognition dataset is constructed, specifically by collecting and integrating traffic sign images with Chinese text on the Internet, annotating the traffic sign images in the dataset, including annotating the traffic signs, annotating the text areas, and annotating the text contents, and dividing the annotated images into a training set and a test set.

3. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 1, characterized in that: The specific construction method of the traffic sign detection network improved by yolov5s in step (2) includes the following sub-steps: (2.1) The backbone network in yolov5s is lightweighted, and the backbone network CSPDarknet53 of yolov5s is replaced with a lightweight GhostNet network; the SE attention mechanism module in the model is replaced with the SRM module to obtain an improved traffic sign detection network based on yolov5s; (2.2) Train the traffic sign detection network improved based on yolov5s. Output a large number of images of the training set in the CTSTD dataset to the improved yolov5s network for training to obtain the optimal weight of the network model. Output the images of the test set to the improved yolov5s network configured with the optimal weight for testing. The evaluation indicators of network performance are average precision AP and frame rate FPS.

4. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 1, characterized in that: The specific construction method of the text detection network improved based on DBNet in step (3) includes the following sub-steps: (3.1) Construct a text detection network. The improved DBNet network is used as the detection model in the text detection stage. The DBNet network includes feature extraction, feature fusion and Head network. The feature extraction part uses the ResNet18 residual network as the backbone network. In the conv3, conv4 and conv5 stages of the feature extraction network, all 3x3 convolutional layers are replaced with spatial and channel reconstruction convolution SCConv; for the feature fusion part, an improved spatial and channel mixed attention mechanism is used to enhance feature expression, that is, after the last step of upsampling and feature fusion operations, an attention mechanism CBAM that mixes the spatial attention mechanism and the channel attention mechanism is added; (3.2) Train the traffic sign text detection network. Extract the traffic sign area images from the images in the CTSTD dataset through the traffic sign detection network. After the traffic sign area images are processed to a uniform size, they are input into the text detection network improved based on DBNet for training. After obtaining the optimal weight of the network model, the network performance is tested using the test set data. The network performance is evaluated using the F1 value and the frame rate FPS.

5. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 4, characterized in that: The implementation process of the spatial attention mechanism in step (3.1) is as follows: first, the feature map output by the channel attention is subjected to maximum pooling and average pooling operations along the channel dimension to generate features of different context scales; Then a splicing operation is performed along the channel dimension. Specifically, the two feature maps are spliced ​​through a 7×7 convolution layer with an input channel of 2 and an output channel of 1, and finally the generated spatial attention weights are adjusted; finally, the channel attention weights are constrained between 0 and 1 through the Sigmoid activation function.

6. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 4, characterized in that: The implementation process of the channel attention mechanism in the step (3.1) is as follows: first, perform global average pooling and global maximum pooling on the input feature map to generate two feature vectors containing the number of channels to obtain the global features of each channel; then input the above two feature vectors containing the number of channels into a shared multi-layer perceptron to learn the attention weight of each channel. This operation is to construct a fully connected layer through 1×1 convolution to learn the weight of each channel; finally, the channel attention weight is constrained between 0 and 1 through the Sigmoid activation function.

7. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 1, characterized in that: The specific construction of the text recognition network improved based on CRNN in step (4) includes the following sub-steps: (4.1) Construct a text recognition network. The text recognition network uses an improved CRNN network as the recognition model. The CRNN network consists of a convolutional layer for feature extraction, a recurrent layer for learning and prediction, and a transcription layer for decoding and classification. Improve the activation function of the convolutional layer and replace the original activation function with the Softplus activation function. The Softplus activation function conforms to the biological model of neuron activation and avoids the gradient being 0 during training. (4.2) Train the traffic sign text detection network. The text area extracted in step (3) is processed into a uniform size and input into the text recognition network improved by CRNN for training. After obtaining the optimal weight of the network model, the network performance is tested with the test set data. The evaluation indicator text recognition accuracy is used to evaluate the recognition accuracy.

8. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 7, characterized in that: The Softplus activation function in step (4.1) is defined as Softplus( x )=ln(1+ e x ), which is smooth and has continuous derivatives over the entire real number domain.

9. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 7, characterized in that: In the step (4.1), the recurrent layer uses a bidirectional long short-term memory network Bi-LSTM to capture the contextual information in the sequence using the forward and backward networks respectively, and then fuses the information in the two directions by adding and splicing; after Bi-LSTM processing, a probability distribution will be output for each time step, which contains the predicted probabilities of all characters. These probability distributions constitute a posterior probability matrix, in which each row corresponds to a time step and each column corresponds to a character category; the transcription layer uses this posterior probability matrix as input and processes it using the CTC loss function. In CTC, one output sequence corresponds to multiple paths. The task of the transcription layer is to find the label sequence with the highest probability combination based on the prediction of each frame, that is, the maximum probability path; after CTC translation, the sequence feature information learned by the network is converted into the final recognition text.

10. The method for detecting and recognizing traffic sign text for an unmanned driving system according to claim 7, characterized in that: In the step (4.2), the evaluation index text recognition accuracy is used to evaluate the recognition accuracy, and its calculation formula is: , Indicates the number of samples that are correctly identified. Represents the total number of samples in the test set.

Images