A traffic sign recognition method

Abstract

The application relates to a traffic sign recognition method, a local channel weight calculation instead of global complex operation is carried out by an MLCA unit, a self-adaptive pooling selection unit matches a pooling mode on demand, while guaranteeing the feature extraction effect, and the redundant operation amount of the backbone network is effectively reduced. The neck network only fuses the key feature maps output by the backbone network, and repeated operation on the full-size image is avoided; the detection head focuses on the second feature map to directly output the result, and the overall operation amount is further controlled. The traffic sign recognition method has the beneficial effects of high operation efficiency of the backbone network and controllable overall process operation.

Description

Technical Field

[0001] This invention relates to the field of traffic sign technology, and in particular to a traffic sign recognition method. Background Technology

[0002] In intelligent transportation systems, traffic signs carry critical traffic information such as prohibition, warning, and instruction, and their recognition accuracy directly affects vehicle safety and traffic management efficiency. Traditional traffic sign recognition methods often rely on manually designed features (such as color, shape, and texture), which are prone to insufficient feature extraction and poor recognition robustness in complex scenarios (such as sudden changes in lighting, sign wear, and occlusion). Existing deep learning-based recognition methods suffer from shortcomings such as insufficient targeting of backbone network feature extraction and loss of details due to fixed pooling methods, making it difficult to meet the requirements of high-precision recognition. Therefore, there is an urgent need for a method that can optimize feature extraction and adapt to traffic sign recognition scenarios. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a traffic sign recognition method with superior backbone network computation efficiency and controllable overall process computation.

[0004] The technical solution of this invention is as follows:

[0005] A traffic sign recognition method includes a target recognition model training phase and a target recognition phase, with the following specific steps:

[0006] Target recognition model training phase:

[0007] Various traffic sign images are acquired and indexed to form a training dataset. The images in the training dataset are preprocessed to obtain preprocessed images as model input.

[0008] Construct a target recognition model, which includes, in sequence:

[0009] The backbone network module integrates an MLCA unit and an adaptive pooling selection unit. The MLCA unit is used to calculate local channel weights on the received initial feature map. The adaptive pooling selection unit selects local average pooling or global average pooling on the initial feature map according to the weight of each channel to obtain several first feature maps.

[0010] The neck network module is used to fuse the first feature maps to form several second feature maps;

[0011] The detection head module is used to extract target-related information from each second feature map and output target detection results that meet the accuracy requirements.

[0012] Target identification stage:

[0013] The collected traffic signs are input into the trained target recognition model to obtain the traffic sign recognition results.

[0014] Preferably, the neck network module is a bidirectional feature pyramid network, which adopts a dynamic weighted fusion strategy for the fusion process of each first feature map, and realizes cross-scale feature integration through bidirectional interactive paths from bottom to top and from top to bottom to generate the second feature map.

[0015] In any of the above schemes, it is preferred that the neck network preprocesses each first feature map through a dynamic downsampling unit before transmitting each first feature map to the bidirectional feature pyramid network. The preprocessing process of the dynamic downsampling unit is as follows: adaptive weights are generated by learning, and the input first feature maps are weighted using the adaptive weights.

[0016] The preprocessed first feature maps are then transmitted to a bidirectional feature pyramid network for further processing.

[0017] In any of the above schemes, the preferred implementation process of the dynamic downsampling is as follows:

[0018] S 11 A weight map of the same size as the input is generated by stacking a 1 × 1 convolution with a sigmoid function.

[0019] S 22 The pixels are weighted and summed within a sampling window with a stride of 2;

[0020] The formula for calculating the downsampled output is:

[0021]

[0022] Where N(i,j) is a 2 × 2 window centered at (i,j).

[0023] In any of the above schemes, it is preferred that the learning method of the downsampling unit is any one of the following: a learning method based on convolutional layers, a learning method based on attention mechanisms, or a learning method combined with a loss function.

[0024] In any of the above schemes, it is preferred that the detection head module, when performing the target position regression task, uses the Inner-Slou loss function to calculate the deviation between the predicted bounding box and the true bounding box, and the Inner-Slou loss function explicitly introduces a shape constraint factor R. S .

[0025] In any of the above schemes, the Inner-Slou loss function is preferably expressed as follows:

[0026] L Inner-Slou=1-Iou(1- )

[0027] Where w p h p To predict the width and height of the bounding box, w t h t This represents the actual frame width and height.

[0028] In any of the above schemes, it is preferred that the total model loss calculated by the detection head module is in the form of a multi-loss fusion, including target category loss, target location regression loss, and confidence loss, wherein the target location regression loss adopts the Inner-Slou loss function, and the fusion expression of the total loss is:

[0029] L total =λ1 . L cls +λ2 . L Inner-Slou +λ3 . L conf

[0030] In the formula, L represents the total loss of the model. cls For the target category loss, L Inner-Slou For the target location regression loss based on the Inner-Slou function, L conf Let λ1, λ2, and λ3 be the target confidence loss, and let λ1 + λ2 + λ3 = 1.

[0031] In any of the above schemes, it is preferred that, during the target recognition model training phase, various traffic sign images are acquired, including collecting RGB three-channel color images containing various traffic signs under different weather and lighting conditions.

[0032] In any of the above schemes, it is preferred that during the target recognition model training stage, enhancement operations are performed on the acquired traffic sign images. The enhancement operations employ at least one of random brightness adjustment, random contrast adjustment, and random flipping, and the parameters of each operation are randomly generated by a preset probability distribution.

[0033] The traffic sign recognition method of the present invention replaces the complex global calculation with local channel weight calculation by using MLCA unit, and the adaptive pooling selection unit matches the pooling mode as needed. While ensuring the feature extraction effect, it effectively reduces the redundant computation of the backbone network.

[0034] The neck network fuses only the key feature maps output by the backbone network, avoiding repeated computation on the full-size image; the detection head focuses on the second feature map and directly outputs the result, further controlling the overall computational load. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the traffic sign recognition method of the present invention;

[0036] Figure 2 This is a performance comparison chart of the target detection model under different improved strategies of the present invention. Detailed Implementation

[0037] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Example 1:

[0039] This invention relates to the field of computer vision and intelligent transportation technology, specifically applied to the automatic recognition of signs in traffic scenarios. It can serve systems such as autonomous driving, intelligent traffic monitoring, and vehicle driver assistance, and falls under the category of target detection and recognition technology.

[0040] The traffic sign recognition method of the present invention includes two core stages: a target recognition model training stage and a target recognition stage. The specific implementation steps of each stage are as follows:

[0041] The core of the target recognition stage is to build a training dataset, complete image preprocessing, and build a target recognition model with a specific backbone network, so as to provide a mature model foundation for subsequent recognition stages.

[0042] In the training dataset preparation and image preprocessing steps, the traffic sign images should come from at least the following sources: 1. Real-time dynamic images captured by fixed road surveillance cameras (such as urban roads and highway monitoring); 2. Images captured by vehicle-mounted cameras under different driving conditions (low speed, high speed, turning); 3. Annotated images from publicly available standard traffic sign datasets (such as GTSRB, TT100K, etc.); 4. Synthetic images generated by simulating extreme scenarios (such as heavy rain, dense fog, backlight). Ensure that the images cover all categories of traffic signs, including "prohibition," "warning," "indication," and "direction," and include samples of different sizes and states (new signs, worn signs, partially occluded signs) to guarantee the diversity and generalization of the dataset.

[0043] In order to further simulate complex real-world scenarios and enhance robustness, various traffic sign images can be enhanced. The enhancement operation adopts at least one of random brightness adjustment, random contrast adjustment, and random flipping, and the parameters of each operation are randomly generated by a preset probability distribution.

[0044] During image indexing, professional annotation tools such as LabelImg and CVAT can be used to manually index the acquired images. To meet the input requirements for model training, the annotation content must clearly indicate the "category label" of the traffic sign (such as "No Left Turn" or "Caution Pedestrians"). In the specific annotation process, the "true bounding box" of the traffic sign is annotated based on the image pixel coordinates (usually represented by the x / y coordinates of the top-left and bottom-right corners of a rectangle), ensuring that the bounding box accurately covers the sign area without any extra background or missing sign content. After indexing is completed, a training dataset corresponding to the "image file + annotation format file" is formed.

[0045] The purpose of image preprocessing is to eliminate interference factors during image acquisition, unify the model input format, and improve model training stability and feature extraction efficiency. Specific methods include: 1. Size normalization: Adjusting all images in the training dataset to a uniform resolution to avoid feature calculation dimension confusion during model training due to differences in image size.

[0046] 2. Pixel value normalization: The RGB channel value of each pixel in the image, i.e. (0-255), is mapped to the range of [0,1]. The calculation method is "pixel value ÷ 255". By reducing the numerical range, the computational load of the model is reduced, and the gradient explosion problem caused by excessively large values ​​is avoided.

[0047] If the dataset has a small sample size or a single scene, random image enhancement can be added to increase sample diversity, such as "random horizontal flip" (simulating the difference in the left and right driving perspective of a vehicle) and "brightness fine-tuning" (adapting to changes in day and night lighting based on the original brightness).

[0048] In the process of building the target recognition model, the core structure of the target recognition model includes, in sequence, the backbone network module, the neck network module, and the detection head module. The functions and implementation details of each module are as follows:

[0049] The backbone network module can adopt the lightweight C2f module based on YOLOv8 in the existing technology, which integrates MLCA unit and adaptive pooling selection unit to extract features from the input image during the preprocessing process and form several first feature maps.

[0050] Specifically, after receiving the preprocessed image, the backbone network module extracts the "initial feature map" of the image through multi-layer convolution, activation, pooling, and other operations. Then, it optimizes the feature representation through MLCA units and adaptive pooling selection units, and finally outputs several "first feature maps".

[0051] The MLCA unit (Multi-Local Channel Attention Unit) replaces multi-head self-attention with a combination of "local average pooling + global average pooling + 1 × 1 convolution," introducing only a minimal number of convolution operations in key layers. This reduces the high floating-point overhead of the Query-Key-Value dot product and highlights small target features through channel reweighting, resulting in an increase of less than 1% in parameter count and FLOPs. Specifically, the initial feature map output by a layer of the backbone network (its dimensions are typically represented as "C × H × W," where C is the number of channels, H is the feature map height, and W is the feature map width) is... In the local channel weight calculation process, the initial feature map is first divided into multiple local channel groups according to the channel dimension. For each local channel group, the local mean of all channels in the group is calculated (i.e., the average pixel value of each channel, reflecting the feature intensity of that channel). A small fully connected layer (with the local mean dimension as input and the same number of channels as the local channel group as output) and a sigmoid activation function are used to generate the "channel weight vector" of the local channel group (the weight value ranges from 0 to 1, representing the importance of the corresponding channel). The channel weight vector is multiplied "channel by channel" with the feature map of the original local channel group to obtain the weighted local channel group features. All weighted local channel group features are concatenated to output the feature map processed by the MLCA unit.

[0052] The adaptive pooling selection unit dynamically selects the pooling method based on the channel weights output by the MLCA unit, preserving key feature details. Specifically, before the pooling process, the "weight mean" of each channel in the feature map output by the MLCA unit is calculated based on the channel weights, and a preset threshold is set. If the weight mean of a channel is greater than or equal to the threshold, it is determined to be a "high-importance channel" (this channel contains key details of traffic signs, such as the text and image edges of the signs); if the weight mean is less than the threshold, it is determined to be a "low-importance channel" (this channel mostly contains background or redundant features). This reduces the computational load during the pooling process while ensuring accurate recognition.

[0053] The neck network module receives several first feature maps output by the backbone network and integrates feature information at different scales through feature fusion techniques (such as feature splicing, weighted summation, etc.) to make up for the deficiencies of single-scale features and generate "second feature maps" to make the feature expression richer.

[0054] During the fusion process, it is necessary to ensure that the dimensions of the first feature maps at different scales are matched. The specific fusion method can be selected according to the model training requirements. The core goal is to make the second feature map contain both the "detail features of small-scale identifiers" and the "semantic features of large-scale identifiers".

[0055] The detection head module is used to receive the second feature map output by the neck network and extract the category information of traffic signs through classification and regression calculations.

[0056] Specifically, convolutional layers and activation functions are used to predict the category of each "candidate anchor point" on the second feature map, outputting the probability that the anchor point belongs to a certain type of traffic sign. The offset between the candidate anchor point and the ground truth bounding box is output by the convolutional layer to correct the anchor point's position, resulting in a "predicted bounding box". The predicted bounding boxes are then filtered, retaining only detection results with a category probability ≥ a preset accuracy threshold, ultimately outputting the traffic sign detection results that meet the accuracy requirements.

[0057] In the target recognition stage, based on a well-trained target recognition model, the actual traffic signs are identified. The specific steps are as follows:

[0058] Traffic sign image acquisition: Traffic scene images are acquired in real time through vehicles, road surveillance cameras, etc. The acquisition resolution is consistent with the resolution of the pre-processed images during the training phase to avoid size differences affecting recognition accuracy.

[0059] Perform the same preprocessing operations on the acquired real-time images as during the training phase to ensure that the image format input to the model is consistent with the training data.

[0060] The preprocessed real-time image is input into the "trained target recognition model". The model sequentially goes through the feature extraction of the backbone network, the feature fusion of the neck network, and the information extraction of the detection head, and finally outputs the recognition result of the traffic sign in the image.

[0061] Example 2:

[0062] Based on Example 1, to improve the accuracy of feature fusion, the neck network module in this example adopts a bidirectional feature pyramid network. Specifically, the bottom-up path corresponds to the "forward output of the feature extraction network," and the extracted first feature maps are sorted by scale from bottom to top and denoted as P1, P2, P3, ..., P... n Each feature map layer retains the original detailed information at that scale, such as P1 retaining pixel-level details of the image and P4 retaining target category-level semantics.

[0063] Top-down path: Starting from the first feature map with the largest scale, upsampling is used to reduce its scale to be similar to the feature map of the previous layer. Figure 1 This process ensures that high-level semantic information is preserved; then, "feature alignment" is performed with the first feature map of the same scale in the bottom-up path to ensure spatial location matching, preparing for subsequent fusion.

[0064] Example 3:

[0065] Based on Example 2, to reduce the computational load during the fusion process and improve the fusion efficiency of the neck network, the neck network preprocesses each first feature map through a dynamic downsampling unit before transmitting it to the bidirectional feature pyramid network. The preprocessing process of the dynamic downsampling unit involves: generating adaptive weights through learning; using these adaptive weights to weight the input first feature maps; and then transmitting the preprocessed first feature maps to the bidirectional feature pyramid network for processing. This weakens redundant features such as background (low-weight regions) and retains effective features (high-weight regions), avoiding invalid features from occupying BiFPN's computational resources. Specifically, this is achieved by generating a weight map of the same size as the input through a 1 × 1 convolution superimposed with a sigmoid function, followed by weighted summation of pixels within a sampling window with a stride of 2. The calculation formula for the downsampled output is:

[0066]

[0067] Where N(i,j) is a 2 × 2 window centered at (i,j).

[0068] The learning method of the downsampling unit can be any one of the following: learning method based on convolutional layers, learning method based on attention mechanism, or learning method combined with loss function.

[0069] Example 4:

[0070] Building upon Examples 1-3, to accurately constrain the bounding box shape, reduce regression bias, and improve robustness in complex backgrounds, this example introduces an Inner-Slou loss function when performing target location regression. This function calculates the deviation between the predicted bounding box and the true bounding box. Furthermore, the Inner-Slou loss function explicitly incorporates a shape constraint factor R. S .

[0071] Specifically, the first step is to calculate the basic Inner-Slou value, including: 1. Defining the coordinates and width / height parameters of the predicted bounding box and the ground truth bounding box, and determining the minimum inscribed rectangle that can simultaneously contain both. 2. Calculating the intersection area of ​​the predicted bounding box and the ground truth bounding box, and the minimum inscribed rectangle. 3. Using the ratio of the intersection area of ​​the predicted bounding box and the ground truth bounding box to the minimum inscribed rectangle, the basic Inner-Slou value is obtained, and the bounding box position constraint benchmark is constructed.

[0072] Step 2: Calculate the shape constraint factor, including: 1. Calculate the aspect ratio of the predicted bounding box and the actual bounding box respectively. 2. Based on the absolute deviation of the aspect ratio, calculate the shape constraint factor through a preset exponential function formula to achieve the quantitative constraint that "the greater the shape deviation, the greater the shape constraint factor value", and construct the bounding box shape constraint term.

[0073] Step 3: Construct the loss function and optimize the model, including: 1. Weighting and fusing the basic Inner-Slou values ​​with the weight coefficients to obtain the final Inner-Slou loss function. 2. During training, using this loss function as the optimization objective, updating the network parameters through the backpropagation algorithm to minimize the loss value and achieve synchronous and accurate regression of the bounding box position and shape.

[0074] Wherein: the expression for the Inner-Slou loss function is:

[0075] L Inner-Slou =1-Iou(1- )

[0076] Where w p h p To predict the width and height of the bounding box, w t h t This represents the actual frame width and height.

[0077] The total model loss calculated by the detection head module is a multi-loss fusion form, including target category loss, target location regression loss, and confidence loss. The target location regression loss employs the Inner-Slou loss function. The fusion expression for the total loss is:

[0078] L total =λ1 . L cls +λ2 . L Inner-Slou +λ3 . L conf

[0079] In the formula, L represents the total loss of the model. cls For the target category loss, L Inner-Slou For the target location regression loss based on the Inner-Slou function, L conf Let λ1, λ2, and λ3 be the target confidence loss, and let λ1 + λ2 + λ3 = 1.

[0080] The above-described embodiments are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A traffic sign recognition method characterized by, The process includes the target recognition model training phase and the target recognition phase, with the following specific steps: Target recognition model training phase: Various traffic sign images are acquired and indexed to form a training dataset. The images in the training dataset are preprocessed to obtain preprocessed images as model input. Construct a target recognition model, which includes, in sequence: The backbone network module integrates an MLCA unit and an adaptive pooling selection unit. The MLCA unit is used to calculate local channel weights on the received initial feature map. The adaptive pooling selection unit selects local average pooling or global average pooling on the initial feature map according to the weight of each channel to obtain several first feature maps. The neck network module is used to fuse the first feature maps to form several second feature maps; The detection head module is used to extract target-related information from each second feature map and output target detection results that meet the accuracy requirements. Target identification stage: The collected traffic signs are input into the trained target recognition model to obtain the traffic sign recognition results.

2. The traffic sign recognition method of claim 1, wherein, The neck network module is a bidirectional feature pyramid network. Its fusion process for each first feature map adopts a dynamic weighted fusion strategy. It achieves cross-scale feature integration through bidirectional interactive paths from bottom to top and from top to bottom to generate the second feature map.

3. The traffic sign recognition method of claim 2, wherein, Before transmitting each first feature map to the bidirectional feature pyramid network, the neck network first preprocesses each first feature map through a dynamic downsampling unit. The preprocessing process of the dynamic downsampling unit is as follows: adaptive weights are generated by learning, and the input first feature maps are weighted using the adaptive weights. The preprocessed first feature maps are then transmitted to a bidirectional feature pyramid network for further processing.

4. The traffic sign recognition method of claim 2, wherein, The implementation process of the dynamic downsampling is as follows: S 11 A weight map of the same size as the input is generated by a 1 x 1 convolution superimposed with a sigmoid. S 22 Weighted sum of pixels within a sampling window of stride 2; The formula for calculating the downsampled output is: where, where N(i,j) is a 2 x 2 window centered at (i,j).

5. The traffic sign recognition method of claim 3, wherein, The learning method of the downsampling unit can be any one of the following: learning method based on convolutional layers, learning method based on attention mechanism, or learning method combined with loss function.

6. The traffic sign recognition method of claim 1, wherein, The detection head module adopts an Inner-Slou loss function to calculate the deviation between a predicted bounding box and a real bounding box when performing a target position regression task, and a shape constraint factor R is explicitly introduced in the Inner-Slou loss function S .

7. The traffic sign recognition method of claim 6, wherein, The expression for the Inner-Slou loss function is: L Inner-Slou =1-Iou(1- ) Where w p h p To predict the width and height of the bounding box, w t h t This represents the actual frame width and height.

8. The traffic sign recognition method of claim 7, wherein, The total model loss calculated by the detection head module is a multi-loss fusion form, including target category loss, target location regression loss, and confidence loss. The target location regression loss employs the Inner-Slou loss function. The fusion expression for the total loss is: L total =λ1∙L cls +λ2∙L Inner-Slou +λ3∙L conf where L is the total loss of the model cls L is the target class loss Inner-Slou L is the target position regression loss based on the Inner-Siou function conf L is the target confidence loss, λ1, λ2, and λ3 are weight coefficients, and λ1+λ2+λ3=1.

9. The traffic sign recognition method of claim 1, wherein, During the target recognition model training phase, various traffic sign images are acquired, including RGB three-channel color images containing various traffic signs under different weather and lighting conditions.

10. The traffic sign recognition method of claim 9, wherein, During the target recognition model training phase, enhancement operations are performed on the acquired traffic sign images. The enhancement operations employ at least one of random brightness adjustment, random contrast adjustment, and random flipping, and the parameters of each operation are randomly generated by a preset probability distribution.

Images