Traffic sign identification method and device, and storage medium
By building a traffic sign recognition model based on the YOLOv5s architecture and utilizing high-resolution shallow feature maps and UPPANet feature fusion, the accuracy and missed detection rate issues of small-size traffic sign recognition under low computing power conditions were solved, achieving efficient traffic sign recognition.
Patent Information
- Application Number
- CN202510887400.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-09-26
AI Technical Summary
Existing technologies have difficulty achieving high-precision recognition of small-sized traffic signs under low computing power conditions, and there are occlusions and blur interferences, resulting in low recognition accuracy, high missed detection rate, and high computational complexity.
A traffic sign recognition model based on the YOLOv5s architecture is constructed. By extracting high-resolution shallow feature maps in the lightweight residual structure module of the backbone network, and using the UPPANet structure for feature fusion, combined with a small target detection head, the recognition capability is improved.
It achieves real-time, high-precision recognition of small targets on resource-constrained embedded platforms, improving recognition accuracy and reducing missed detection rates. It is suitable for mid- and low-end fuel vehicles and entry-level new energy vehicles.
Smart Images
Figure CN120708191A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision target detection, and in particular to a traffic sign recognition method, device and storage medium. Background Art
[0002] In intelligent transportation and assisted driving systems, traffic sign recognition is a critical component for ensuring driving safety and decision-making accuracy. Current mainstream object detection algorithms, such as YOLOv5, have been widely used for traffic sign recognition. However, due to the computing power constraints of embedded hardware platforms (e.g., no more than 2TOPS), achieving high-precision recognition under low computing power conditions still faces the challenge of recognizing small traffic signs. Many traffic signs occupy a relatively small pixel area in the camera image, and are subject to interference such as occlusion and blur, resulting in low recognition accuracy for conventional multi-scale detection mechanisms. Therefore, improvements to existing technologies are needed.
[0003] The above information is presented as background information only to assist with an understanding of the present disclosure and is not a determination or admission that any of the above may be applicable as prior art with respect to the present disclosure. Summary of the Invention
[0004] The present invention provides a traffic sign recognition method, device and storage medium to solve the problems existing in the prior art.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A traffic sign recognition method, comprising:
[0007] Build a traffic sign recognition model based on the YOLOv5s architecture; the traffic sign recognition model includes a backbone network, a feature fusion network, multiple original detection heads, and a small target detection head;
[0008] Extracting a shallow feature map from the first lightweight residual structure module of the backbone network;
[0009] Inputting the shallow feature map into the feature fusion network, fusing it with other scale feature maps, and generating a fused feature map;
[0010] The fused feature map is input into the small target detection head for processing, and the output detection result is used together with the detection results of the multiple original detection heads to identify traffic signs.
[0011] Optionally, the backbone network is a YOLOv5s backbone network, and the shallow feature map is an output feature map of the first lightweight residual structure module in the backbone network.
[0012] Optionally, the spatial resolution of the fused feature map is higher than the maximum value of the spatial resolutions of the multiple original detection head input feature maps.
[0013] Optionally, the feature fusion network includes multiple fusion levels for realizing multi-level fusion of feature maps of different scales.
[0014] Optionally, the fusion levels include an intermediate scale, a top scale, and a bottom scale;
[0015] The intermediate-scale feature maps are connected by weighted fusion;
[0016] The feature maps of the top scale and the bottom scale are connected through a channel splicing operation;
[0017] The weight parameters of the weighted fusion are learned during the network training process.
[0018] Optionally, the feature fusion network is a UPPANet structure, and the UPPANet structure is composed of a feature pyramid network and a path aggregation network.
[0019] Optionally, a residual structure with an attention mechanism is formed in one or more lightweight residual structure modules in the backbone network, and the residual structure with an attention mechanism is constructed by introducing a polarized self-attention mechanism module between its splicing operation and addition operation.
[0020] Optionally, the small target detection head is used to receive the fused feature map, extract features through a convolutional layer, and output detection box categories and position parameters.
[0021] The present invention also provides a traffic sign recognition device, comprising a memory and a processor, wherein the processor is configured to execute any of the traffic sign recognition methods described above.
[0022] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute any of the traffic sign recognition methods described above.
[0023] Compared with the prior art, the present invention has the following beneficial effects:
[0024] The present invention provides a traffic sign recognition method, device, and storage medium. By extracting high-resolution shallow feature maps in the shallow lightweight residual structure module of the backbone network, guiding the cross-scale transmission of features at different levels, and inputting the fused feature maps into the small target detection head, the method, device, and storage medium can achieve real-time and high-precision recognition of small targets on resource-constrained embedded platforms.
[0025] The present invention has other features and advantages that will be apparent from or will be described in detail in the accompanying drawings and the following detailed description incorporated herein, which together serve to explain certain principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0027] Figure 1 is a flow chart of a traffic sign recognition method provided by an embodiment of the present invention;
[0028] Figure 2 is a network structure diagram of a traffic sign recognition model constructed in a traffic sign recognition method provided by an embodiment of the present invention;
[0029] Figure 3 1 is a diagram of a C3 module before and after optimization involved in a traffic sign recognition method provided by an embodiment of the present invention;
[0030] Figure 4 It is a structural diagram of a weighted bidirectional feature pyramid network in the prior art;
[0031] Figure 5 This is a diagram of an optimized feature fusion network structure in a traffic sign recognition method provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects achieved by this application, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application and are therefore only examples and are not intended to limit the scope of protection of this application.
[0033] References to "embodiments" herein mean that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the word "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the various technical features mentioned in the embodiments can be combined in any manner to form a corresponding implementable technical solution.
[0034] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms herein is only for describing specific embodiments and is not intended to limit this application.
[0035] In the description of this application, the term "and / or" is used to describe a logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and both A and B exist. In addition, the character " / " in this document generally indicates that the objects before and after are in a logical "or" relationship.
[0036] In this application, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, priority or sequence relationship between these entities or operations.
[0037] Without further limitations, in this application, the words "include", "comprise", "have" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those defined elements, but also other elements not explicitly listed, or elements inherent to such process, method or product.
[0038] Consistent with the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceed" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise clearly and specifically limited.
[0039] In the description of the embodiments of the present application, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, it should not be understood as a limitation on the embodiments of the present application.
[0040] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms "installed", "connected", "connected", "fixed", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art of the present application, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.
[0041] Currently, existing traffic sign recognition methods have the following problems when deployed on low-computing vehicle platforms such as mid- and low-end fuel vehicles and entry-level new energy vehicles:
[0042] 1. Low recognition accuracy. For example, for a certain model of vehicle from a certain manufacturer, the accuracy of logo classification in third-party test data was 79.4%.
[0043] 2. The missed detection rate is high. For example, for a certain model of vehicle from a certain manufacturer, the measured recall rate in rainy and foggy weather conditions is only 61.2%.
[0044] 3. High computational complexity and large memory usage lead to deployment difficulties. Traffic sign detection and recognition (TSR) algorithms deployed on mid- and low-end fuel vehicles are limited by cost (low-computing-power ECU, monocular camera) and real-time requirements (<100ms delay), resulting in significant performance differences from high-end smart vehicles.
[0045] To solve the aforementioned problems, the present invention aims to provide a traffic sign recognition solution with low computational complexity and high recognition accuracy, which is suitable for embedded deployment and compatible with resource-constrained on-board computing platforms such as mid- and low-end fuel vehicles and entry-level new energy vehicles.
[0046] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0047] Please refer to Figure 1 and Figure 2 , an embodiment of the present invention provides a traffic sign recognition method. For example, the method is applicable to a low-computing-power vehicle-mounted platform with a computing power not exceeding 2TOPS.
[0048] In this embodiment, the traffic sign recognition method includes:
[0049] S1. Build a traffic sign recognition model based on the YOLOv5s architecture. The traffic sign recognition model includes a backbone network, a feature fusion network, multiple original detection heads, and a small target detection head.
[0050] Please refer to Figure 2 , which is a network structure diagram of the constructed traffic sign recognition model. In this embodiment, a small target detection head is added based on the YOLOv5s network to recognize traffic signs in low-resolution images.
[0051] Specifically, the small object detection head structure is similar to that of YOLO's original detection head. Drawing on its structure, it generally includes three standard convolutional layers and an output layer, responsible for feature extraction, prediction box generation, and outputting category and location information. This structure efficiently processes responses from the fused feature map, improving detection performance.
[0052] In this embodiment, the small target detection head is specifically used to receive the fused feature map, classify and locate the response results output by the fused feature map, and extract the regional response through a multi-layer convolution module after receiving the fused feature map. Its output includes a bounding box of category and positioning, which is used to improve the detection accuracy of low-resolution or long-distance traffic signs.
[0053] Furthermore, the backbone network is the YOLOv5s backbone network, and the shallow feature map is the output feature map of the first lightweight residual structure module (C3 module) in the backbone network.
[0054] It's understandable that the C3 module is the fundamental unit for extracting spatial structural information in YOLOv5. Extracting features through the first C3 module in the backbone network fully preserves the fine-grained information of the original image, making it particularly suitable for identifying and localizing small objects. The first C3 module in the backbone network extracts spatial details from the first few layers of the original image. Located at the shallowest layer of the backbone in the YOLOv5 architecture, its output feature map is used to preserve detailed information about small objects, including rich spatial details such as edges and textures, making it particularly important for identifying object edges in complex backgrounds.
[0055] Furthermore, the spatial resolution of the fused feature map is higher than the maximum spatial resolution of the input feature maps of multiple original detection heads. Its spatial resolution is specifically 160×160, which is higher than the highest input resolution (80×80) of the original three detection heads of YOLOv5s, and is more conducive to locating traffic sign areas with a small pixel ratio.
[0056] Based on this, the use of high-resolution feature maps can expand the receptive field, improve the distinguishability of small-sized targets in the feature maps, and help improve the accuracy of detection responses.
[0057] Since the shallow feature map output by the first C3 module of the backbone network has a higher spatial resolution, after being fused through the feature fusion network (Neck network), a 160×160 fused feature map is obtained. When these fused feature maps are further used for small target detection head processing, the recognition ability of small-sized traffic signs can be improved and can adapt to the deployment requirements of vehicle-mounted terminals not exceeding 2TOPS.
[0058] S2. Extract shallow feature maps from the first lightweight residual structure module of the backbone network.
[0059] In some optional embodiments, one or more lightweight residual structure modules (C3 modules) in the backbone network form a residual structure with an attention mechanism (Polarized Self-Attention, PSA). The residual structure with an attention mechanism is constructed by introducing a polarized self-attention mechanism module between its concatenation operation (Concat) and addition operation (Add), that is, a C3PSA structure.
[0060] Please refer to Figure 3 ,in Figure 3 (a) is the C3 module diagram before optimization. Figure 3 (b) is a diagram of the optimized C3 module in this embodiment. By embedding the polarized self-attention mechanism within the C3 structure of YOLOv5s, this embodiment enhances the network's robustness in complex environments such as those with drastic lighting changes and partially occluded targets, and mitigates false detections caused by background interference. This makes it particularly suitable for recognition tasks in occluded or low-contrast scenarios. Because it is inserted between the Concat and Add functions in the original structure, the resulting C3PSA structure enhances the model's responsiveness to key features, making it particularly effective in scenarios with occlusion and complex lighting.
[0061] S3. Input the shallow feature map into the feature fusion network and fuse it with other scale feature maps to generate a fused feature map.
[0062] In this embodiment, the feature fusion network is a UPPANet structure.
[0063] The existing YOLOv5s Neck network uses a combined FPN and PANet architecture for feature fusion. While this structural combination significantly enhances the network's feature fusion capabilities, it exponentially increases the number of network model parameters. Furthermore, the input features of the PANet module are derived solely from the output of the FPN processing, failing to fully utilize the initial feature information of the backbone network. This lack of original features can cause feature bias during training, negatively impacting detection performance.
[0064] In order to make full use of the original feature information of the backbone network, the Google Brain research team optimized and improved the PANet architecture in 2020 and proposed a weighted bidirectional feature pyramid network (BiFPN, Bidirectional Feature Pyramid Network). Its structure is as follows Figure 4 Compared to PANet, BiFPN utilizes three key optimizations: First, it reduces model complexity and computational overhead by eliminating single-input edge nodes; second, it introduces skip connections between input and output nodes at the same level to enhance feature interaction; and third, it uses a learnable feature weighting mechanism to dynamically assess the importance of each feature, thereby improving feature fusion efficiency. Although the BiFPN network can more fully utilize feature information at different levels, the introduction of this weighting mechanism also significantly increases the number of model parameters.
[0065] Therefore, in order to effectively utilize the feature information of the backbone network without increasing the network parameters as much as possible, this embodiment optimizes the PANet network based on the idea of BiFPN cross-scale fusion. In addition, in order to match the detection layer of FPN and Prediction part, this embodiment deepens the 3-layer FPN of YOLOv5s to 4 layers. The optimized feature fusion network structure is as follows Figure 5 As shown, it is defined as UPAN feature fusion network below.
[0066] In this embodiment, improvements are made to the BiFPN fusion network. The resulting UPPANet architecture combines a Feature Pyramid Network (FPN) with a Path Aggregation Network (PANet). This UPPANet architecture implements cross-level information transfer by adding fusion paths. By combining the FPN pyramid and PAN path aggregation mechanisms, UPPANet introduces more upsampling and downsampling paths during the information fusion process. It also employs a lightweight weighting strategy to ensure high cross-scale transfer efficiency and a low parameter count, making it suitable for deployment on low-computing platforms.
[0067] Based on this, the UPPANet fusion structure in this embodiment is lighter than BiFPN. Combining the advantages of FPN and PAN, it uses shallow features to guide deep semantics, effectively improving adaptability to multi-scale targets. The UPPANet structure uses multi-level fusion of feature maps extracted from the backbone, and a lightweight weighting strategy ensures high cross-scale transfer efficiency and low parameter count, making it suitable for deployment on low-computing platforms.
[0068] In some optional embodiments, the feature fusion network includes multiple fusion levels for implementing multi-level fusion of feature maps of different scales, and the fusion levels include an intermediate scale, a top scale, and a bottom scale.
[0069] By efficiently aggregating features of different scales through the UPPANet feature fusion network, we can construct a fused feature map that combines spatial details and semantic expression. This map is then fed into the newly added small target detection head for detection, effectively improving the recognition accuracy of small-sized traffic signs in complex backgrounds.
[0070] Specifically, in this embodiment, feature maps of intermediate scales are connected through weighted fusion; feature maps of top scales and bottom scales are connected through channel splicing operations; wherein the weight parameters of weighted fusion are learned during the network training process.
[0071] like Figure 5 As shown in the figure, in the fusion structure, feature weights are transferred between layers P4 and P5 through weighted fusion, and channel cascade splicing is used between layers P3 (bottom layer) and P6 (top layer). A mean summation method is used throughout the bottom-to-top fusion path to ensure the effective fusion of multi-layer semantic information. By using weighted fusion for the intermediate layers, the influence of features from different layers is dynamically adjusted, improving the fusion effect. Since the top layer (P6) and the bottom layer (P3) have significantly different features, channel splicing is used to preserve their integrity.
[0072] S4. Input the fused feature map into the small target detection head for processing, and the output detection result is used together with the detection results of multiple original detection heads to recognize traffic signs.
[0073] Compared with the original YOLOv5s, this embodiment effectively improves the detection accuracy of small-sized traffic signs by adding a small target detection head and a fusion path while ensuring lightweight deployment of the model. It is particularly suitable for vehicle-mounted terminals with computing resources not exceeding 2TOPS.
[0074] The present invention also provides a traffic sign recognition device, comprising a memory and a processor, wherein the processor is configured to execute the traffic sign recognition method as described in the above embodiment.
[0075] The present invention also provides a computer-readable storage medium storing a computer program. When the program is executed by a processor, the processor executes the traffic sign recognition method as described in the above embodiment.
[0076] To verify the effectiveness of the present invention, an experiment was conducted on a traffic sign dataset. The experimental results showed that the proposed network model achieved 4.5% and 5.2% higher precision (P) than the YOLOv8 and YOLOv7-tiny models, respectively. Its recall (R) was significantly improved by 7.5% and 9.9% compared to the YOLOv8 and YOLOv7-tiny models, respectively. Its mean average precision (mAP) reached 83.4%. The proposed network model exhibits higher recognition accuracy and lower missed detection rates in traffic sign recognition tasks. Furthermore, the model is relatively low in parameters and can be deployed on low-computing vehicle platforms, such as mid-range and low-end fuel vehicles and entry-level new energy vehicles.
[0077] In summary, the embodiment of the present invention improves the C3 module of YOLOv5s by introducing the PSA polarized attention mechanism, which can significantly enhance the feature extraction and discrimination capabilities of the network. At the same time, the cross-scale fusion strategy is adopted to optimize the network architecture, which not only greatly improves the recognition accuracy but also effectively reduces the missed detection rate.
[0078] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concepts of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of protection of this application.
Claims
1. A traffic sign recognition method, comprising: Build a traffic sign recognition model based on the YOLOv5s architecture; The traffic sign recognition model includes a backbone network, a feature fusion network, multiple original detection heads and a small target detection head; Extracting a shallow feature map from the first lightweight residual structure module of the backbone network; Inputting the shallow feature map into the feature fusion network, fusing it with other scale feature maps, and generating a fused feature map; The fused feature map is input into the small target detection head for processing, and the output detection result is used together with the detection results of the multiple original detection heads to identify traffic signs.
2. The traffic sign recognition method according to claim 1, characterized in that: The backbone network is a YOLOv5s backbone network, and the shallow feature map is the output feature map of the first lightweight residual structure module in the backbone network.
3. The traffic sign recognition method according to claim 1, characterized in that: The spatial resolution of the fused feature map is higher than the maximum value of the spatial resolutions of the multiple original detection head input feature maps.
4. The traffic sign recognition method according to claim 1, characterized in that: The feature fusion network includes multiple fusion levels for realizing multi-level fusion of feature maps of different scales.
5. The traffic sign recognition method according to claim 4, characterized in that: The fusion levels include an intermediate scale, a top scale, and a bottom scale; The intermediate-scale feature maps are connected by weighted fusion; The feature maps of the top scale and the bottom scale are connected through a channel splicing operation; The weight parameters of the weighted fusion are learned during the network training process.
6. The traffic sign recognition method according to any one of claims 1 to 5, characterized in that: The feature fusion network is a UPPANet structure, which is composed of a feature pyramid network and a path aggregation network.
7. The traffic sign recognition method according to any one of claims 1 to 5, characterized in that: A residual structure with an attention mechanism is formed in one or more lightweight residual structure modules in the backbone network, and the residual structure with an attention mechanism is constructed by introducing a polarized self-attention mechanism module between its splicing operation and addition operation.
8. The traffic sign recognition method according to any one of claims 1 to 5, characterized in that: The small target detection head is used to receive the fused feature map and output the detection frame category and position parameters after extracting features through a convolutional layer. 9 . A traffic sign recognition device, comprising a memory and a processor, wherein the processor is configured to execute the traffic sign recognition method according to claim 1 . 10 . A computer-readable storage medium storing a computer program, wherein when the program is executed by a processor, the processor is caused to execute the traffic sign recognition method according to claim 1 .