A traffic sign detection method based on dense connection attention

Abstract

The application relates to a traffic sign detection method based on dense connection attention, and belongs to the intelligent traffic field. In view of the problem that the traffic sign target is small in a wide field of view in an automatic driving target detection task, a traffic sign detection algorithm DCAM-YOLO based on dense connection attention is provided. A dense connection type spatial attention mechanism module DCAM is embedded between a feature enhancement network and a detection head. The DCAM performs multiple convolutions on input features to obtain multiple feature maps with the same size, then excavates spatial attention features along a channel direction to form a spatial attention feature network layer, and finally fuses the multiple spatial attention feature network layers obtained in a dense connection mode to form dense connection type attention weights, so that the utilization rate of effective information in the input features is increased, higher-level fusion is realized, and the feature extraction is strengthened.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation and relates to a traffic sign detection method based on dense connection attention. Background Technology

[0002] Traffic sign detection not only provides effective road condition data support for driver assistance systems, but also avoids tedious and error-prone manual annotation in the creation of high-precision maps. Therefore, in-depth research on traffic sign detection systems has significant practical value in improving road safety and can also promote the development of autonomous driving technology.

[0003] Traditional traffic sign detection algorithms consist of two stages: region of interest (RoI) extraction and RoI classification. In the RoI extraction stage, sliding windows of different scales and proportions are typically used to scan the entire image to obtain potential target regions. In the RoI classification stage, manually designed features such as HOG (histograms of oriented gradients), Gabor, and Haar-like features are commonly used, combined with machine learning algorithms to determine the RoI category. Due to issues such as lighting, deformation, and occlusion, traditional methods struggle to achieve satisfactory results in practical traffic sign detection tasks.

[0004] In recent years, with the continuous development and deepening of convolutional neural networks in the field of computer vision, traffic sign detection algorithms based on deep learning have also made great progress. Existing detection methods can be divided into two-stage methods and one-stage methods. Two-stage methods, represented by Faster R-CNN (region-convolutional neural network), use RPN (region proposal network) to generate proposal boxes at the feature level by sharing convolutional features, and then use the convolutional features of the proposal box region for classification and target box localization learning. It has the characteristics of high accuracy but slow speed. One-stage object detection methods, represented by YOLO (you only look once) and SSD (single shot detector), unify the target box localization and recognition tasks according to the regression logic, and the convolutional neural network completes the prediction in one go at the output layer. It has the characteristics of fast speed but low accuracy.

[0005] In real-world traffic sign detection scenarios, image backgrounds are complex and diverse, containing various billboards, distracting objects, and other warning signs. These fake traffic signs are easily confused with genuine traffic signs in terms of shape and color, leading to false detections. Furthermore, to obtain road information in advance, traffic signs captured by vehicle-mounted cameras are generally small in absolute pixel size and occupy a very small proportion of the overall image. The small absolute size of traffic signs means they contain less effective information and more noise, making them prone to false detections and missed detections in blurry situations. Similarly, the relatively small size of the target means that the image contains less feature information about that target, which is easily lost during convolutional feature extraction, thus leading to missed detections. Attention mechanisms have been proven effective in improving the detection performance of small targets in many computer vision tasks. This method simulates the process by which the human brain extracts external information; that is, the human visual system locally focuses on certain areas of an image, and by investing more attention in the focused area, effective detailed information is obtained. Attention mechanisms enable humans, with limited visual perception capabilities, to reasonably suppress and enhance massive amounts of input information, greatly improving the information processing capacity of the human visual system. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a traffic sign detection method based on dense connection attention.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A traffic sign detection method based on dense connection attention, comprising the following steps:

[0009] S1: Input the image into the backbone network of the object detection network, and use the backbone network to extract the effective information related to traffic signs from the image;

[0010] S2: The effective information extracted from the backbone network is fed into the feature fusion network in the target detection network. This feature fusion network can achieve the fusion of different features. The fused features have rich semantic and location information related to the detected target. The semantic and location information can improve the accuracy of target detection.

[0011] S3: Input the fused features into the Dense Connection Attention Mechanism (DCAM) module. Using the attention mechanism, higher weights are assigned to small targets, making the target detection network pay more attention to the feature information of these small targets and improving the detection accuracy of small targets.

[0012] S4: Input the features processed by the densely connected spatial attention module into the detection head of the object detection network. Use the anchor box-based multi-scale object detection method to classify and locate the features to obtain the final object detection result.

[0013] Optionally, in S2, during network training, feature layers of different depths for the same target are fused; this results in unequal weights for features of the same type but different sizes to the fused output features; a weighted bidirectional feature pyramid network (BiFPN) is introduced to replace the feature pyramid network in YOLOv5; the weighted bidirectional feature pyramid network, referencing the attention mechanism, adds weights to the fused features of different sizes, dynamically and learnably adjusting the contribution of each scale, enabling the network to better fuse features of different sizes; residual connections are added to enhance the expressive power of features; for nodes with single input and output edges, BiFPN is used instead of FPN to improve the model's feature fusion capability.

[0014] Optionally, in S3, a densely connected attention mechanism module (DCAM) is designed. The design principle of the DCAM is to fully mine the effective information in the input features without excessively increasing the network parameters and floating-point calculations, thus ensuring the applicability of the plug-and-play DCAM in different object detection networks and the real-time performance of the object detection algorithm. The DCAM performs multiple convolutions on the input features to obtain multiple feature maps of the same size. Then, it mines spatial attention features along the channel direction of the obtained feature maps to form a spatial attention feature network layer. The multiple spatial attention feature network layers are fused together using a dense connection method to form a densely connected attention weight, thereby increasing the utilization rate of effective spatial information in the input features.

[0015] Optionally, the DCAM dense connection attention mechanism module includes an attention generation module, a feature enhancement module, and an attention feature fusion module;

[0016] Attention generation module: Let the feature X input to the spatial attention generation module be C*H*W, where C is the number of channels of the feature, and H and W are the length and width of the feature, respectively. First, max pooling and average pooling operations are performed on the input feature along the channel direction to obtain two 1*H*W features. Then, a 3*3 convolution module is used to fuse the two features. Finally, the fused features are normalized and processed by activation functions to obtain the required spatial attention.

[0017] Feature enhancement module: While ensuring the extraction of feature maps of the same size, it reduces computational parameters and floating-point operations by using a lightweight hourglass network. Assuming the feature input to the feature enhancement module is w*h*d, where d is the number of channels and w and h are the length and width of the feature, the input feature is reduced in dimensionality by a 1*1 convolution with d / 2 channels, resulting in a feature with only d / 2 channels. Then, a 3*3 convolution is used to increase the dimensionality of the feature, restoring the number of channels to d. Finally, the resulting feature is added to the original feature to obtain the final enhanced feature.

[0018] Attention Fusion Module: Let Mi be the spatial attention of the i-th feature enhancement module, i = 1, 2, 3, 4, ..., and C be the feature concatenation. The feature FRMi generated by the feature enhancement module FRMi is w*h*d. Then the spatial attention M1, M2...Mi is w*h*1, where 1 is the number of channels of the feature, and w and h are the length and width of the feature, respectively. The i attention features are concatenated to form an attention feature of size w*d*i. Then the concatenated features are fused using a 1*1 convolution to generate a fused attention of size w*d*1. Finally, the fused attention is multiplied by the feature FRMi.

[0019] The beneficial effects of this invention are as follows:

[0020] (1) This method proposes a novel dense connection type attention mechanism module DCAM, which can fuse a set of attention features in a dense connection manner, thereby more efficiently enhancing the features of effective information and improving the detection accuracy of small targets;

[0021] (2) In the feature fusion network, this paper introduces a weighted bidirectional feature pyramid network to weight the feature maps at different scales, thereby ensuring the efficiency of feature fusion.

[0022] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0024] Figure 1 This is a diagram of the network structure for a densely connected attention mechanism.

[0025] Figure 2A detailed diagram of a traffic sign detection algorithm based on dense connection attention;

[0026] Figure 3 This is a network structure diagram of the attention fusion module. Detailed Implementation

[0027] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0029] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0030] 1. Densely Connected Attention Mechanism Module

[0031] The design principle of the DCAM densely connected attention mechanism module is to fully extract effective information from the input features without excessively increasing network parameters and floating-point computations. This ensures the plug-and-play applicability of the DCAM attention mechanism module across different object detection networks and the real-time performance of object detection algorithms. The structure diagram of DCAM is shown below. Figure 1As shown in the figure, A represents the Attention Generation Module, Mi (i = 1, 2, 3, 4, ...) represents the generated attention feature maps, FRM represents the Feature Reference Module, and ACM represents the Attention Concat Module. In the following sections, this paper will introduce the structure and function of each module in turn.

[0032] Attention Generation Module: Since dense connections require the fusion of a large number of attention features, this paper selects spatial attention features that are easier to splice. Assuming that the feature X input to the spatial attention generation module is C*H*W, firstly, max pooling and average pooling operations are performed on the input feature along the channel direction to obtain two 1*H*W features respectively. Then, a 3*3 convolution module is used to fuse the two features. Finally, the fused features are normalized and processed by activation functions to obtain the required spatial attention.

[0033] Feature Enhancement Module (FRM): While ensuring the extraction of feature maps of the same size, it minimizes computational parameters and floating-point operations by employing a lightweight hourglass network. Assuming the input feature to the feature enhancement module is w*h*d, the input feature is first reduced in dimensionality by a 1*1 convolution with d / 2 channels, resulting in a feature with only d / 2 channels. Then, a 3*3 convolution is used to increase the dimensionality of the feature, restoring the number of channels to d. Finally, the resulting feature is added to the original feature to obtain the final enhanced feature.

[0034] Attention Fusion Module (ACM): The structure of ACM is as follows Figure 3 As shown, assume Mi is the spatial attention of the i-th FRM, and C is the feature concatenation (Concat). Assume the feature FRMi generated by the feature enhancement module FRMi is w*h*d, then the spatial attentions M1, M2...Mi are w*h*1. First, feature concatenation is used to concatenate the i attention features to form an attention feature of size w*d*i. Then, the concatenated features are fused using a 1*1 convolution to generate a fused attention of size w*d*1. Finally, the fused attention is multiplied by the feature FRMi.

[0035] 2. Weighted bidirectional feature fusion network

[0036] During network training, because different targets have different sizes, features of large targets can be preserved as convolution deepens, while features of small targets may disappear. Therefore, it is necessary to fuse feature layers of different depths for the same target. YOLOv5 uses a feature pyramid network and a path aggregation network to fuse features of different scales. The path aggregation network propagates features of different scales bidirectionally, passing stronger semantic information carried by deeper feature layers to shallower feature layers, and simultaneously passing stronger localization information carried by shallower feature layers to deeper feature layers, thus achieving path fusion of feature layers of different sizes.

[0037] While feature pyramid networks can effectively fuse different feature layers, they are essentially just simple concatenation and superposition of different features. However, these different input features have different resolutions and contribute differently to the fused output features. Therefore, directly using a concatenation-fusion method cannot efficiently fuse features of different scales. This leads to unequal weights for features of the same type but different sizes in the fused output features. Larger features are incorporated more into the network, while smaller features contribute less. This is particularly detrimental to the detection of small objects such as traffic signs. To address this issue, a weighted bidirectional feature pyramid network (BiFPN) is introduced to improve the detection network structure.

[0038] 3. Traffic sign detection method based on dense connection spatial attention

[0039] This method uses YOLOv5 object detection network version 5.0 as the baseline network to design the DCAM-YOLO object detection network. The network results are as follows: Figure 2As shown, the DCAM-YOLO network structure consists of a backbone network, a feature fusion network, and a detection head. The backbone network uses the CSPDarkNet feature extraction network, and the feature fusion network consists of a weighted bidirectional feature pyramid network and a path aggregation network (PANet). The detection head uses a multi-scale object detection method based on anchor boxes. To address the problem of small object detection in traffic signs, this paper makes targeted improvements in the following two aspects: 1) Adding a densely connected attention mechanism module (DCAM): DCAM performs multiple convolutions on the input features to obtain multiple feature maps of the same size. Then, it extracts spatial attention features along the channel direction to form a spatial attention feature network layer. Finally, it uses dense connections to fuse the multiple spatial attention feature network layers to form densely connected attention weights, increasing the utilization rate of effective spatial information in the input features. 2) To ensure that the input feature map has rich semantic features, the feature fusion enhancement network is optimized by introducing a weighted bidirectional feature pyramid (BiFPN) network, which connects the input and output nodes of the same layer across layers, achieving a higher level of fusion and strengthening feature extraction.

[0040] 4. The traffic sign detection method based on dense connection spatial attention includes the following steps:

[0041] (1) Input the image into the backbone network of the target detection network, and use the backbone network to extract the effective information related to traffic signs in the image;

[0042] (2) The effective information extracted from the backbone network is fed into the feature fusion network in the target detection network. This network can achieve effective fusion between different features. After fusion, the features have rich semantic and location information related to the target being detected. This information can effectively improve the accuracy of target detection.

[0043] (3) Input the fused features into the densely connected attention module. This module can use the attention mechanism to give higher weights to traffic signs that are small and difficult to detect, so that the target detection network pays more attention to the feature information of these small targets, thereby improving the detection accuracy of small targets.

[0044] (4) Input the features processed by the densely connected spatial attention module into the detection head of the target detection network, and use the anchor box-based multi-scale target detection method to classify and locate the features to obtain the final target detection result.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A traffic sign detection method based on dense connection attention, characterized in that: The method includes the following steps: S1: Input the image into the backbone network of the object detection network, and use the backbone network to extract the effective information related to traffic signs from the image; S2: The effective information extracted from the backbone network is fed into the feature fusion network in the object detection network. This feature fusion network can fuse different features, and the fused features have rich semantic and positional information related to the detected target. The semantic and positional information can improve the accuracy of object detection. During network training, feature layers of different depths of the same target are fused. This results in different sizes of features of the same type having unequal weights on the fused output features. A weighted bidirectional feature pyramid network (BiFPN) is introduced to replace the feature pyramid network in YOLOv5. The weighted bidirectional feature pyramid network, referencing the attention mechanism, adds weights to the fused features of different sizes, dynamically and learnably adjusting the contribution of each scale, so that the network can better fuse features of different sizes. Residual connections are added to enhance the expressive power of features. For nodes with single input and output edges, BiFPN is used instead of FPN to improve the model's feature fusion capability. S3: The fused features are input into the Dense Connection Attention Mechanism (DCAM) module. Using the attention mechanism, higher weights are assigned to small targets, making the target detection network focus more on the feature information of these small targets and improving the detection accuracy of small targets. The DCAM module is designed to fully extract effective information from the input features without excessively increasing network parameters and floating-point computations, ensuring its plug-and-play applicability in different target detection networks and the real-time performance of the target detection algorithm. The DCAM performs multiple convolutions on the input features to obtain multiple feature maps of the same size. Then, spatial attention features are extracted from the obtained feature maps along the channel direction to form a spatial attention feature network layer. These multiple spatial attention feature network layers are fused using a dense connection approach to form densely connected attention weights, increasing the utilization rate of effective spatial information in the input features. The Dense Connection Attention Mechanism (DCAM) module includes an attention generation module, a feature enhancement module, and an attention feature fusion module. Attention generation module: Let the feature X input to the spatial attention generation module be C*H*W, where C is the number of channels of the feature, and H and W are the length and width of the feature, respectively. First, max pooling and average pooling operations are performed on the input feature along the channel direction to obtain two 1*H*W features. Then, a 3*3 convolution module is used to fuse the two features. Finally, the fused features are normalized and processed by activation functions to obtain the required spatial attention. Feature enhancement module: While ensuring the extraction of feature maps of the same size, it reduces computational parameters and floating-point operations by using a lightweight hourglass network. Assuming the feature input to the feature enhancement module is w*h*d, where d is the number of channels and w and h are the length and width of the feature, the input feature is reduced in dimensionality by a 1*1 convolution with d / 2 channels, resulting in a feature with only d / 2 channels. Then, a 3*3 convolution is used to increase the dimensionality of the feature, restoring the number of channels to d. Finally, the resulting feature is added to the original feature to obtain the final enhanced feature. Attention Fusion Module: Let Mi be the spatial attention of the i-th feature enhancement module, i=1,2,3,4,..., and C be the feature concatenation; the feature FRMi generated by the feature enhancement module FRMi is w*h*d, then the spatial attention M1, M2...Mi is w*h*1, where 1 is the number of channels of the feature, and w and h are the length and width of the feature, respectively. The i attention features are concatenated to form an attention feature of size w*d*i using feature concatenation. Then, the concatenated features are fused using a 1*1 convolution to generate a fused attention of size w*d*1. Finally, the fused attention is multiplied by the feature FRMi. S4: Input the features processed by the densely connected spatial attention module into the detection head of the object detection network. Use the anchor box-based multi-scale object detection method to classify and locate the features to obtain the final object detection result.

Images