A road object detection method based on attention feature enhancement module

Abstract

The present invention discloses a road target detection method based on an attention feature enhancement module, and belongs to the technical field of target detection. The present invention first constructs a convolutional neural module to extract features of the road target to be detected in the original image, and obtains input feature maps of different sizes; then constructs an attention feature enhancement module including a CBAM attention mechanism and a semantic enhancement branch, and performs feature enhancement on the acquired feature map; finally, based on the enhanced feature map containing deep semantic information and shallow texture information, a decoupling head is used to perform classification regression to complete the detection of the target. The test results of the BDD100K dataset show that the average accuracy of the method disclosed by the present invention is improved by 1.8%; the test results of the PASCAL VOC 2007 dataset show that the average accuracy of the method disclosed by the present invention is improved by 0.6%.

Description

Technical Field

[0001] The present invention belongs to the technical field of object detection, and in particular relates to a road object detection method based on an attention feature enhancement module. Background Art

[0002] With the continuous increase in the number of cars, travel safety issues have become increasingly prominent. Autonomous driving technology based on computer vision has provided a new solution to traffic problems and has been paid attention to and studied by more and more countries. When detecting road targets, traditional target detection algorithms lack the ability to distinguish between features, and have problems such as high missed detection rate and low recall rate. Therefore, it is of great significance to improve the accuracy of road target detection algorithms in complex traffic scenarios.

[0003] Deep convolutional neural networks are highly robust because they can autonomously learn target features and extract key information. In recent years, there are two main approaches to target detection models based on convolutional neural networks: target candidate boxes and regression. The corresponding algorithms are called two-stage algorithms and single-stage algorithms. Two-stage detection algorithms represented by algorithms such as R-CNN, Fast R-CNN, Faster R-CNN, and R-FCN first extract target candidate boxes, and then use the detection network to complete model training based on the extracted candidate boxes. Single-stage detection algorithms represented by algorithms such as SSD, YOLO, and YOLOv3 directly regress the category and location information of the target through the detection network, and have a higher detection speed. However, since different feature maps or even different regions in the same feature map have different contributions to the target, the features obtained by the current detection algorithm are always extensive and redundant, and cannot accurately meet the needs of the task. Summary of the invention

[0004] To solve the above problems, the present invention discloses a road target detection method based on an attention feature enhancement module, which adds an attention mechanism to extract target features in a differentiated manner, increases the expression of features of task-interested regions, and adds a semantic enhancement branch to propagate features with strong semantics. Compared with other advanced target detection algorithms, the detection accuracy of road targets can be effectively improved.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A road object detection method based on an attention feature enhancement module comprises the following steps:

[0007] Step 1: Obtain image information of the road target to be detected;

[0008] Step 2: construct a convolutional neural network, extract features representing road targets from the image information, and obtain input feature maps of different sizes;

[0009] Step 3: construct an attention feature enhancement module, which includes a CBAM attention mechanism and a semantic enhancement branch. The attention feature enhancement module is used to enhance the input feature map obtained in step 2, thereby improving the feature expression of the task area of ​​interest and obtaining a feature map containing deep semantic information and shallow texture information.

[0010] Step 4: Based on the enhanced feature map obtained in step 3, a decoupled output head is used for classification and regression, and the detection result is output.

[0011] As described above, a road target detection method based on an attention feature enhancement module, further: the specific steps of constructing the attention feature enhancement module in step 1.3 include:

[0012] Step 1.3.1: Input feature map with a size of 1 / 32 of the original image Input into the CBAM module to obtain the feature map Will After convolution, batch normalization and activation function processing, upsampling is performed to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image.

[0013] Step 1.3.2: Input feature map with a size of 1 / 16 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image.

[0014] Step 1.3.3: Enhance the feature map The feature map is input into the CSPLayer unit, and the obtained feature map is upsampled after convolution, batch normalization and activation function processing to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image.

[0015] Step 1.3.4: Input feature map with a size of 1 / 8 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image.

[0016] Step 1.3.5: Enhance the feature map The input is into the CSPLayer unit, and the obtained feature map is used by the decoupling head to detect the target.

[0017] As described above, a road target detection method based on an attention feature enhancement module, further: the CBAM module in step 1.3.1, step 1.3.2 and step 1.3.4 includes two processing steps of generating a channel attention feature map and a spatial attention feature map:

[0018] The input feature map F is first subjected to global average pooling and global maximum pooling operations to obtain the spatial information of the input features and generate and Two spatial context descriptors. and The channel attention map is generated by a multilayer perceptron with one hidden layer. When the size of the input feature map is Channel attention feature map M c (F) The detailed calculation formula is:

[0019]

[0020] in, r represents the reduction value of the bottleneck structure of the multilayer perceptron, which is 16; σ(·) represents the Sigmoid activation function; R(·) represents the ReLU linear rectification function; g(·) is the global average pooling function; δ(·) is the global maximum pooling function.

[0021] The calculation formula of the Sigmoid activation function σ(·) is:

[0022]

[0023] The calculation formula of ReLU linear rectification function R(·) is:

[0024]

[0025] The calculation formula of the global average pooling function g(·) is:

[0026]

[0027] The calculation formula of the global maximum pooling function δ(·) is:

[0028]

[0029] When generating the spatial attention feature map, the input feature map F′ is first pooled along the channel axis including average pooling and maximum pooling, and generated and Two two-dimensional feature maps. The generated two-dimensional feature maps are concatenated in the channel dimension and convolved through a standard convolutional layer to generate a spatial attention feature map M. s (F′). Spatial attention feature map M s The detailed calculation formula of (F′) is:

[0030]

[0031] in, f 7×7 It represents a convolution operation with a convolution kernel of 7×7.

[0032] The calculation formula for the attention feature graph output by the CBAM module is:

[0033]

[0034] As described above, a road target detection method based on an attention feature enhancement module, further: the CSPLayer unit described in step 1.3.3 and step 1.3.5 specifically includes the following steps:

[0035] First, the input feature map F1 is processed by convolution, batch normalization and activation function to obtain the feature map F 11 ;

[0036] Then the input feature map F1 is input into another branch, and convolution operation, batch normalization and activation function processing are performed to obtain the feature map F 21 , F 21 In the successive residual bottleneck block, three operations are performed successively to obtain the feature map F 22 ;

[0037] Finally, the feature map F 11 With the feature map F 22 Concatenate on the channel dimension to obtain the feature map F 31 , and the feature map F 31 Convolution operations, batch normalization, and activation function processing are performed for subsequent operations.

[0038] The beneficial effects of the present invention are:

[0039] The present invention proposes a road target detection method based on an attention feature enhancement module. Compared with the basic YOLOX-L algorithm, the road target detection method mentioned in the present invention can improve detection accuracy while maintaining detection speed. The method proposed in this article improves the average accuracy by 1.8% on the BDD100K dataset and by 0.6% on the PASCAL VOC 2007 test set. With the continuous increase in the number of cars, travel safety issues have become increasingly prominent. Autonomous driving technology based on computer vision has provided a new solution to traffic problems and has received attention and research from more and more countries. However, when traditional target detection algorithms detect road targets in complex traffic scenarios, there are problems such as high missed detection rate and low recall rate. Therefore, it is of great significance to improve the accuracy of road target detection algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of the method;

[0041] Figure 2 CBAM attention model structure diagram;

[0042] Figure 3 AFE-YOLOX algorithm network structure diagram. DETAILED DESCRIPTION

[0043] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0044] The present invention uses the BDD100K dataset and the VOC dataset to experiment with the proposed road object detection method based on the attention feature enhancement module.

[0045] The method proposed in the present invention firstly needs to obtain the image information of the road target to be detected, and on this basis, construct a convolutional neural network to obtain the features of the target to be detected from the image information, and obtain input feature maps of different sizes; then, the attention feature enhancement module composed of the CBAM attention mechanism and the semantic enhancement branch performs feature enhancement on the input feature map to obtain an enhanced feature map containing deep semantic information and shallow texture information; finally, the decoupling head is used to classify and regress the enhanced feature map to obtain the detection result.

[0046] Step 1: Construct a convolutional neural network to obtain the features of the target to be detected from the image information and obtain input feature maps of different sizes:

[0047] Step 1.1: Construct a convolutional neural network for extracting target features

[0048] The constructed neural network structure is as follows Figure 3The input image first passes through the Focus network module to obtain a feature map with a width and height of 1 / 2 of the original image and a number of channels of 4 times the original image. Then, after multiple operations of the Convolutional module and the CSPLayer unit, the input feature maps with sizes of 1 / 8, 1 / 16, and 1 / 32 of the original image are obtained.

[0049] The specific implementation process of the Focus network module is: take the value of every other pixel of the input image, stack the obtained 4 independent feature layers, and concentrate the width and height information into the channel information. The specific implementation process of the Convolutional module is: first perform a convolution operation on the input feature layer, then perform batch normalization on the result of the convolution operation, and finally perform an activation function operation. The specific implementation method of the CSPLayer unit is: first perform a convolution operation, batch normalization and activation function on the input feature map F1 to obtain the feature map F 11 ; Then the input feature map F1 is input into another branch, and convolution operation, batch normalization and activation function processing are performed to obtain the feature map F 21 , F 21 In the successive residual bottleneck block, three operations are performed successively to obtain the feature map F 22 ; Finally, the feature map F 11 With the feature map F 22 Concatenate on the channel dimension to obtain the feature map F 31 , and the feature map F 31 Convolution operations, batch normalization, and activation function processing are performed for subsequent operations.

[0050] Step 2: Construct an attention feature enhancement module, which includes a CBAM attention mechanism and a semantic enhancement branch. The input feature map obtained in step 1 is enhanced by the attention feature enhancement module to obtain an enhanced feature map:

[0051] Step 2.1: Input feature map with a size of 1 / 32 of the original image Input into the CBAM module to obtain the feature map Will After convolution, batch normalization and activation function processing, upsampling is performed to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image.

[0052] Step 2.2: Input feature map with a size of 1 / 16 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image.

[0053] Step 2.3: Enhance the feature map The feature map is input into the CSPLayer unit, and the obtained feature map is upsampled after convolution, batch normalization and activation function processing to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image.

[0054] Step 2.4: Input feature map with a size of 1 / 8 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image.

[0055] Step 2.5: Enhance the feature map The input is into the CSPLayer unit, and the obtained feature map is used by the decoupling head to detect the target.

[0056] Wherein, the CBAM module in step 2.1, step 2.2 and step 2.4 is as follows Figure 2 As shown in the figure, the CBAM module includes two processing steps: generating channel attention feature maps and spatial attention feature maps:

[0057] The input feature map F is first subjected to global average pooling and global maximum pooling operations to obtain the spatial information of the input features and generate and Two spatial context descriptors. and The channel attention map is generated by a multilayer perceptron with one hidden layer. When the size of the input feature map is Channel attention feature map M c (F) The detailed calculation formula is:

[0058]

[0059] in, r represents the reduction value of the bottleneck structure of the multilayer perceptron, which is 16; σ(·) represents the Sigmoid activation function; R(·) represents the ReLU linear rectification function; g(·) is the global average pooling function; δ(·) is the global maximum pooling function.

[0060] When generating the spatial attention feature map, the input feature map F′ is first pooled along the channel axis including average pooling and maximum pooling, and generated and Two two-dimensional feature maps. The generated two-dimensional feature maps are concatenated in the channel dimension and convolved through a standard convolutional layer to generate a spatial attention feature map M. s (F′). Spatial attention feature map M s The detailed calculation formula of (F′) is:

[0061]

[0062] in, f 7×7 It represents a convolution operation with a convolution kernel of 7×7.

[0063] The calculation formula for the attention feature graph output by the CBAM module is:

[0064]

[0065] Step 3: Based on the enhanced feature map obtained in step 2, a decoupled output head is used for classification and regression, and the detection result is output.

[0066] Step 3.1: Perform a 1×1 convolution operation on the enhanced feature map F1 to reduce the dimension and obtain a feature map F with 256 channels. 11 , F 11 The input classification branch performs 3×3 convolution, batch normalization and activation function processing to obtain the feature map F 12 , for F 12 Perform a 1×1 convolution operation to obtain a feature map F with the same number of channels as the number of target categories. 13 ;

[0067] Step 3.2: Perform a 1×1 convolution operation on the enhanced feature map F1 to reduce the dimension and obtain a feature map F with 256 channels. 21 , F 21 The input regression branch performs 3×3 convolution, batch normalization and activation function processing to obtain the feature map F 22 , for F 22 Perform a 1×1 convolution operation to obtain a feature map F with the same number of channels as the number of target coordinates. 23 ;

[0068] Step 3.3: feature map F 22 Perform a 1×1 convolution operation to obtain a feature map F with the same number of channels as the number of anchor boxes. 33 ;

[0069] Step 3.4: transform the feature map F 13 、F 23 and F 33 Perform splicing to obtain the detection result feature map.

[0070] Table 1 compares the overall detection results and various target detection results before and after adding the attention feature enhancement module based on the YOLOX-L algorithm on the BDD100K dataset. Among them, the AFE-YOLOX-L algorithm with the attention feature enhancement module has an average detection accuracy of 59.0% for 7 types of road targets, which is 1.8% higher than the YOLOX-L algorithm, 0.7% higher for person, 0.9% higher for rider, 0.2% higher for car, 0.9% higher for bus, 1.2% higher for truck, 4.4% higher for bike, and 4.4% higher for motor.

[0071] Table 1 Comparison of detection results on the BDD100K dataset

[0072]

[0073]

[0074] Table 2 compares the performance of the AFE-YOLOX-L algorithm with other advanced target detection algorithms on the BDD100K dataset. The comparison shows that the AFE-YOLOX-L algorithm outperforms many advanced target detection algorithms.

[0075] Table 2 Performance comparison of AFE-YOLOX-L and other advanced target detection algorithms on the BDD100K dataset

[0076]

[0077] Table 3 compares the overall detection results and various target detection results of the AFE-YOLOX-L algorithm with PASCAL VOC 2007 trainval and PASCAL VOC 2012trainval as training data sets and PASCAL VOC 2007 test as test set. When the verification input image is 320×320, the average detection accuracy of the AFE-YOLOX-L algorithm with the AFE-YOLOX-L algorithm for 20 types of targets reaches 84.1%, which is 0.6% higher than that of the YOLOX-L algorithm. Among them, 17 types of targets have been improved to varying degrees.

[0078] Table 3 Comparison of detection results of PASCAL VOC 2007 test set

[0079]

[0080] Table 4 compares the performance of the AFE-YOLOX-L algorithm with other advanced object detection algorithms on the PASCAL VOC 2007 test dataset. The comparison shows that the AFE-YOLOX-L algorithm outperforms many advanced object detection algorithms.

[0081] Table 4 Performance comparison of AFE-YOLOX-L and other advanced object detection algorithms on PASCAL VOC 2007 dataset

[0082]

[0083]

[0084] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A road object detection method based on an attention feature enhancement module, characterized in that: The following steps are involved: Step 1: Obtain image information of the road target to be detected; Step 2: construct a convolutional neural network, extract features representing road targets from the image information, and obtain input feature maps of different sizes; Step 3: construct an attention feature enhancement module, which includes a CBAM attention mechanism and a semantic enhancement branch. The attention feature enhancement module is used to enhance the input feature map obtained in step 2, thereby improving the feature expression of the task area of ​​interest and obtaining a feature map containing deep semantic information and shallow texture information. The specific steps include: Step 1.3.1: Input feature map with a size of 1 / 32 of the original image Input into the CBAM module to obtain the feature map Will After convolution, batch normalization and activation function processing, upsampling is performed to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image. Step 1.3.2: Input feature map with a size of 1 / 16 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 16 of the original image. Step 1.3.3: Enhance the feature map The feature map is input into the CSPLayer unit, and the obtained feature map is upsampled after convolution, batch normalization and activation function processing to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image. Step 1.3.4: Input feature map with a size of 1 / 8 of the original image With semantically enhanced feature maps Perform sum operation to obtain feature map Will Input into the CBAM module to obtain the feature map The feature map With feature map Perform concatenation on the channel dimension to obtain a semantically enhanced feature map with a size of 1 / 8 of the original image. Step 1.3.5: Enhance the feature map The feature map is input into the CSPLayer unit and used by the decoupling head to detect the target. Step 4: Based on the enhanced feature map obtained in step 3, a decoupled output head is used for classification and regression, and the detection result is output.

2. According to claim 1, a road target detection method based on an attention feature enhancement module is characterized in that: The CBAM module described in step 1.3.1, step 1.3.2, and step 1.3.4 includes two processing steps of generating a channel attention feature map and a spatial attention feature map: The input feature map F is first subjected to global average pooling and global maximum pooling operations to obtain the spatial information of the input features and generate and two spatial context descriptors; and The channel attention map is generated by a multilayer perceptron with one hidden layer; when the size of the input feature map is Where R 通道数×高×宽 ; Channel attention feature map M c (F) The detailed calculation formula is: in, r represents the reduction value of the bottleneck structure of the multilayer perceptron, which is 16; σ(·) represents the Sigmoid activation function; R(·) represents the ReLU linear rectification function; g(·) is the global average pooling function; δ(·) is the global maximum pooling function; The calculation formula of the Sigmoid activation function σ(·) is: The calculation formula of ReLU linear rectification function R(·) is: The calculation formula of the global average pooling function g(·) is: The calculation formula of the global maximum pooling function δ(·) is: When generating the spatial attention feature map, the input feature map F′ is first pooled along the channel axis including average pooling and maximum pooling, and generated and Two two-dimensional feature maps; the generated two-dimensional feature maps are concatenated in the channel dimension and convolved through a standard convolutional layer to generate a spatial attention feature map M s (F′); spatial attention feature map M s The detailed calculation formula of (F′) is: in, f 7×7 It represents a convolution operation with a convolution kernel of 7×7; The calculation formula for the attention feature graph output by the CBAM module is:

3. According to claim 1, a road target detection method based on an attention feature enhancement module is characterized in that: The specific steps of the CSPLayer unit described in step 1.3.3 and step 1.3.5 include: First, the input feature map F1 is processed by convolution, batch normalization and activation function to obtain the feature map F 11 ; Then the input feature map F1 is input into another branch, and convolution operation, batch normalization and activation function processing are performed to obtain the feature map F 21 , F 21 In the successive residual bottleneck block, three operations are performed successively to obtain the feature map F 22 ; Finally, the feature map F 11 With the feature map F 22 Concatenate on the channel dimension to obtain the feature map F 31 , and the feature map F 31 Convolution operations, batch normalization, and activation function processing are performed for subsequent operations.

Images