A CA-YOLO Object Detection Algorithm Based on Hybrid Dilated Convolution

Abstract

The present invention designs a CA-YOLO object detection algorithm based on hybrid dilated convolution, which mainly includes the following steps: building a YOLOv4 model; adding a CA attention module before the SPP module and adding attention modules to the residual modules of the backbone network; designing a hybrid dilated convolution module and adding it to the backbone network CSPDarkNet-53; changing the number of convolutions in the convolutional layer output by the backbone network and the convolutional layers on both the upper and lower sides of the spatial pyramid structure SPP, and at the same time adding dilated convolution to the front sides of the output ends of the P3 and P4 heads in the PANet structure and the P5 position after extracting from the feature pyramid. Thus, the CA-YOLO model is built; using a pre-trained model to initialize the CA-YOLO model, training the model, introducing a class-smoothing label to improve the loss function to optimize the model, and using the trained CA-YOLO model and YOLOv4 model to perform image detection on the test set and compare the performance.

Description

Technical Field

[0001] The present invention relates to a CA-YOLO object detection algorithm based on hybrid dilated convolution, belonging to the field of image processing. Background Art

[0002] Traditional object detection generally consists of region selection, feature extraction, and a classifier. Its limitations are as follows: on the one hand, region selection based on sliding windows is only applicable to image detection with simple backgrounds and obvious features; on the other hand, the robustness of manually extracted features is poor, and the time cost is high. YOLOv4 was launched by Alexey Bochvskiyskiy, a promoter in the YOLO community, aiming to study a simple and efficient object detection algorithm, finding the best balance among the model input resolution, the number of convolutional layers, and the number of parameters, and introducing a variety of advanced technologies such as the Mish-activation function with better effects and the CIOU metric that comprehensively considers the center distance and aspect ratio of the object bounding box. However, for images with complex environmental backgrounds (complex lighting conditions, too dark or too bright), it will lead to difficulties in the detection task; weather conditions (such as heavy rain, heavy snow, haze, etc.) affecting the line of sight have a large interference on its detection task. In such cases, the object detection model based on YOLOv4 has the following problems:

[0003] 1. Low accuracy.

[0004] 2. Serious missed detections. Summary of the Invention

[0005] Object of the Invention: Aiming at the problems existing in the above-mentioned prior art, the present invention provides a CA-YOLO object detection algorithm based on hybrid dilated convolution. The main objectives are to solve the problems of low accuracy and serious missed detections of the YOLOv4 model; enhance the representation of objects of interest in the YOLOv4 model; improve the extraction of deep features by the YOLOv4 model; enhance the ability of the YOLOv4 model to obtain position information; and improve the generalization ability of the YOLOv4 model.

[0006] Technical Solution:

[0007] A CA-YOLO object detection algorithm based on hybrid dilated convolution, characterized by comprising the following steps:

[0008] Step 1: Build a YOLOv4 model;

[0009] Step 2: Add a CA attention module to the YOLOv4 model;

[0010] Step 3: Design a hybrid dilated convolution module and add it to the YOLOv4 model;

[0011] Step 4: Add a convolutional layer to the YOLOv4 model and add dilated convolutions to the PANet structure;

[0012] Step 5: Train the model, optimize the model by improving the loss function, and compare the performance of different models.

[0013] The said Step 1 includes:

[0014] Step 1.1: Use CSPDarknet-53 as the backbone network, use SPP as an additional module for the Neck part, use PANet as the feature fusion module for the Neck part, and use YOLOv3 as the detection head part.

[0015] The said Step 2 includes:

[0016] Step 2.1: Add a CA attention module before the SPP module built in Step 1.1 to integrate the effective information of the feature map, enhance the feature expression of the model, and enable the SPP module to play its role to the greatest extent;

[0017] Step 2.2: Add a CA attention module to the residual module of the backbone network built in Step 1.1.

[0018] The said Step 3 includes:

[0019] Step 3.1: Process the input feature map with a 3×3 dilated convolution with a dilation rate of 2 to obtain a feature Figure 1 ;

[0020] Step 3.2: Process the feature Figure 1 obtained in Step 3.1 with 3×3 dilated convolutions with dilation rates of 1 and 3 respectively to obtain a feature Figure 2 and a feature Figure 3 , merge the two feature maps to obtain a feature Figure 4 ;

[0021] Step 3.3: Process the feature Figure 4 obtained in Step 3.2 with a 3×3 dilated convolution with a dilation rate of 1 to obtain a feature Figure 5 , and the mixed dilated convolution module is built;

[0022] Step 3.4: Add the mixed dilated convolution module to the backbone network CSPDarkNet-53 built in Step 1.1. The said Step 4 includes:

[0023] Step 4.1: In the convolutional layers output by the backbone network built in Step 1.1 and the convolutional layers above and below the Spatial Pyramid Pooling (SPP) structure, change the original three-convolution operation with convolutional kernels of 1×1, 3×3, and 1×1 to a four-convolution operation with convolutional kernels of 1×1, 3×3, 1×1, and 3×3.

[0024] Step 4.2: Add dilated convolutions at the front side of the output ends of P3 and P4 heads in the PANet structure built in Step 1.1 and at the P5 position after feature extraction from the Feature Pyramid.

[0025] The said Step 5 includes:

[0026] Step 5.1: Train the model. Use PASCAL VOC 2007 trainval and PASCAL VOC 2012 trainval as the training datasets, and PASCAL VOC 2007 test as the test set. Use the model weights pre-trained on the ImageNet dataset as the model initialization weights. The training process is divided into two stages, namely the frozen backbone training stage and the unfrozen backbone training stage. Set the batch size to 8, the number of training epochs to 40, and the learning rate to 1e-3 during the frozen stage. Set the batch size to 4, the number of training epochs to 20, and the learning rate to 1e-4 during the unfrozen stage.

[0027] Step 5.2: Optimize the model by improving the loss function, and introduce class-smoothing labels to soften the original labels. The formula is as follows:

[0028]

[0029] In the formula, ε is the smoothing factor, represents the one-hot true label of the current sample, and N class represents the number of classes;

[0030] Step 5.3: Test the model. Use the mean Average Precision (mAP) as the evaluation metric for the object detection model.

[0031] Beneficial effects:

[0032] 1. Solve the problems of low accuracy and serious missed detections of the YOLOv4 model;

[0033] 2. Enhance the representation of the objects of interest in the YOLOv4 model;

[0034] 3. Improve the extraction of deep features by the YOLOv4 model;

[0035] 4. Enhance the ability of the YOLOv4 model to obtain position information;

[0036] 5. Improve the generalization ability of the YOLOv4 model. Brief Description of the Drawings

[0037] Figure 1 is the network model diagram of the YOLOv4 model;

[0038] Figure 2 is the structural diagram of the hybrid dilated convolution module designed by the present invention;

[0039] Figure 3 is the overall architecture diagram of the present invention;

[0040] Figure 4 is the result diagram of the comparative experiment of the CA attention module;

[0041] Figure 5 is the result diagram of the comparative experiment of the hybrid dilated convolution module;

[0042] Figure 6 is the result diagram of the comparative experiment of class label smoothing;

[0043] Figure 7 is the result diagram of the comparative experiment of different algorithms. Detailed Description of the Preferred Embodiment

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. Therefore, the detailed description of the embodiments of the present invention provided below in the accompanying drawings is not intended to limit the scope of the claimed invention.

[0045] As shown in the figure, a CA-YOLO object detection algorithm based on hybrid dilated convolution includes the following steps:

[0046] Step 1: Build the YOLOv4 model;

[0047] Step 2: Add the CA attention module to the YOLOv4 model;

[0048] Step 3: Design a hybrid dilated convolution module and add it to the YOLOv4 model;

[0049] Step 4: Add a convolutional layer to the YOLOv4 model and add dilated convolution to the PANet structure;

[0050] Step 5: Train the model, optimize the model by improving the loss function, and compare the performance of different models.

[0051] The said Step 1 includes:

[0052] Step 1.1: Use CSPDarknet-53 as the backbone network, use SPP as an additional module in the Neck part, use PANet as the feature fusion module in the Neck part, and use YOLOv3 as the detection head part.

[0053] The said step 2 includes:

[0054] Step 2.1: Add a CA attention module before the SPP module built in step 1.1 to integrate the effective information of the feature map, enhance the feature expression of the model, and enable the SPP module to play its role to the greatest extent;

[0055] Step 2.2: Add a CA attention module to the residual module of the backbone network built in step 1.1.

[0056] The said step 3 includes:

[0057] Step 3.1: Process the input feature map with a 3×3 dilated convolution with a dilation rate of 2 to obtain a feature Figure 1 ;

[0058] Step 3.2: Process the feature Figure 1 obtained in step 3.1 with 3×3 dilated convolutions with dilation rates of 1 and 3 respectively to obtain a feature Figure 2 and a feature Figure 3 , and merge the two feature maps to obtain a feature Figure 4 ;

[0059] Step 3.3: Process the feature Figure 4 obtained in step 3.2 with a 3×3 dilated convolution with a dilation rate of 1 to obtain a feature Figure 5 , and the mixed dilated convolution module is completed;

[0060] Step 3.4: Add the mixed dilated convolution module to the backbone network CSPDarkNet-53 built in step 1.1. The said step 4 includes:

[0061] Step 4.1: In the convolutional layers output by the backbone network built in step 1.1 and the convolutional layers on the upper and lower sides of the spatial pyramid structure SPP, change the original three-convolution with convolutional kernels of 1×1, 3×3, and 1×1 to four-convolutions with convolutional kernels of 1×1, 3×3, 1×1, and 3×3.

[0062] Step 4.2: Add dilated convolutions at the front sides of the output ends of the P3 and P4 heads in the PANet structure built in step 1.1 and at the P5 position after extracting from the feature pyramid.

[0063] The said step 5 includes:

[0064] Step 5.1: Train the model. Use PASCAL VOC 2007 trainval and PASCAL VOC 2012 trainval as the training datasets, and PASCAL VOC 2007 test as the test set. Use the model weights pre-trained on the ImageNet dataset as the initial weights of the model. The training process is divided into two stages, namely the frozen backbone training stage and the unfrozen backbone training stage. Set the batch size to 8, the number of training epochs to 40, and the learning rate to 1e-3 during the frozen stage. Set the batch size to 4, the number of training epochs to 20, and the learning rate to 1e-4 during the unfrozen stage.

[0065] Step 5.2: Optimize the model by improving the loss function, and introduce class smoothed labels to soften the original labels. The formula is as follows:

[0066]

[0067] where ε is the smoothing factor, represents the one-hot true label of the current sample, and N class represents the number of classes;

[0068] Step 5.3: Test the model. Use mAP as the evaluation metric for the object detection model.

[0069] The present invention solves the shortcomings of the YOLOv4 model in object detection by adding a CA attention module, a hybrid dilated convolution module, increasing the convolutional structure and dilated convolution, and improving the loss function, thereby improving the detection effect.

[0070] Among them, Figure 1 is the network model diagram of the YOLOv4 model. Among them, CSPDarknet-53 serves as the backbone network backbone, SPP serves as an additional module in the Neck part, PANet serves as the feature fusion module in the Neck part, and YOLOv3 serves as the detection head part.

[0071] Figure 2 is the structural diagram of the hybrid dilated convolution module designed by the present invention. Dilated convolution has a calculation method similar to a chessboard. Adding only one scale of dilated convolution will cause local information loss, and there is no correlation between the information obtained at a long distance. Therefore, the present invention designs a hybrid convolution HDC (Hybrid Dilated Convolution) module to fuse dilated convolutions with different dilation rates. Without increasing the training parameters and losing the resolution of the feature map, the receptive field of the feature layer is increased, enabling the feature extraction network to obtain richer feature information and enhancing the detection effect on the target.

[0072] Figure 3 This is the overall architecture of the present invention.

[0073] Figure 4 and 5 6 and 7 are respectively the comparison experiment results of the CA attention module, the hybrid dilated convolution module, the class label smoothing comparison experiment, and the different algorithm comparison experiments conducted on the PASCAL VOC 2007 test dataset. It can be seen from the experimental results that the CA attention module adopted by the present invention can significantly improve the network performance. With different added positions and quantities, the improvement effects are also different. Moreover, when the Coordinate attention module is added before the SPP spatial pyramid module, the performance improvement effect is the best; the hybrid dilated convolution module designed by the present invention and the operations of adding convolutional layers and dilated convolutions in the network can improve the network performance; the class label smoothing design adopted by the present invention also has a certain improvement on the network performance. Generally speaking, the present invention solves the shortcomings of the YOLOv4 model in object detection by adding the CA attention module, the hybrid dilated convolution module, adding convolutional layers and dilated convolutions, and improving the loss function on the YOLOv4 model, thereby improving the detection effect.

[0074] The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A CA-YOLO object detection method based on hybrid dilated convolution, characterized in that It includes the following steps: Step 1: Build the YOLOv4 model; Step 1.1: Use CSPDarknet-53 as the backbone network, use SPP as the additional module in the Neck part, use PANet as the feature fusion module in the Neck part, and use YOLOv3 as the detection head part; Step 2: Add the CA attention module to the YOLOv4 model; Step 3: Design a hybrid dilated convolution module and add it to the YOLOv4 model; Step 3.1: Process the input feature map with a 3×3 dilated convolution with a dilation rate of 2 to obtain Feature Map 1; Step 3.2: Process Feature Map 1 obtained in Step 3.1 with 3×3 dilated convolutions with dilation rates of 1 and 3 respectively to obtain Feature Map 2 and Feature Map 3, and merge the two feature maps to obtain Feature Map 4; Step 3.3: Process Feature Map 4 obtained in Step 3.2 with a 3×3 dilated convolution with a dilation rate of 1 to obtain Feature Map 5, and the hybrid dilated convolution module is built; Step 3.4: Add the hybrid dilated convolution module to the backbone network CSPDarkNet-53 built in Step 1.1; Step 4: Add a convolutional layer to the YOLOv4 model and add dilated convolution to the PANet structure; Step 4.1: In the convolutional layer output by the backbone network built in Step 1.1 and the convolutional layers on the upper and lower sides of the Spatial Pyramid Pooling (SPP) structure, change the original three-convolution operation with convolution kernels of 1×1, 3×3, and 1×1 to a four-convolution operation with convolution kernels of 1×1, 3×3, 1×1, and 3×3; Step 4.2: Add dilated convolution at the front side of the output ends of P3 and P4 heads in the PANet structure built in Step 1.1 and at the P5 position after extracting from the feature pyramid; Step 5: Train the model, improve the loss function to optimize the model, and compare the performance of different models; Among them, introduce class smoothed labels to soften the original labels; the formula is as follows: where ε is the smoothing factor, represents the one-hot true label of the current sample, and N class represents the number of classes.

2. The CA-YOLO object detection method based on hybrid dilated convolution according to claim 1, characterized in that, The said Step 2 includes the following steps: Step 2.1: Add the CA attention module before the SPP module built in Step 1.1 to integrate the effective information of the feature map, enhance the feature expression of the model, and enable the SPP module to play its role to the greatest extent; Step 2.2: Add the CA attention module to the residual module of the backbone network built in Step 1.

1.

3. A CA-YOLO object detection method based on hybrid dilated convolution according to claim 1, wherein The said Step 5 includes the following steps: Step 5.1: Train the model; use PASCAL VOC 2007 trainval and PASCAL VOC 2012 trainval as the training datasets, and PASCAL VOC 2007 test as the test set; use the model weights pre-trained on the ImageNet dataset as the model initialization weights; The training process is divided into two stages, namely the frozen backbone training stage and the unfrozen backbone training stage; Set the batch size to 8, the number of training epochs to 40, and the learning rate to 1e-3 in the frozen stage; set the batch size to 4, the number of training epochs to 20, and the learning rate to 1e-4 in the unfrozen stage; Step 5.2: Test the model; use mAP as the evaluation metric for the object detection model.

Images