An improved target detection algorithm for YOLOv4-tiny network

Abstract

The invention discloses a target detection algorithm for improving the YOLOv4-tiny network, belonging to the technical field of computer vision target detection, and comprising the following steps: the activation function of the convolution module in the backbone structure is modified to the RReLU activation function; the feature fusion part effective feature layer transmission feature information stage is improved to a double-branch structure, the residual branch introduces dilated convolution to extract shallow features, and the feature is fused with the feature extracted by the other branch, and the fused feature information is integrated with the original feature information retained by the empty residual edge; the CA attention mechanism is added to the input stage of the improved double-branch structure in the feature fusion part and the upsampling stage of the lower effective feature layer; the YOLO Head part is improved to allocate the bounding box position prediction and generation and feature classification tasks respectively to the double-branch structure, and introduces global average pooling to assist the feature classification task. The invention can improve the detection accuracy and greatly enhance the detection effect of small targets.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision target detection, and is a target detection algorithm for an improved YOLOv4-tiny network. Background Art

[0002] Object detection is one of the important tasks of computer vision. It is a technology for identifying and locating desired objects in images. In recent years, object detection has been widely used in many fields of artificial intelligence and information technology, including robot vision, consumer electronics, security, autonomous driving, human-computer interaction, etc. With the improvement of computer hardware level, the development of deep learning technology and the introduction of various high-quality object detection data sets in recent years, more and more excellent object detection algorithms have emerged.

[0003] As a classic algorithm in the one-stage YOLO algorithm, YOLOv4-tiny has fast network detection speed and few parameters. It can be well adapted to industrial production and applications, and greatly improves the computational efficiency of the target detection algorithm. However, the YOLOv4-tiny network detection accuracy is low, and the detection effect on small targets is not ideal. In order to solve the problem of low detection accuracy, YOLOv4-tiny needs to be improved. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a target detection algorithm of an improved YOLOv4-tiny network, which can improve the detection accuracy, fully extract shallow information, and enhance the detection effect of small targets.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An improved target detection algorithm of the YOLOv4-tiny network includes the following steps:

[0007] S1: Modify the LeakyReLU activation function in the original convolution module of the YOLOv4-tiny network backbone network and introduce the RReLU activation function;

[0008] S2: Improve the feature fusion part of the YOLOv4-tiny network. In the stage of transmitting feature information in the effective feature layer of the feature fusion part, residual edges are introduced to form a dual-branch structure. In the residual branch, dilated convolution is introduced to extract shallow features. After the other branch extracts features, feature fusion is performed with shallow features. Empty residual edges are introduced to retain the original feature information and the fused feature information for feature integration.

[0009] S3: Add CA attention mechanism in the input stage of the improved dual-branch structure in the feature fusion part and the upsampling stage of the lower effective feature layer;

[0010] S4: The YOLO Head part of the YOLOv4-tiny network is improved. The dual-branch structure assigns the tasks of bounding box position prediction and generation and feature classification respectively, and introduces global average pooling to assist the feature classification task.

[0011] A further improvement of the technical solution of the present invention is that in the RReLU activation function of S1, (0.1-0.3) is selected as the uniform distribution interval of the negative slope, and the RReLU activation function formula is as follows:

[0012]

[0013] Among them, x ji Represents the input of each layer, y ji represents the output of each layer, a ji Represents a random value drawn from a uniform distribution (0.1-0.3).

[0014] A further improvement of the technical solution of the present invention is that S2 specifically includes the following steps:

[0015] After the backbone network transfers feature information to the feature fusion part, S21 introduces the residual edge and improves it into a double-branch structure;

[0016] The dual-branch structure continues to perform 3×3 ordinary convolution feature extraction operations on the original branch side, and uses dilated convolution to extract features on the added residual side. The dilated convolution formula is as follows:

[0017] k'=d×(k-1)+1

[0018] In the formula, k represents the input convolution kernel size, d represents the dilation coefficient, and k' represents the equivalent convolution kernel size after the dilated convolution operation;

[0019] In order to ensure that the input and output sizes remain unchanged after the dilated convolution operation, the following convolution input and output formulas must be met:

[0020]

[0021] Among them, Z1 and Z2 are the sizes of the output and input feature maps respectively, P is the zero-filled row, S is the step size, and k is the convolution kernel size;

[0022] S22 introduces an empty residual edge to transmit the original feature information, and performs 1×1 convolution feature integration processing on the feature information output by the dual-branch structure and the original feature information transmitted through the empty residual edge.

[0023] A further improvement of the technical solution of the present invention is that the expansion coefficient is set to 3, and after substituting into the expansion convolution formula, the original 3×3 convolution is equivalent to the receptive field effect of 7×7 convolution.

[0024] A further improvement of the technical solution of the present invention is that S4 specifically includes the following steps:

[0025] S41 changes the 3×3 convolution structure in the original YOLO Head part's 3×3 and 1×1 convolution structures to a dual-branch network structure;

[0026] The first layer of the S42 dual-branch structure is a 3×3 convolution operation to replace the positioning effect of the prediction box alone;

[0027] Perform 1×1 convolution on the input to reduce the dimension, and improve the 3×3 convolution operation to 1×3 and 3×1 spatially separable convolution to reduce the number of parameters;

[0028] The second layer of the S43 dual-branch structure is a 5×5 convolution operation that replaces the task of classifying the detection results alone;

[0029] Perform 1×1 convolution on the input to reduce the dimension, add a global average pooling layer after the dimension reduction, and improve the 5×5 convolution to 1×5 and 5×1 spatially separable convolutions;

[0030] After the dual-branch feature information of S44 is fused, a 1×1 convolution operation is performed to achieve the required output feature dimension.

[0031] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0032] 1. While ensuring that the improved network structure continues to maintain YOLOv4-tiny and a faster detection speed, the present invention improves the detection accuracy of the network structure for the target, especially greatly improves the detection accuracy for small targets, and can be more applicable to target detection work.

[0033] 2. In the training phase of the present invention, a ji is a value randomly drawn from a uniform distribution. ji The value of has a range, which can greatly solve the problem of the convolution module losing nonlinear activation effect for different target sizes, thereby improving the detection accuracy.

[0034] 3. The present invention introduces a dual-branch structure and an expanded convolution to enhance the extraction of shallow features, fuse the deep and shallow feature information, enhance the feature extraction effect, and introduce the residual branch of the original effective layer feature information and the enhanced extracted feature integration processing, so that the feature information contains both the rich semantic information of the original effective feature layer and the deep and shallow layer information after being processed by the multi-level feature fusion structure.

[0035] 4. The present invention improves detection accuracy by adding the CA attention mechanism to enhance the network's extraction of key information.

[0036] 5. The present invention establishes a dual-branch structure to make the classification and positioning tasks branched. While the two tasks do not affect each other, the classification task uses global pooling to integrate global information to improve the classification effect and achieve the effect of feature enhancement. The integration of global information can greatly improve the detection of small targets. In addition, the introduction of spatially separable convolution will not significantly increase the number of parameters, which can ensure the lightweight effect while improving the detection accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall structure diagram of the original YOLOv4 tiny in the present invention;

[0038] Figure 2 This is the mAP graph of the original YOLOv4-tiny structure in this invention;

[0039] Figure 3 It is the mAP graph of the activation function modified for the YOLOv4-tiny structure in the present invention;

[0040] Figure 4 This is a diagram of the improvement of the feature fusion part of YOLOv4-tiny in the present invention;

[0041] Figure 5 This is the mAP graph after the improvement of the feature fusion part of YOLOv4-tiny in the present invention;

[0042] Figure 6 This is a diagram of adding an attention mechanism to the feature fusion part of YOLOv4-tiny in the present invention;

[0043] Figure 7 This is the mAP graph of adding attention mechanism to the feature fusion part of YOLOv4-tiny in the present invention;

[0044] Figure 8 This is a diagram of the improvement of the YOLOv4-tiny YOLO Head in the present invention;

[0045] Fig. 9 It is the mAP value after the improvement of YOLOv4-tiny YOLO Head in this invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0047] The YOLOv4-tiny network structure is divided into three parts: backbone network part, feature fusion part, and YOLO Head part. Figure 1 shown.

[0048] Part of the VOC2007+VOC2012+COCO datasets are selected as the experimental dataset support for the present invention, and data enhancement techniques such as Gaussian noise and grayscale are performed on the datasets, and the datasets are divided into training set, validation set, and test set according to the ratio of 8:1:1.

[0049] The original YOLOv4-tiny network structure was trained and tested. The experimental results are as follows Figure 2 Shown

[0050] An improved target detection algorithm of the YOLOv4-tiny network includes the following steps:

[0051] S1: Modify the LeakyReLU activation function in the original convolution module of the YOLOv4-tiny network backbone network, introduce the RReLU activation function, and select (0.1-0.3) as the uniform distribution interval of the negative slope. The RReLU activation function formula is as follows:

[0052]

[0053] Among them, x ji Represents the input of each layer, y ji represents the output of each layer, a ji Represents a random value drawn from a uniform distribution (0.1-0.3).

[0054] For S1, the ablation experiment was performed and the experimental results are as follows Figure 3 As shown in the figure, the detection accuracy is improved by 0.19%. ji The range of values ​​can greatly solve the problem of the convolution module losing nonlinear activation effect for different target sizes, thereby improving detection accuracy.

[0055] S2: Improve the feature fusion part of the YOLOv4-tiny network. In the stage of transmitting feature information in the effective feature layer of the feature fusion part, residual edges are introduced to form a dual-branch structure. In the residual branch, dilated convolution is introduced to extract shallow features. The other branch extracts features and then fuses them with shallow features.

[0056] The fused feature information is integrated with the original feature information retained by the empty residual edge to achieve a fully integrated feature information effect. The improvement is shown in Figure 2. Figure 4 As shown, the specific steps include:

[0057] After the backbone network transfers feature information to the feature fusion part, S21 introduces the residual edge and improves it into a double-branch structure;

[0058] The dual-branch structure continues to perform 3×3 ordinary convolution feature extraction operations on the original branch side, and uses dilated convolution to extract features on the added residual side. The dilated convolution formula is as follows:

[0059] k'=d×(k-1)+1

[0060] In the formula, k represents the input convolution kernel size, d represents the dilation coefficient, and k' represents the equivalent convolution kernel size after the dilated convolution operation;

[0061] In the present invention, the expansion coefficient is set to 3. After substituting into the above formula, the original 3×3 convolution is equivalent to 7×7 convolution, the receptive field becomes larger, and more sufficient feature information can be extracted.

[0062] In order to ensure that the input and output sizes remain unchanged after the dilated convolution operation, the following convolution input and output formula must be met:

[0063]

[0064] Among them, Z1 and Z2 are the sizes of the output and input feature maps respectively, P is the zero-padding row, S is the step size, and k is the convolution kernel size. According to the above formula, the output sizes of each branch are guaranteed to be the same after the dilated convolution operation, so that feature fusion can be performed, and the zero-padding row and step size are set.

[0065] S22 introduces an empty residual edge to transmit the original feature information, and performs 1×1 convolution feature integration processing on the feature information output by the dual-branch structure and the original feature information transmitted through the empty residual edge;

[0066] For S2, the ablation experiment was performed and the experimental results are as follows Figure 5 As shown in the figure, the detection accuracy is improved by 1.9%. By improving to a dual-branch structure and introducing dilated convolution to enhance the extraction of shallow features, the deep and shallow features are fused to enhance the feature extraction effect. The residual branch of the original effective layer feature information is introduced to integrate the enhanced extracted features, so that the feature information contains both the rich semantic information of the original effective feature layer and the deep and shallow layer information after being processed by the multi-level feature fusion structure, which improves the detection accuracy and the detection effect of small targets.

[0067] S3: Add CA attention mechanism in the input stage of the improved dual-branch structure in the feature fusion part and the upsampling stage of the lower effective feature layer;

[0068] The basic concepts of the CA attention mechanism are as follows:

[0069] The channel attention is decomposed into two 1D feature encoding processes, which aggregate features along two spatial directions respectively. In this way, long-range dependencies can be captured along one spatial direction while precise position information can be preserved along another spatial direction. The resulting feature maps are then encoded into a pair of direction-aware and position-sensitive attention mAPs, respectively, which can be complementarily applied to the input feature maps to enhance the representation of the attended object.

[0070] Since the CA attention mechanism is a plug-and-play module, we can directly add the CA attention mechanism to the improved position, such as Figure 6 shown.

[0071] For S3, the ablation experiment was performed and the experimental results are as follows Figure 7 As shown in the figure, the detection accuracy is improved by 0.22%, which shows that the addition of the CA attention mechanism strengthens the network's extraction of key information and thus improves the detection accuracy.

[0072] S4: Improve the YOLO Head part of the YOLOv4-tiny network. The dual-branch structure allocates the tasks of bounding box position prediction and generation and feature classification respectively, and introduces global average pooling to assist the feature classification task;

[0073] The YOLO Head part of the original YOLOv4-tiny network only obtains the output size through 3×3 and 1×1 convolution operations, where the 3×3 convolution operation is to expand the number of output channels, and the 1×1 convolution operation is used to adjust the number of channels. The original YOLO Head part is equivalent to positioning the prediction box and classifying the detection results at the same time, which will affect the detection speed and accuracy of the structure. Therefore, the present invention establishes a dual-branch structure to perform the positioning and classification tasks in two branches respectively. The improved structure is as follows Figure 8 As shown, the specific steps include:

[0074] S41 changes the 3×3 convolution structure in the original YOLO Head part's 3×3 and 1×1 convolution structures to a dual-branch network structure;

[0075] The first layer of the S42 dual-branch structure is a 3×3 convolution operation to replace the positioning effect of the prediction box alone;

[0076] Since adding a parallel network will affect the number of parameters and thus the network detection speed, it is necessary to first perform 1×1 convolution on the input to reduce the dimension, and improve the 3×3 convolution operation to 1×3 and 3×1 spatially separable convolution to reduce the number of parameters;

[0077] The second layer of the S43 dual-branch structure is a 5×5 convolution operation that replaces the task of classifying the detection results alone;

[0078] Similarly, the input is subjected to 1×1 convolution dimensionality reduction processing, and a global average pooling layer is added after the dimensionality reduction processing to better match the category with the feature map, and to reduce the number of parameters to prevent overfitting and integrate global information to enhance the robustness of the input image. Similarly, the 5×5 convolution is improved to 1×5 and 5×1 spatially separable convolutions.

[0079] After the dual-branch feature information of S44 is fused, a 1×1 convolution operation is performed to achieve the required output feature dimension.

[0080] For S4, the ablation experiment was performed and the experimental results are as follows Fig. 9 As shown in the figure, the detection accuracy is improved by 1.17%, which shows that the dual-branch structure is established to make the classification and positioning tasks branched. While the two tasks do not affect each other, the use of global pooling to integrate global information in the classification task can improve the classification effect and achieve the effect of feature enhancement. In addition, the detection of small targets is greatly improved due to the integration of global information. In addition, the introduction of spatially separable convolution will not significantly increase the number of parameters, which can ensure the lightweight effect while improving the detection accuracy of small targets.

[0081] Example

[0082] Step 1: Select part of the VOC2007+VOC2012+COCO dataset as your own dataset and divide the dataset into training set, validation set, and test set according to the ratio of 8:1:1.

[0083] Step 2: Build an improved YOLOv4-tiny target detection network as follows:

[0084] Improve the backbone structure, and change the original LeakyReLU activation function to RRelu activation function in the convolution module of DarknetConv2D-BN-LeakyReLU initial feature extraction part in CSPDarknet53_tiny;

[0085] To improve the feature fusion part, after the YOLOv4-tiny backbone structure passes information to the feature fusion stage, the residual edge is introduced to form a dual-branch structure. One branch performs a 3×3 ordinary convolution to extract features to maintain the original (13,13,512) (26,26,256) feature map size, and the other branch uses an expanded convolution with an expansion factor of 3 to extract shallow features. The zero-padding rows are set to 3 to ensure that the input and output sizes are the same. After feature fusion, the dimension is set to out_channels / / 2, and then the feature integration processing is performed with the original valid layer features that introduce empty residual edges, while still maintaining the original dimension size.

[0086] The CA attention mechanism is added to the input stage of the improved dual-branch structure in the feature fusion part and the upsampling stage of the lower effective feature layer.

[0087] In the YOLO Head part, the parallel network is improved. The first layer is a 3×3 convolution operation to replace the positioning effect of the prediction box alone. Due to the addition of a parallel network, the input is first subjected to 1×1 convolution dimensionality reduction and spatially separable convolution improvement; the second layer of 5×5 convolution operation is also subjected to dimensionality reduction and spatially separable convolution processing, and a global average pooling operation is added after dimensionality reduction.

[0088] In the experimental operation, freeze-thaw training is adopted, which can put more resources on the following network parameters and then train them together, so that both time and resources can be well utilized.

[0089] The network was trained for 100 epochs, frozen for 50 epochs, and then unfrozen for 50 to 100 epochs.

[0090] The batch size of each batch is set to 8. After unfreezing the training, the batch size of each batch is set to 4.

[0091] The input image size is 416×416.

[0092] The base learning rate is 0.0001, the learning rate attenuation coefficient gamma is 0.92, the learning rate decrease interval step size is set to 1, and one GPU is used for training.

[0093] The neural network optimizer optimizer uses Adam.

[0094] The experiment in this embodiment was carried out under the hardware conditions of NVIDIA GTX 1080Ti.

[0095] The loss function selects the CIOU loss function. The CIOU loss includes three items: overlapping area, center point distance, and aspect ratio. The specific formula is as follows:

[0096] L CIOU =1-IOU(A,B)+ρ 2 (A ctr ,B ctr ) / c 2 +αυ

[0097]

[0098] where w gt and h gt Represent the width and height of the real box respectively, w and h represent the width and height of the predicted box respectively, and IOU represents the ratio of the intersection and union between the predicted box and the real box.

[0099] Input the data set images to conduct experiments and observe the experimental results.

[0100] The mAP value of the original network structure is Figure 2 As shown in the figure, the next ablation experiment is to make modifications based on the previous one to obtain the experimental results. The results of each ablation experiment are shown in the figure below. Figure 3 , 5 , 7, and 9.

[0101] Experimental results analysis

[0102] Although the FPS dropped slightly from 63 to 55 due to a slight increase in the number of parameters, the weight file size was reduced to 1 / 3 of the original, the accuracy was improved by 3.48%, and the loss curve continued to decline, achieving a better stable YOLOv4-tiny effect while improving the accuracy.

[0103] In summary, the present invention can improve detection accuracy, fully extract shallow information, and enhance the detection effect of small targets.

Claims

1. An improved target detection method of YOLOv4-tiny network, characterized by: The following steps are involved: Select part of VOC2007+VOC2012+COCO datasets as target detection datasets; S1: Modify the LeakyReLU activation function of the convolution module in the YOLOv4-tiny network backbone structure and introduce the RReLU activation function; S2: Improve the feature fusion part of the YOLOv4-tiny network. In the stage of transmitting feature information in the effective feature layer of the feature fusion part, residual edges are introduced to form a dual-branch structure. In the residual branch, dilated convolution is introduced to extract shallow features. After the other branch extracts features, feature fusion is performed with shallow features. Empty residual edges are introduced to retain the original feature information and the fused feature information for feature integration. S3: Add CA attention mechanism in the input stage of the improved dual-branch structure in the feature fusion part and the upsampling stage of the lower effective feature layer; S4: Improve the YOLO Head part of the YOLOv4-tiny network. The dual-branch structure allocates the tasks of bounding box position prediction and generation and feature classification respectively, and introduces global average pooling to assist the feature classification task; S5. Input the target detection dataset images into the improved YOLOv4-tiny network for experiments and observe the experimental results.

2. A target detection method according to claim 1 that improves the YOLOv4-tiny network, characterized in that: In the RReLU activation function of S1, (0.1-0.3) is selected as the uniform distribution interval of the negative slope. The RReLU activation function formula is as follows: Among them, x ji Represents the input of each layer, y ji represents the output of each layer, a ji Represents a random value drawn from a uniform distribution (0.1-0.3).

3. A target detection method according to claim 1 that improves the YOLOv4-tiny network, characterized in that: S2 specifically includes the following steps: After the backbone network transfers feature information to the feature fusion part, S21 introduces the residual edge and improves it into a double-branch structure; The dual-branch structure continues to perform 3×3 ordinary convolution feature extraction operations on the original branch side, and uses dilated convolution to extract features on the added residual side. The dilated convolution formula is as follows: k'=d×(k-1)+1 In the formula, k represents the input convolution kernel size, d represents the dilation coefficient, and k' represents the equivalent convolution kernel size after the dilated convolution operation; In order to ensure that the input and output sizes remain unchanged after the dilated convolution operation, the following convolution input and output calculation formula must be met: Among them, Z1 and Z2 are the sizes of the output and input feature maps respectively, P is the zero-padded row, and S is the step size; S22 introduces an empty residual edge to transmit the original feature information, and performs 1×1 convolution feature integration processing on the feature information output by the dual-branch structure and the original feature information transmitted through the empty residual edge.

4. A target detection method according to claim 3 that improves the YOLOv4-tiny network, characterized in that: The expansion coefficient is set to 3. After substituting it into the expansion convolution formula, the original 3×3 convolution is equivalent to the receptive field effect of 7×7 convolution.

5. A target detection method according to claim 1 that improves the YOLOv4-tiny network, characterized in that: S4 specifically includes the following steps: S41 changes the 3×3 convolution structure in the original YOLO Head part's 3×3 and 1×1 convolution structures to a dual-branch network structure; The first layer of the S42 dual-branch structure is a 3×3 convolution operation to replace the positioning effect of the prediction box alone; Perform 1×1 convolution on the input to reduce the dimension, and improve the 3×3 convolution operation to 1×3 and 3×1 spatially separable convolution to reduce the number of parameters; The second layer of the S43 dual-branch structure is a 5×5 convolution operation that replaces the task of classifying the detection results alone; Perform 1×1 convolution on the input to reduce the dimension, add a global average pooling layer after the dimension reduction, and improve the 5×5 convolution to 1×5 and 5×1 spatially separable convolutions; After the dual-branch feature information of S44 is fused, a 1×1 convolution operation is performed to achieve the required output feature dimension.

Images