Lightweight target detection method

By replacing YOLOv4's backbone network with Mobilenetv2 and embedding the attention mechanism, combining deep separable convolution and ASFF algorithm, the problem of low detection accuracy of lightweight object detection network is solved, and efficient feature extraction and accuracy improvement is achieved. The average accuracy of 82.31% and the detection speed of 22.10FPS.

CN119006971APending Publication Date: 2024-11-22SOUTHWEAT UNIV OF SCI & TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310562084.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing lightweight object detection network has low detection accuracy and cannot meet the requirements of higher detection accuracy.

Method used

Replace YOLOv4's backbone network CSPdarknet53 with Mobilenetv2, and embed the Coordinate Attention module in Mobilenetv2 to enhance feature extraction capabilities, replace the standard convolution layer in PANet as a depth separable convolution, and apply it in the detection head. The ASFF algorithm performs adaptive feature weighted fusion, using ReLU6 activation function to reduce the accuracy loss of low-precision devices.

Benefits of technology

The detection accuracy is improved, the parameters of the lightweight network model are reduced, the detection speed is improved, the average accuracy is 82.31%, the model size is only 52.65M, and the detection speed FPS is 22.10, exceeding the accuracy and speed of YOLOv4-tiny and YOLO5s.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure FT_1
    Figure FT_1
  • Figure FT_2
    Figure FT_2
Patent Text Reader

Abstract

The invention discloses a lightweight target detection method with relatively high precision, and the method comprises the steps: improving a YOLOv4 network model, replacing a backbone network of the YOLOv4 with Moilentv2, and replacing a neck structure of the YOLOv4 and 3 * 3 convolution in a detection head with deep separable convolution, thereby greatly reducing the quantity of network parameters; a CA attention module is embedded in the backbone network, the problem that the deep separable convolution feature extraction capability is insufficient is solved by establishing a dependency relationship of channel dimensions, and the algorithm detection precision is further improved; the PANet is combined with a feature weighted fusion algorithm ASFF, and the multi-scale feature extraction capability of the neck structure is optimized. According to the method, the network model parameter scale is reduced under the condition of slightly reducing the target detection precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to target detection based on deep learning methods. Background Art

[0002] Object detection algorithms based on deep learning can generally be divided into two methods: anchor-based and anchor-free. The anchor-based method mainly manually designs multiple prior boxes (anchor-boxes), and then detects the objects in each box separately. The anchor-free method uses some techniques to save the tedious step of manually designing the prior box and the huge memory overhead it brings, such as methods based on the joint expression of multiple key points, methods based on single center point prediction, and various detection boxes and target boxes (ground truth) matching algorithms. Compared with anchor-based, the anchor-free method does not require post-processing and has a faster detection speed, but the feature of no prior box constraints will lead to a large number of false detections.

[0003] The anchor-based method can be roughly divided into two-stage and one-stage target detection algorithms. The two-stage target detection algorithm represented by the R-CNN (Region-Convolution Neural Networks) series first extracts the prior box where the target may exist, and then performs more subtle screening and fine-tuning on the extracted prior box to finally obtain the detection result. The one-stage target detection algorithm represented by the YOLO (You Only Look Once) series and SSD directly classifies the target in the prior box and regresses the position and size of the detection box of the specific target. Due to the problems of positive and negative samples and the imbalance of difficult and easy samples, the one-stage target detection algorithm is difficult to achieve the accuracy level of the two-stage target detection algorithm, but this simple structural design also makes the detection speed of the one-stage detector far exceed that of the two-stage detector. Most practical applications have certain requirements for detection speed, so many studies in recent years have focused on the one-stage detection algorithm.

[0004] YOLOv4 (You Only Look Once version 4) and YOLOv5 (You Only Look Once version 5) are two single-stage target detection algorithms proposed by different teams. Both are improvements on YOLOv3 (You Only Look Once version 3). The performance of YOLOv4 and YOLOv5 is very similar, but YOLOv4 is more customizable. YOLOv4-tiny and YOLOv5s are lightweight networks of YOLOv4 and YOLOv5 respectively. Although the model is very small, only about 20M, the mean average precision (mAP, mean Average Precision) is low and cannot meet the requirements of higher detection accuracy. Therefore, it is necessary to propose a more accurate lightweight target detection algorithm. Summary of the invention

[0005] The present invention aims to solve the problem of low detection accuracy of existing lightweight object detection networks.

[0006] First, the backbone network CSPdarknet53 used by YOLOv4 is replaced with Mobilenetv2. After the replacement, the network model parameters are greatly reduced. Considering that the deep separable convolution has limited ability to extract information in the channel dimension, the attention mechanism (Coordinate Attention) module is embedded in Mobilenetv2 to enhance the feature extraction ability of the backbone network in the channel dimension. The CA module structure is as follows Figure 1 shown.

[0007] The attention module is embedded in the reverse residual block of Mobilenetv2. The specific embedding method is as follows: Figure 2 shown.

[0012] After the features are deep convolution, they are passed through the CA module to generate feature maps with channel attention based on the X and Y directions, and then multiplied with the output of the deep convolution to establish the channel dimension dependency of the features. The backbone network structure of the lightweight target detection network model is shown in Table 1. Among them, t, c, n and s are all parameters for establishing the network. t represents the coefficient of channel expansion before deep convolution, c represents the number of output channels, n represents the number of times the module is repeated in each group of networks, and s represents the step size corresponding to the convolution network of the first module of each group of networks. The first reverse residual block does not expand the number of channels, and the channel dimension information is small, so the CA module is not added. Finally, feature maps with scales of 132×320, 262×96 and 522×32 are selected and sent to the subsequent network.

[0013] Table 1 MA-YOLOv4 backbone network structure

[0014] Input Operator t c n s <![CDATA[416 2 ×3]]> Conv2d - 32 1 2 <![CDATA[208 2 ×32]]> IRblock 1 16 1 1 <![CDATA[208 2 ×16]]> IRblock+CA 6 24 2 2 <![CDATA[104 2 ×24]]> IRblock+CA 6 32 3 2 <![CDATA[52 2 ×32]]> IRblock+CA 6 64 4 2 <![CDATA[52 2 ×64]]> IRblock+CA 6 96 3 1 <![CDATA[26 2 ×96]]> IRblock+CA 6 160 3 2 <![CDATA[13 2 ×160]]> IRblock+CA 6 320 1 1

[0015] PANet uses a large number of standard convolutional networks to complete feature extraction and downsampling, which is a large part of the model memory overhead in the entire neck structure. In order to further reduce its model parameters, the 3×3 standard convolutional layer in PANet and the three detection heads (YOLO Head) is replaced with a depth-wise separable convolution. At the same time, the activation function also needs to be changed. The activation range of ReLU is not limited, and the output range is 0 to positive infinity. If the activation value is very large and distributed in a large range, the low-precision 16-bit floating point number in the mobile device will not be able to accurately describe such a large range of values, resulting in precision loss. ReLU6 compresses the output range to within 6. The specific calculation formula is as follows (1):

[0016]

[0017] Replacing the Leaky ReLU activation function in PANet with the ReLU6 activation function can effectively reduce the loss of detection accuracy in low-precision devices.

[0018] In the original PANet, the extracted features of three different scales are directly sent to the detection head to complete the final target detection, while the lightweight target detection network model uses the ASFF algorithm to perform adaptive weighted feature fusion on the three-layer features output by PANet. Features of different scales cannot be directly fused. The present invention designs the corresponding scale change strategies for the 1-(132×512), 2-(262×256) and 3-(522×128) feature scales output by PANet, as follows:

[0019] (1) 1→2: First, the number of channels is compressed by 1×1 convolution, and then the features are interpolated by the nearest neighbor method. 2 →26 2 Upsampling;

[0020] (2) 1→3: First, the number of channels is compressed by 1×1 convolution, and then the features are interpolated by the nearest neighbor method. 2 →52 2 Upsampling;

[0021] (3) 2→1: Directly complete channel expansion and 26 through a set of 3×3 convolutional blocks 2 →13 2 Downsampling work;

[0022] (4) 2→3: First, the channel dimension is compressed by 1×1 convolution, and then the features are interpolated by nearest neighbor method. 2 →52 2 Upsampling;

[0023] (5) 3→1: First, use the maximum pooling layer to complete 52 2 →26 2 Then a set of 3×3 convolution blocks are used to simultaneously complete channel expansion and 26 2 →13 2 Downsampling work;

[0024] (6) 3→2: Directly complete channel expansion and 52 through a set of 3×3 convolutional blocks 2 →26 2 downsampling work.

[0025] The specific weighted calculation method formula is as follows (2):

[0026]

[0027] in represents the input feature vector, l Represents the scale of the corresponding feature of the current detection head, n Represents the scale before feature transformation (for YOLOv4 ), ( i , j ) represents the position of the eigenvector, Represents the scale transformation of features, Representatives to be sent in l The feature vector of the scale detection head, Represent the weights corresponding to the feature vectors at different scales and positions, , and meet , the specific calculation formula is as follows (3):

[0028]

[0029] It can be seen is defined by the softmax formula, where They are This mapping is completed through a 1×1 convolutional layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the Coordinate Attention network structure described in the present invention is shown.

[0031] Figure 2A schematic diagram of the combined structure of Coordinate Attention and Mobilenetv2 described in the present invention is shown. DETAILED DESCRIPTION

[0032] Hereinafter, the lightweight object detection method of the present invention will be described in detail with reference to exemplary embodiments and accompanying drawings.

[0033] Exemplary Embodiments

[0034] In an exemplary embodiment of the present invention, the lightweight target detection method comprises the following steps:

[0035] S31. Use the PASCAL VOC dataset, which consists of 20 categories of images: aero, bike, bird, boat, bottle, bus, car, cat, chair, cow, table, dog, horse, motorbike, person, plant, sheep, sofa, train, tv. A total of 16,511 images from the training set and validation set of VOC2007 and VOC2012 are used as the training dataset for this experiment, and a total of 4,952 images from the VOC2007 test set are used as the test dataset for this experiment, and data enhancement is achieved through the Mosaic algorithm;

[0036] S32. Add a fully connected network as a classification layer to the backbone network of the lightweight object detection network and complete the classification task pre-training on the ImageNet dataset;

[0037] S33, load the pre-trained weights, set the initial learning rate to 0.001, select the Adam optimizer, set the momentum parameter to 0.937, the weight decay coefficient to 0, select the cosine annealing decay algorithm for learning rate decay, set the maximum learning rate to 0.01 times the initial learning rate, set the number of epochs of all samples to 170, freeze the backbone network for the first 50 epochs to speed up the training, set the frozen training batch size to 32, and set it to 16 after thawing;

[0038] S34. In the test phase, the mean average precision (mAP) is used to evaluate the detection performance. The precision (Precision) and recall (Recall) are calculated by the intersection over union (IoU). Then a PR (Precision-Recall) curve is drawn. A line is drawn to the left of the peak point and intersects with the vertical line of the previous peak. The area is calculated as AP (Average Precision). This method is used to calculate the AP of all categories, and then the average is mAP. Test results

[0039] The experimental platform system used for training and testing uses Win10, the CPU uses Core i5-10400F, and the GPU uses NVIDIA GeForce RTX 3060. Experiments on the PASCAL VOC dataset show that the average precision (mAP) of the lightweight object detection network reaches 82.31%, which is 4.53 percentage points lower than YOLOv4, 9.94 percentage points higher than YOLOv4-tiny, 7.61 percentage points higher than YOLOv5s, and 4.97 percentage points higher than EfficientDet-d0; the model size is only 52.65M, which is 191.64M smaller than YOLOv4, and the detection speed FPS is 22.10. The AP values ​​of detection accuracy of different categories are shown in Table 2.

[0040] Table 2 Test results

[0041] Target Type Accuracy Recall AP Car 92.17% 81.35% 92% Aeroplane 94.65% 80.70% 90% Motorbike 94.06% 82.77% 90% Bus 91.24% 83.10% 90% Cat 92.90% 84.08% 90% Bicycle 90.16% 81.60% 89% Dog 84.52% 82.62% 88% Train 90.70% 82.98% 88% Horse 93.44% 81.90% 87% Person 90.35% 74.05% 86% Cow 87.44% 79.92% 85% Bird 88.31% 74.07% 84% Sheep 81.86% 76.45% 83% Tvmonitor 86.22% 71.10% 80% Sofa 71.10% 78.24% 79% Dining table 70.18% 74.27% 79% Boat 76.11% 65.40% 76% Bottle 80.63% 54.16% 67% Chair 70.96% 50.66% 60% Pottedplant 73.87% 44.17% 58%

Claims

1. A lightweight target detection method, characterized in that: The lightweighting refers to reducing the number of network model parameters; The target detection refers to predicting the rectangular range frame of the target to be detected by using the network model; The network model refers to a deep learning model obtained by modifying part of the YOLOv4 structure using an improved method; The improved method refers to replacing the backbone network of YOLOv4 with Mobilenetv2, replacing the YOLOv4 neck structure and the 3×3 convolution in the detection head with a depth-separable convolution, embedding a CA attention module in the replaced backbone network, and combining PANet with the feature weighted fusion algorithm ASFF to obtain an improved lightweight target detection network model; The MobileNetv2 includes a standard convolutional layer, a depth-separable convolutional layer, a pooling layer, an activation layer, a normalization layer, an attention module, and a residual module supporting the network; The depth-wise separable convolution refers to significantly reducing the amount of convolution operation data through grouped convolution; The pooling layer refers to the maximum pooling layer; The activation layer refers to the Relu6 activation function; The normalization refers to batch normalization; The attention module refers to the attention module embedded in the 2nd to 6th groups of reverse residual blocks in Mobilenetv2, and the specific position is after the group convolution; The residual module refers to a skip connection method that skips the intermediate network and fuses feature maps using a point-by-point addition method; The feature map is the three-layer feature map that is finally output by the backbone network and sent to the subsequent neck network; The three layers of feature maps are images with scales of 52×52×128, 26×26×256, and 13×13×512 respectively; The neck network refers to a lightweight neck network based on PANet; The weighted feature fusion algorithm ASFF is to scale features of different scales to the same scale through nearest neighbor interpolation and convolution operations to complete the fusion work; The detection head refers to a network module constructed using standard 1×1 convolution and 3×3 convolution, with a total of three detection head modules corresponding to feature maps of three scales; The scales of the feature maps corresponding to the three scales are 52×52×128, 26×26×256, and 13×13×512 respectively; A lightweight target detection method comprises the following steps: S1. Use the training data set to train the network model and obtain the complete model parameters; S2, load the trained model parameters and test the network model using the test data set; Training the network model includes the following steps: S101, pre-training the backbone network of the network model on the ImageNet dataset to obtain pre-trained model parameters; S102, loading pre-trained model parameters; S103, set the initial learning rate to 0.001, the momentum parameter of the Adam optimizer to 0.937, the weight decay coefficient to 0, set the learning rate decay to the cosine annealing decay algorithm, the maximum learning rate to 0.01 times the initial learning rate, the number of epochs of the full sample to 170, and in order to speed up the training speed, freeze the backbone network in the first 50 epochs, and set the frozen training batch size Batchsize to 32, and set it to 16 after thawing; S104, inputting a total of 16,511 images of the training set and validation set of the public data set PASCAL VOC 2007 and PASCAL VOC 2012 into the network model training as the training data set; S105, obtaining training model parameters of the network model on the PASCAL VOC 2007 and PASCAL VOC 2012 datasets; Testing the network model involves the following steps: S201, loading the trained model parameters on the network model; S202, input the test set of 4952 pictures in the public data set PASCAL VOC2007, and obtain the prediction results; S203, calculating the precision rate (Precision) and the recall rate (Recall) according to the prediction results; S204, calculating the AP value of each category according to the precision and recall; S205: Summarize the AP values ​​of all categories and take their average value to obtain the final detection result mAP.

Citation Information

Cited By

  • High-performance construction site safety helmet detection method based on YOLO11 improvement

    CN120564109A