A multi-target detection method based on an improved YOLOv5s network model

Abstract

This invention discloses a multi-target detection method based on an improved YOLOv5s network model. It utilizes a backbone network to extract feature information from the target image. Then, a global context module is introduced into the backend of the backbone network to obtain global relevance feature information of the target image, associating it with all regions of the entire image. Next, feature map information is further obtained through a neck module. An adaptive weight allocation module is integrated after the neck module (C3) to feed the feature map information obtained from the global context module into the adaptive weight allocation module, obtaining key feature information of the detected target, and thus acquiring predicted feature information. Finally, the predicted information is passed through a prediction network model based on an improved loss function to obtain the target's attribute label, thus determining the detection effect of the target. This improves the accuracy of multi-target detection and enhances the detection performance.

Description

Technical Field

[0001] This invention proposes a multi-target detection method based on an improved YOLOv5s network model, belonging to the field of target detection technology. Background Technology

[0002] In the field of computer vision, object detection has always been a hot research topic. It studies how to identify targets of interest and determine their location among numerous images, and has been widely applied in intelligent transportation, intelligent security, defect recognition, medical imaging, face detection, and remote sensing, with extremely broad application prospects. Currently, the main object detection algorithms based on deep learning theory include popular algorithms such as YOLOv3, R-CNN, CenterNet, YOLOv5, and YOLOv7. Among them, convolutional neural networks (CNNs) and their improved versions, such as fast R-CNN, R-CNN, and faster-RCNN, complete object detection in two steps. For example, Jiang et al. (Jiang D, Li G, Tan C, et al. Semantic segmentation for multiscaletarget based on object recognition using the improved Faster-RCNN model[J]. Future Generation Computer Systems, 2021, 123: 94-104.) improved the fusion method of RGB images and depth images to address the influence of uneven lighting in the environment. While enhancing the feature information of the fused image, it also improved the efficiency of model training. Meanwhile, in order to meet the needs of operating on multi-scale target objects, the non-maximum suppression algorithm was improved to enhance the model's performance. Yue Lei et al. (Yue Lei, Zhu Chengjie, Zhang Yi et al. Research on power operation safety early warning based on improved Faster R-CNN [J]. Automation Instrumentation, 2023, 44(09):76-81.) modified the feature network by combining the region proposal network and the residual network, and then used the convolutional neural network (R-CNN) for target detection. The results showed that the algorithm improved both accuracy and real-time performance. However, the two-step algorithm has problems such as cumbersome calculation, slow speed and low false negative rate. To improve these problems, scholars have proposed single-stage (single-step) detection algorithms, such as the mainstream YOLO family. Ge Wen et al. (Ge Wen, He Zhuoying, Qu Lele. Research on vehicle detection model based on improved YOLOv5[J]. Computer and Information Technology, 2023, 31(06):19-22.) used a bidirectional feature pyramid network to extract fine-grained features of different images, and combined the spatial pyramid pool fast structure and ELU activation function to propose an improved YOLO5 detection model, which improved the accuracy of vehicle model detection.Yu Chuntao et al. (Yu Chuntao, Li Jinyang, Shi Wenqiang et al. A Hypocotyl Color Detection Model for Hybrid Soybean Seedlings Based on Improved YOLOv7 [J]. Journal of China Agricultural University, 2024, 29(02): 11-22.) used the CARRAFE operator to improve the upsampling speed of YOLOv7 and used the SE attention mechanism module to obtain deeper feature information of the image. The results showed that the detection accuracy of the hypocotyl color of hybrid soybean seedlings was significantly higher than that of the traditional YOLOv7 detection model. The accuracy of the single-stage detection algorithm was improved, and the real-time performance was also improved. Although the YOLO family of algorithms has improved the target detection performance compared with the two-stage algorithm, there are still some shortcomings, such as insufficient concentration of receptive field, insufficient acquisition of target detail information, and difficulty in detecting overlapping targets. To solve the above problems, improved versions of YOLO have been proposed by researchers, such as YOLOv4, YOLOv5, YOLO5s, YOLOv5xYOLOv5s and other popular improved algorithms. YOLOv5s stands out for its high detection efficiency due to its novel convolutional modules and improved accuracy through multi-scale training. Furthermore, its modular design allows for easy adjustment and expansion of the network structure, making it a general-purpose model to some extent. This is the primary reason for choosing YOLOv5s as the object detection algorithm in this invention.

[0003] With the rapid development of deep network technology, although object detection algorithms have made significant breakthroughs, there are still many challenges and key technologies that need to be addressed. The main reasons for these problems are: (1) When acquiring contextual information, in order to obtain the global correlation features of the entire region, a large number of convolutional layers need to be added, resulting in a complex network structure, slow training speed, and low network efficiency. (2) Due to the complexity of the detection environment, the background environment has strong interference with the detection model, making the detection model insensitive to key features and difficult to effectively acquire key features. (3) In the design of the visual detection loss function, the difference between the predicted box and the set box can only be obtained synchronously according to the set width and height. It is difficult to adjust the width and height of the predicted box separately and adjust the difference between the predicted box and the set box at different scales. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes an improved multi-target detection method based on the YOLOv5s network, utilizing methods such as a backbone feature extraction network, a global context module, an adaptive weight allocation scheme, and an improved loss function.

[0005] The technical solution of this invention is:

[0006] A multi-object detection method based on an improved YOLOv5s network model is characterized by the following steps: Step 1: Construct the backbone network of the YOLOv5s object detection model, and use the backbone network to extract the feature information of the image to be detected to obtain feature information at different levels; then, select feature data with a pixel area of ​​640×640 from the feature data at different levels, and use the global context module to obtain the context data of the target features at different levels.

[0007] Step 2: Input the information obtained from the global context module into the Neck module for convolution, upsampling and concatenation operations to obtain C3 feature data;

[0008] Step 3: Pass the fused and transformed predicted feature map through the prediction network. Based on the three attributes of the detected target, the detection result can be obtained. The detection layer mainly includes a three-scale prediction head, corresponding to the downsampled feature map and multiple scale prediction heads. Target detection is performed on the corresponding feature map based on the previously designed anchor boxes. The target detection model, which includes a backbone feature extraction network, a global context module, an adaptive weight allocation weighted fusion method, and the corresponding prediction network, is trained. During the implementation process, each generation of the prediction model is continuously trained and tested to select the optimal network weights, while ensuring that the weights of the global context module are also optimal. Finally, the trained target detection model is obtained.

[0009] In Step 1, the standard convolution module in the backbone feature extraction network is used to calculate convolutions on the detected target image data and improve the convergence rate. The slicing module is mainly used to slice the detected target image or picture. Specifically, it performs a neighbor downsampling operation within the range of the detected target image, that is, it selects a value every other pixel. Following the above operation, four sampled image data of the detected target can be obtained. Since these four sampled images are highly identical in terms of information representation, the slicing operation will not lose the information of the detected target. Following the above steps, the width and height information of the predicted bounding boxes of the detected targets are converted into channel dimensions. Although the width and height of the predicted bounding boxes become 0.5 times their original size, the input channel dimensions expand to 4 times their original size. Then, convolution is performed on the expanded image to obtain a sampled feature map of twice the size without information loss. For example, 640×640×3 feature data is first transformed into 640×640×12 feature map data through the slicing operator, and then, through convolution, a 640×640×32 feature map data is obtained. The spatial pyramid pooling module extracts the target features from the feature map using pooling kernels. The output size is fixed according to the spatial pyramid pooling module, and finally, it is passed to the subsequent fully connected layers for output data processing.

[0010] A global context module is fused after the backbone feature extraction network to obtain global region correlations in the image, avoiding the problem of adding multiple convolutional layers. First, the self-attention mechanism in the global context module establishes the correlation between distant pixels in the feature image. The nonlocal neural network module calculates the function size between every two pixels to obtain the global correlation performance of the image. Second, the current image feature information is weighted and aggregated with the remaining image pixels. Finally, the global feature information of the current pixel in the current image is obtained. The formula for calculating the global correlation of each pixel is:

[0011]

[0012]

[0013] In the formula, j represents the index value of the pixel position, i represents the index value of the pixel position in all images, y represents the input feature map data of the nonlocal neural network module, λ(y) represents the normalization coefficient, and g(y) represents the input feature map data of the nonlocal neural network module. j ,y i A larger h(y) value indicates a stronger correlation between pixel j and pixel i. i ) represents the magnitude of the eigenvalue of y at position i.

[0014] Step two, following C3, introduces an adaptive weight optimization strategy. This module, added after the Neck module, comprises three parts: a shrinkage mechanism, a stimulus mechanism, and a broadcast mechanism. The shrinkage mechanism compresses the feature map dimension (N, C, W, H) to (N, C, 1, 1) using global pooling, effectively further integrating the data obtained from the previous global pooling module. The stimulus mechanism uses two fully connected layers to determine the weight contribution between different channels. Finally, the broadcast mechanism multiplies the weight values ​​with the original feature map to redistribute the weights according to their importance. The adaptive weight optimization strategy formula is as follows:

[0015]

[0016] f=σ1[W2σ2[W1I inout (4)

[0017]

[0018] In the formula, I in (i,j) represents the original weight values ​​of the input feature map channels, and W1 and W2 represent the first and second fully connected operations, respectively.

[0019] σ1[·] and σ2[·] represent the Sigmoid and ReLU activation functions, respectively. This indicates the weight redistribution information following the broadcast mechanism.

[0020] The loss function of the object detection model used in step three is defined as follows:

[0021]

[0022] In the formula, L Io L represents the overall regression loss term for the detection box. di L represents the confidence loss term for the detected target. as The value represents the aspect ratio penalty for the detected target, Io represents the overlap between the setpoint and the predicted bounding box, and ρ represents the aspect ratio penalty. 2 (w,w gt ) and C w These describe the difference in width between the predicted bounding box and the set bounding box, and the width of the minimum bounding rectangle, respectively. ρ 2 (b,b gt ) describes the distance between the center points of the bounding box and the prediction box; c describes the length of the diagonal of the minimum bounding rectangle; λ w and λ h These represent the wide weight and high weight of the detected target, respectively. The purpose of designing these two coefficients is to fine-tune the difference between the predicted bounding box and the set box, thereby improving the overlap between the predicted bounding box and the set box.

[0023] The main beneficial effects obtained by this invention are as follows: (1) This invention first extracts the feature information of the detected target based on the backbone feature network; it uses the global context module to obtain the context data of the detected target, obtains the global correlation of the image, avoids the problem of adding multiple convolutional layers, and improves detection efficiency and training speed. (2) In order to improve the sensitivity of the model to the key features of the detected target and avoid background interference, this invention introduces an adaptive weight allocation strategy into the Neck. Through the three mechanisms of shrinkage mechanism, stimulation mechanism and broadcast mechanism, the weight allocation between different channels is achieved to achieve the purpose of prioritizing the main and secondary, and enhances the key feature information of the detected target. (3) In the loss function, factors are introduced into the height and width aspect ratio penalty terms of the detected target respectively, so as to realize the real-time adjustment of the difference between the predicted box and the set box at different scales, and realize the efficient matching of the predicted box and the set box. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the overall target detection algorithm proposed in this invention.

[0026] Figure 2 This is a flowchart of the target detection algorithm model proposed in this invention.

[0027] Figure 3 This is a schematic diagram of the global context module proposed in this invention.

[0028] Figure 4 This is a schematic diagram of the adaptive weight allocation strategy proposed in this invention.

[0029] Table 1 shows the comparison results of MS COCO ablation experiments on the dataset.

[0030] Table 2 shows the comparison results of ablation experiments using the BSDS500 dataset.

[0031] Table 3 compares the performance of the proposed algorithm and the object detection algorithm on the MS COCO dataset.

[0032] Table 4 compares the performance of the proposed algorithm and the object detection algorithm on the BSDS500 dataset. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] A multi-target detection method based on an improved YOLOv5s network model includes the following steps: Step 1: As Figure 1As shown, feeding image information from the COCO or BSDS500 datasets into the backbone network yields feature data at different scales, labeled R1-R4. R1 represents the Conv and C3 functions; after convolution in R1, the image size is 320×320×32, and after C3, the image dimension is 160×160×64. R2, R3, and R4 represent the same operations; finally, after R4, the dimension is 20×20×512. Then, small, medium, and large target features (128×80×80, 256×40×40, and 512×20×20 respectively) are fed into the R5 global context module for further feature extraction. Here, we take a 128×80×80 image dimension as an example to illustrate the implementation process of the global context module. The feature map with a dimension of 128×80×80 is fed into the global context module, and dimensionality reduction is performed using three 1×1 convolutions with 128 channels, resulting in three outputs: 128×80×80, 128×8080, and 8080×128. Then, the 128×80×80 and 128×8080 are multiplied by a matrix to obtain an output of 8080×8080. After passing through the Softmax function, it is then multiplied by the 8080×128 feature map to obtain the 128×80×80 feature map. Finally, residual connections and other operations are used to obtain a 128×80×80 feature map with the same dimension as the input feature map. Following this method, feature maps with dimensions of 256×40×40 and 512×20×20 can also obtain the final output feature map through the global context module, realizing the global correlation performance calculation of the feature map image.

[0035] like Figure 1 In the diagram, R6, R7, and R8 represent the Neck module. R6 represents convolution, R7 represents upsampling, and R8 represents concatenation. Taking a 128×80×80 feature map as an example, the main working process of the Neck module is explained. When the 128×80×80 feature map output by C3 is fed into R6, 64×80×80 feature data is obtained. After upsampling by R7, 64×160×160 feature data is obtained. Finally, the concatenation operation by R8 yields 128×160×160 feature information. Similarly, feature maps with dimensions of 256×40×40 and 512×20×20 can also yield corresponding feature data with dimensions of 256×80×80 and 512×40×40.

[0036] Step Two: As Figure 1In the Neck section, after C3, an adaptive weight optimization strategy is added to obtain weight allocation information for different weight priority levels. Taking a 128×160×160 dimension as an example, the specific implementation process is illustrated. First, the 128×160×160 feature map is fed into the shrinking module for global average pooling, obtaining a 128×1×1 vector, where each channel can be described by a single numerical value. Then, after the first fully connected layer of the stimulation mechanism, a 128 / 4×1×1 vector is obtained. A second fully connected layer then yields another 128×1×1 vector. Next, a sigmoid activation function is applied to obtain a 128×1×1 weight vector f. Finally, the original 128×160×160 feature map is weighted and fused with the 128×1×1 vector to obtain an enhanced 128×160×160 feature map. Weight allocation with different priority levels was implemented. Similarly, the medium target feature data with dimensions of 256×80×80 and the large target feature data with dimensions of 512×40×40 can be fused into feature data of 256×80×80 and 512×40×40 dimensions through an adaptive weight allocation strategy.

[0037] Step 3: The fused and transformed predicted feature maps with dimensions of 128×160×160, 256×80×80, and 512×40×40 are passed through an R10 prediction network based on a loss function. Based on the three attributes of the detected target, the detection result can be obtained. The detection layer mainly consists of three-scale prediction heads, corresponding to downsampled feature maps and prediction heads at multiple scales. Target detection is performed on the corresponding feature maps based on the previously designed anchor boxes.

[0038] The hardware platform configuration for implementing this invention is: E5-2650V4 CPU, 1TB memory, NVIDIA RTX Titan graphics card; the software platform configuration is: Ubuntu 18.04, Python 3.8, PyTorch 2.1.2.

[0039] To demonstrate the advantages of the proposed object detection algorithm model, we selected the publicly available datasets MS COCO and BSDS500 for comparative experiments. In the ablation experiment, the mainstream CBAM mechanism was compared with that of Cao et al. [Woo S, Park J, Lee JY, et al. Cbam: Convolutional block attention module [C] / / Proceedings of the Europeanconference on computer vision (ECCV). 2018:3-19.]. The mainstream detection algorithms were selected for comparison, such as the faster R-CNN proposed by Cao et al. [Cao C, Wang B, Zhang W, et al. An improved faster R-CNN for small object detection [J]. IEEE Access, 2019, 7:106838-106846.], the YOLOX-Tiny proposed by Chen et al. [Chen Gaoru, Chen Hui, Gao Xiang et al. Substation safety wearing behavior detection based on YOLOX-Tiny cascade algorithm [J]. Journal of Fuzhou University (Natural Science Edition), 2023, 51(01):56-61.], and the Resnet-50 proposed by Rezende et al. [Rezende E, Ruppert G, Carvalho T, et al.]. al.Malicious software classification using transfer learning of resnet-50deep neural network[C] / / 2017 16th IEEEinternational conference on machine learning and applications(ICMLA).IEEE,2017:1011-1014.], YOLOv5 proposed by Liu et al.[Liu Y, Lu BH, Peng J, et al.Research on the use of YOLOv5 object detection algorithm in mask wearing recognition[J].World Scientific Research Journal,2020,6(11):276-284.], YOLOv5s proposed by Wang et al.[Wang D,He D.Seven object detection algorithms were proposed, including Channel-pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning [J]. Biosystems Engineering, 2021, 210: 271-281., DeepReg proposed by He et al. [W.He, X.Zhang, F.Yin, and C.Liu. Deep direct regression for multi-oriented scene text detection. In Proc. Int.Conf.Comput.Vision, 2017.], and SE-SSD proposed by Zheng et al. [Zheng W, Tang W, Jiang L, et al. SE-SSD: Self-ensembling single-stage object detector from point cloud [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 14494-14503.]. Tables 1 and 2 show the statistical results of comparing the mainstream CMAM mechanisms using the MS COCO dataset and the BSDS500 dataset, respectively. .

[0040] Table 1 Comparison of MS COCO ablation experimental results

[0041]

[0042] Table 2 Comparison Results of Ablation Experiments Using BSDS500 Dataset

[0043]

[0044] A comparison of the results in Tables 1 and 2 shows that the algorithm proposed in this invention has better detection capabilities. Although CMAM can simultaneously obtain feature data from both spatial and channel aspects, the computational complexity increases with the number of images, leading to a decrease in the model's generalization ability. This invention employs an adaptive weight allocation strategy. Although it can only obtain feature data for each channel, the global context module can acquire global feature information, further compensating for the lack of spatial feature information. Furthermore, the proposed algorithm is computationally simple and has strong model generalization ability, thus exhibiting superior efficiency and performance. Table 3 shows a comparison of the performance of the proposed algorithm and the object detection algorithm on the MS COCO dataset. Table 4 shows a comparison of the performance of the proposed algorithm and the object detection algorithm on the BSDS500 dataset.

[0045] Table 3 compares the performance of the proposed algorithm and the object detection algorithm on the MS COCO dataset.

[0046]

[0047] Table 4 compares the performance of the proposed algorithm and the object detection algorithm on the BSDS500 dataset.

[0048]

[0049] As shown in Table 3, compared with the other seven algorithms, the proposed algorithm shows significant improvements in recall, precision, and balanced F-score. For example, in terms of precision, the proposed algorithm achieves an 8.66% improvement over Faster R-CNN, an 8.55% improvement over DeepReg, a 7.79% improvement over SE-SSD, a 5.23% improvement over YOLOX-Tiny, a 4.39% improvement over ResNet-50, a 3.15% improvement over YOLOv5, and a 1.59% improvement over YOLOv5s. Table 4 shows that in terms of precision improvement, the proposed object detection algorithm achieves an 8.99% improvement over Faster R-CNN, an 8.28% improvement over DeepReg, a 7.86% improvement over SE-SSD, a 6.07% improvement over YOLOX-Tiny, a 4.98% improvement over ResNet-50, a 4.16% improvement over YOLOv5, and a 2.2% improvement over YOLOv5s. Based on the results of the ablation experiments and the comparison experiments of target detection algorithms in Tables 1, 2, 3 and 4, it can be concluded that the target detection algorithm model proposed in this invention outperforms the other 7 target detection algorithms on the MS COCO and BSDS500 public datasets.

Claims

1. A multi-target detection method improved based on a YOLOv5s network model, characterized in that: The steps include: Step 1: Construct the Backbone network of the YOLOv5s object detection model, use the backbone network to extract the feature information of the image to be detected, and obtain feature information at different levels; then, select the feature data with a pixel area of ​​640×640 from the feature data obtained at different levels, and use the global context module to obtain the context data of the target features at different levels. Step 2: Input the information obtained from the global context module into the Neck module for convolution, upsampling and concatenation operations to obtain C3 feature data; Step 3: Pass the fused and transformed predicted feature map through the prediction network to obtain the detection result of the detected target based on the three attributes of the detected target. The detection layer mainly includes a three-scale prediction head, corresponding to the downsampled feature map and multiple scale prediction heads. Target detection is performed on the corresponding feature map based on the previously designed anchor boxes. The target detection model, which includes a backbone feature extraction network, a global context module, an adaptive weight allocation weighted fusion method, and the corresponding prediction network, is trained. During the implementation process, each generation of the prediction model is continuously trained and tested to select the optimal network weights, while ensuring that the weights of the global context module are also optimal. Finally, the trained target detection model is obtained. Step two, following C3, introduces an adaptive weight optimization strategy. This module, added after the Neck module, comprises three parts: a shrinkage mechanism, a stimulus mechanism, and a broadcast mechanism. The shrinkage mechanism reduces the feature map dimension through global pooling. Compress to This is equivalent to a further fusion of the data obtained from the global module above; the stimulation mechanism realizes the weight contribution between different channels through two fully connected layers. Finally, the broadcast mechanism is used to multiply the weight values ​​with the original feature map to achieve the purpose of redistributing the weight values ​​between different channels according to their primary and secondary importance. The adaptive weight optimization strategy formula is as follows: In the formula, This represents the original weight values ​​of the input feature map channels. and These represent the first and second full-connect operations, respectively. and They represent and Activation function This indicates the redistribution of weights after the broadcast mechanism; The loss function of the object detection model used in step three is defined as follows: In the formula, This represents the overall regression loss term for the detection box. This represents the confidence loss term for the detected target. This indicates the aspect ratio penalty for the detected target. This indicates the overlap between the set value and the prediction box. and These describe the difference between the width of the predicted bounding box and the set bounding box, and the width of the minimum bounding rectangle, respectively. Describe the distance between the center points of the bounding box and the prediction box. This describes the length of the diagonal of the minimum bounding rectangle; and These represent the wide weight and high weight of the detected target, respectively. The purpose of designing these two coefficients is to fine-tune the difference between the predicted bounding box and the set box, thereby improving the overlap between the predicted bounding box and the set box.

2. The multi-target detection method based on the improved YOLOv5s network model according to claim 1, characterized in that: In Step 1, the standard convolution module in the backbone feature extraction network calculates convolutions on the detected target image data and improves the convergence rate. The slicing module mainly performs slicing on the detected target image or picture. Specifically, it performs neighbor downsampling within the range of the detected target image, that is, selects a value every other pixel. Following the above operation, four sampled image data of the detected target are obtained. Since these four sampled images have the same height in terms of information representation, the slicing operation does not lose the information of the detected target. Following the above operation, the width and height information of the predicted bounding box of the detected target are converted into channel dimensions. Although the width and height information of the predicted bounding box become 0.5 times the original, the input channel dimension expands to 4 times the original. Then, convolution calculation is performed on the expanded image to obtain a sampled feature map of 2 times the size without information loss. After the backbone feature extraction network, a global context module is fused to obtain global region associations in the image, avoiding the problem of adding multiple convolutional layers. First, the self-attention mechanism in the global context module is used to establish the association between distant pixels in the feature image. The nonlocal neural network module uses the function size between every two pixels to obtain the global correlation performance of the image. Second, the current image feature information is weighted and aggregated with the remaining image pixels. Finally, the global feature information of the current pixel in the current image is obtained. The formula for calculating the global correlation of each pixel is: In the formula, The index value representing the pixel position. This represents the index value of a pixel in all images. This represents the input feature map data of a nonlocal neural network module. Represents the normalization coefficient. The larger the value, the more pixels... and pixels The greater the correlation; express exist The magnitude of the feature value at the location.

Images