An object detection method based on adaptive multi-level fusion feature pool
By introducing a multi-level fusion feature pool and batch channel mean structure in the YOLOv5 network, the problem of insufficient number of feature channels and insufficient importance distinction is solved, the detection accuracy and inference speed are improved, and the calculation amount is reduced and the accuracy is maintained.
Patent Information
- Application Number
- CN202211617845.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-15
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2042-12-15
AI Technical Summary
The existing single-level object detection algorithm has problems in the YOLO series network structure of the number of feature channels, insufficient distinction between feature channels and inability to reasonably allocate through self-learning, resulting in insufficient balance between detection accuracy and inference speed.
The neck network of YOLOv5 network has introduced a multi-level fusion feature pool, batch channel mean structure and mean compression expansion structure. The importance of feature channels is adjusted adaptively, and the multi-level fusion feature pool is used to fuse features at different levels, and the batch channel mean structure and mean compression and expansion structure are used to compress and expand features to generate weights to solve the above problems.
The utilization rate of feature channels is improved, detection accuracy is maintained, the amount of computation is reduced, and the inference speed is improved. The overall computational complexity of the YOLOv5 model is reduced by 1/5 and the mAP fluctuates within a small range.
Smart Images
Figure CN116342987B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence, deep learning and target detection, and in particular to a target detection method based on an adaptive multi-level fusion feature pool. Background Art
[0002] In recent years, the field of object detection has developed rapidly. As application scenarios have become increasingly diverse, the problems that need to be solved have also become increasingly complex. Ultimately, however, object detection algorithms must address two key issues: detection accuracy and inference speed. Based on these two key issues, object detection is divided into two main areas: two-stage object detection and single-stage object detection. As the name suggests, two-stage object detection algorithms require two steps to detect an object. Therefore, these algorithms focus more on detection accuracy and perform poorly in terms of inference speed. Since the introduction of single-stage object detection algorithms, optimizing and balancing detection accuracy and inference speed has been a key focus in this area because they simplify the detection steps.
[0003] Single-stage object detection technology offers advantages such as fast inference speed, low equipment overhead, and low model computational complexity. It facilitates rapid identification of targets within massive images or real-time video streams, and the execution of logical inferences based on the identification results. Single-stage object detection algorithms are particularly well-suited for applications requiring stringent real-time performance, such as security and industrial applications. In security, object detection is primarily used to address real-time analysis of camera video feeds, including hazardous material warning (detecting objects that pose a public safety risk, such as knives, guns, and flammable and explosive materials), perimeter warning (setting safe or dangerous areas to prevent entry), and fire warning (identifying and detecting fires). These real-time video analyses are highly valuable for public safety prevention and early warning. Furthermore, in industrial applications such as real-time defect detection and production line anomaly identification, single-stage object detection algorithms can help companies reduce costs, increase efficiency, and improve quality and production, demonstrating their practical application value. Currently, representative single-stage object detection algorithms include YOLOv1, SSD, YOLOv2, YOLOv3, YOLOv4, Scaled-YOLOv4, YOLOv5, YOLOv6, and YOLOv7. Furthermore, innovative variants of the YOLO algorithm have been developed, such as YOLOF, PP-YOLO, and YOLOX. These algorithms excel in both detection accuracy and inference speed. However, three major challenges remain:
[0004] 1. In the neck structure of the YOLO series network structure, the feature branch fusion process has the problem of too few feature channels and cannot fully grasp the global features;
[0005] 2. The feature channels after feature branch fusion do not distinguish the importance of all feature channels;
[0006] 3. It is not possible to reasonably allocate the importance of feature channels through self-learning. Summary of the Invention
[0007] To solve the above problems, the present invention provides a target detection method based on an adaptive multi-level fusion feature pool, comprising the following steps:
[0008] S1. Build an improved YOLOv5 network based on the existing YOLOv5 network. The improved YOLOv5 network includes a backbone network, a neck network, and a head network. The neck network incorporates a multi-level fusion feature pool, a batch channel mean structure, and a mean compression and expansion structure.
[0009] S2. Obtain an image dataset to train an improved YOLOv5 network, perform iterative training using a loss function, and adjust network parameters.
[0010] S3. Use the trained improved YOLOv5 network for target detection to obtain target detection results.
[0011] Furthermore, the improved YOLOv5 network includes a second CSP1 module, a third CSP1 module and a second Conv module;
[0012] The output features of the second CSP1 module are obtained, and the down-sampling results are obtained by passing them through a convolution layer with a stride of 2 and a convolution kernel of 3×3, and the down-sampling results are used as the first input features of the multi-level fusion feature pool; the output features of the third CSP1 module are obtained, and the down-sampling results are used as the second input features of the multi-level fusion feature pool; the output features of the second Conv module are obtained, and the up-sampled results are used as the third input features of the multi-level fusion feature pool; the multi-level fusion feature pool fuses the first input features, the second input features and the third input features to obtain a feature splicing map, and the feature splicing map is input into the batch channel mean structure, and the output of the batch channel mean structure is sent to the mean compression and expansion structure.
[0013] Furthermore, the improved YOLOv5 network also includes a first CSP2 module, a second CSP2 module and a third CSP2 module; the first input feature is multiplied by the output of the mean compression and expansion structure, and the multiplication result is upsampled and input into the first CSP2 module; the second input feature is multiplied by the output of the mean compression and expansion structure, and the multiplication result is input into the second CSP2 module; the third input feature is multiplied by the output of the mean compression and expansion structure, and the multiplication result is downsampled and input into the third CSP2 module.
[0014] Furthermore, the scales of the first input feature, the second input feature, and the third input feature are all the same.
[0015] Furthermore, the batch channel mean structure includes a spatial mean module and a batch mean module; first, the spatial mean of the output feature map of the multi-level fusion feature pool is calculated by the spatial mean module; then the spatial mean of the output feature map is sent to the batch mean module to calculate the batch mean.
[0016] Furthermore, the calculation formulas of the spatial mean module and the batch mean module are:
[0017]
[0018]
[0019] in, represents the spatial mean, represents the batch mean, w represents the width of the output feature map of the multi-level fusion feature pool, h represents the height of the output feature map of the multi-level fusion feature pool, b represents the number of training batches, and p ij Represents the feature value of the i-th column and j-th row in the output feature map p, q m Represents the spatial mean of the m-th output feature map.
[0020] Furthermore, the mean compression and expansion structure uses two 1×1 convolutions to compress and expand the output of the batch channel mean structure in sequence, and uses the sigmoid function to process the output of the mean compression and expansion structure to obtain channel features.
[0021] Beneficial effects of the present invention:
[0022] The present invention proposes a multi-level fusion feature pool structure. By applying this multi-level fusion feature pool structure to fuse the feature outputs of different levels in the neck part PAN structure, the problem of too few feature channels is solved. In addition, the present invention combines the batch channel mean structure and the mean compression and expansion structure for the fused channel features, compresses and expands the fused channel features, and multiplies the expanded features with the fused channel features. In this way, weights for different channel features are generated, and the weights are associated with the feature data of the detected image itself, solving the problem of the importance of different channels. Finally, because the batch channel mean features need to undergo channel compression and expansion through convolution, the parameters of the convolution kernel need to be trained and learned, solving the problem of allocating weights to different channels through self-learning.
[0023] This paper proposes and uses a new structure based on the algorithm model of yolov5, which reduces the overall computational complexity by 1 / 5 and keeps the mAP fluctuation within a very small range, without much impact on the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a flow chart of the target detection method of the present invention;
[0025] Figure 2 The improved YOLOv5 network structure of the present invention is added with a multi-level fusion feature pool, a batch channel mean structure, and a mean compression and expansion structure;
[0026] Figure 3 Schematic diagram of the feature fusion process of the multi-level fusion feature pool of the present invention;
[0027] Figure 4 This is a schematic diagram of the mean compression and expansion structure of the present invention;
[0028] Figure 5 This is a Sigmoid function curve diagram used in the present invention. DETAILED DESCRIPTION
[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0030] The present invention provides a target detection method based on adaptive multi-level fusion feature pool. Figure 1 As shown, the following steps are included:
[0031] S1. Build an improved YOLOv5 network based on the existing YOLOv5 network. The improved YOLOv5 network includes a backbone network, a neck network, and a head network. The neck network incorporates a multi-level fusion feature pool, a batch channel mean structure, and a mean compression and expansion structure.
[0032] S2. Obtain an image dataset to train an improved YOLOv5 network, perform iterative training using a loss function, and adjust network parameters.
[0033] S3. Use the trained improved YOLOv5 network for target detection to obtain target detection results.
[0034] In one embodiment, the present invention adds a multi-level fusion feature pool, a batch channel mean structure and a mean compression expansion structure to the neck part of the existing YOLOv5 network to form Figure 2The improved YOLOv5 network structure shown.
[0035] Specifically, the multi-level fusion feature pool forms a feature pool of a mid-scale feature map by fusing the output features of the three branches of the neck part; Figure 2 and Figure 3 As shown, from the second CSP1 module ( Figure 2 The first branch extends from the CSP1 layer (numbered 3), which outputs the feature map of N channels ( Figure 3 The N-channel feature map is downsampled by a convolution layer with a step size of 2 and a convolution kernel of 3×3, and the downsampled result is used as the first input feature of the multi-level fusion feature pool; from the third CSP1 module ( Figure 2 The second branch extends from the CSP1 layer (numbered 4), which outputs a feature map of 2N channels ( Figure 3 The 4-level output feature map in the 2N channel is used as the first input feature of the multi-level fusion feature pool; from the second Conv module ( Figure 2 The Conv layer with the label 5) extends the third branch, which outputs the feature map of 4N channels ( Figure 3 The 5-level output feature map in the 4N channel is upsampled by adjacent interpolation, and the upsampling result is used as the third input feature of the multi-level fusion feature pool; the multi-level fusion feature pool structure fuses the first input feature, the second input feature and the third input feature to obtain a feature splicing map, and forms a multi-level fusion feature pool.
[0036] Specifically, the scales of the first input feature, the second input feature, and the third input feature are all the same, and are all mesoscale features.
[0037] Specifically, if Figure 2 As shown, the batch channel mean structure includes a spatial mean module and a batch mean module (assuming that the tensor dimensions of the batch training data in calculating the spatial mean and batch mean are B, C, H, W, where B is the number of batches, C is the number of channels, H is the number of feature map rows, and W is the number of feature map columns, then the tensor dimensions after calculating the spatial mean are B, C, 1, 1, and then the tensor dimensions after calculating the batch mean are 1, C, 1, 1. Spatial mean: Calculate the mean of the feature map of each channel in each picture of the training batch, that is, calculate the mean of the width and height of the feature map of the channel to obtain the spatial features of the channel; batch mean: calculate the mean of the calculated result of the spatial mean, and then calculate the batch dimension to obtain the batch mean); first, the spatial mean of the output feature map of the multi-level fusion feature pool is calculated by the spatial mean module; then the spatial mean of the output feature map is sent to the batch mean module to calculate the batch mean. The spatial mean includes the width mean and height mean of the output feature map. The calculation formulas of the spatial mean module and the batch mean module are respectively:
[0038]
[0039]
[0040] in, represents the spatial mean, represents the batch mean, w represents the width of the output feature map of the multi-level fusion feature pool, h represents the height of the output feature map of the multi-level fusion feature pool, b represents the number of training batches, and p ij Represents the feature value of the i-th column and j-th row in the output feature map, q m Indicates the feature map value (spatial mean) of the mth image.
[0041] Specifically, if Figure 2 As shown in the figure, the mean compression expansion structure includes two parts: channel compression and channel expansion. Figure 4 As shown in the figure, it mainly uses two 1×1 convolutions to compress and expand the output of the batch channel mean structure in sequence, and uses the sigmoid function to process the output of the mean compression and expansion structure to obtain channel features; the specific calculation process is shown in the following formula:
[0042]
[0043] Among them, y represents the output tensor value after the sigmoid function, x represents the tensor input to the sigmoid function, and e is the infinite non-repeating decimal 2.71828...; the sigmoid function curve is as follows Figure 5 shown.
[0044] Specifically, the multi-level fusion feature pool fuses the first input feature, the second input feature, and the third input feature to obtain a feature splicing graph, which is input into the batch channel mean structure, and then the output of the batch channel mean structure is sent to the mean compression expansion structure; then Figure 2 As shown, the first input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is upsampled and input into the first CP2 module ( Figure 2 The second input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is input into the second CSP2 module ( Figure 2 The third input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is downsampled and input into the third CSP2 module ( Figure 2 CSP2 layer with label number 8).
[0045] In the present invention, unless otherwise clearly stipulated and limited, the terms "installation", "setting", "connection", "fixation", "rotation" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.
[0046] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.
Claims
1. A target detection method based on adaptive multi-level fusion feature pool, characterized in that: The following steps are involved: S1. Build an improved YOLOv5 network based on the existing YOLOv5 network. The improved YOLOv5 network includes a backbone network, a neck network, and a head network. The neck network incorporates a multi-level fusion feature pool, a batch channel mean structure, and a mean compression and expansion structure. The improved YOLOv5 network includes a second CSP1 module, a third CSP1 module and a second Conv module; Get the output features of the second CSP1 module and pass them through a convolution layer with a stride of 2 and a convolution kernel of 3×3 to obtain a downsampled result, which is used as the first input feature of the multi-level fusion feature pool; Obtain the output features of the third CSP1 module and use them as the second input features of the multi-level fusion feature pool; Obtain the output features of the second Conv module, upsample them through the nearest interpolation algorithm, and use the upsampled results as the third input features of the multi-level fusion feature pool; The multi-level fusion feature pooling structure fuses the first input feature, the second input feature, and the third input feature to obtain a feature splicing map, which is input into the batch channel mean structure, and the output of the batch channel mean structure is sent to the mean compression and expansion structure; The batch channel mean structure includes a spatial mean module and a batch mean module. First, the spatial mean of the output feature map of the multi-level fusion feature pool is calculated by the spatial mean module. Then, the spatial mean of the output feature map is sent to the batch mean module to calculate the batch mean. The mean compression and expansion structure uses two 1×1 convolutions to compress and expand the output of the batch channel mean structure in sequence, and uses the sigmoid function to process the output of the mean compression and expansion structure to obtain channel features; S2. Obtain an image dataset to train an improved YOLOv5 network, perform iterative training using a loss function, and adjust network parameters. S3. Use the trained improved YOLOv5 network for target detection to obtain target detection results.
2. The target detection method based on the adaptive multi-level fusion feature pool according to claim 1, characterized in that: The improved YOLOv5 network also includes a first CSP2 module, a second CSP2 module and a third CSP2 module; the first input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is upsampled and input into the first CSP2 module; the second input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is input into the second CSP2 module; the third input feature is multiplied by the output of the mean compression expansion structure, and the multiplication result is downsampled and input into the third CSP2 module.
3. The target detection method based on adaptive multi-level fusion feature pool according to claim 1, characterized in that: The scales of the first input feature, the second input feature, and the third input feature are all the same.
4. The target detection method based on adaptive multi-level fusion feature pool according to claim 1, characterized in that: The calculation formulas for the spatial mean module and the batch mean module are: in, represents the spatial mean, represents the batch mean, w represents the width of the output feature map of the multi-level fusion feature pool, h represents the height of the output feature map of the multi-level fusion feature pool, b represents the number of batch training, p ij Represents the eigenvalue of the i-th row and j-th column of the output feature map of the multi-level fusion feature pool, q m Represents the spatial mean of the m-th output feature map.