Multi-scale mark detection method based on YOLOv8

Abstract

The invention discloses a target detection method and system for enhancing multi-scale target detection based on YOLOv8, and the method comprises the steps: designing a cross-scale feature recombination module guided by channel attention, and constructing a feature enhancement scheme adaptive to a target scale through the collaborative design of dynamic space transformation and channel re-calibration, richer feature interaction and fusion are realized; the neck network adopts switchable cavity convolution, receptive fields of targets with different scales are dynamically matched under the condition that the number of parameters or the calculated amount is not increased, and the flexibility of the network to the size and the scale of features is improved; space depth conversion convolution is introduced, so that the representation capability of the model is improved; deformable convolution is integrated in a C2f module of a backbone network, a sampling position is dynamically adjusted through grouping offset learning, and geometric adaptability to irregular targets is improved. A large number of experiments on PASCAL VOC and MS COCO data sets prove the effectiveness of the model provided by the invention.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a multi-scale target detection method with an improved YOLOv8 architecture. Background Art

[0002] As one of the core tasks in the field of computer vision, object detection aims to identify the object categories in images and achieve precise positioning. With the rapid development of deep learning technology, object detection algorithms have been widely used in medical diagnosis, autonomous driving, intelligent monitoring and other fields. In particular, object detection models based on convolutional neural networks have continuously promoted technological progress, from two-stage networks (such as FastR-CNN, Faster R-CNN, Mask R-CNN) to one-stage models (such as the YOLO series, SSD, RetinaNet, etc.). Among them, the YOLO (You Only Look Once) series of algorithms has significantly improved object detection performance by optimizing network structure and introducing innovative technologies. As a classic version of the series, YOLOv8 has attracted much attention for its balance between speed and accuracy, becoming an important representative in the field of real-time object detection.

[0003] However, processing multi-scale features is a key challenge in object detection. Objects in images vary in size and shape, and traditional convolutional neural networks often struggle to effectively capture these variations. To address this issue, many researchers have proposed multi-scale feature fusion methods, combining feature maps from different layers to enhance the model's detection capabilities for both small and large objects. The YOLO family of models typically utilizes feature pyramid networks and their variants for multi-level feature fusion. However, current methods suffer from certain drawbacks: First, their feature extraction capabilities are limited, failing to fully leverage the rich information from features at different layers. Although FPN networks are designed to capture multi-level features, in practice, their ability to extract features from objects of varying scales remains insufficient, resulting in poor detection performance for both small and large objects in complex scenes. Second, their multi-scale feature fusion capabilities are limited. Existing feature fusion strategies struggle to fully integrate feature information from different layers. In the feature pyramid, upsampling and downsampling processes result in information loss, preventing lossless information transfer and hindering the YOLO model's ability to more efficiently fuse information. This limitation prevents the model from fully leveraging the complementary strengths of features at each layer when processing multi-scale objects, further leading to performance bottlenecks. Therefore, proposing a method that can achieve dynamic fusion of features at different levels while balancing the accuracy and efficiency of multi-scale target detection has become an urgent need in the current target detection field. Summary of the Invention

[0004] This paper aims to propose a method for multi-scale object detection that effectively improves the feature pyramid structure of current object detection algorithms. Without increasing the number of parameters or computational complexity, the method's feature extraction and characterization capabilities are optimized, thereby improving the algorithm's detection performance.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a multi-scale target detection method based on YOLOv8, comprising the following steps:

[0006] S1. Preprocess the input image;

[0007] S2. Input the preprocessed image into the backbone network, integrate DCNv3 variable convolution in C2f, dynamically adjust the sampling position, extract multi-scale features through multiple C2f_DCNv3 modules, and gradually downsample to generate three key feature layers;

[0008] S3. Select target level P i As the fusion backbone, the scale alignment unit SA is used to align the resolution and number of channels;

[0009] S4. Based on the fusion backbone P i Generate channel attention weight M c (P i ), which is applied to the remaining feature layers to guide feature fusion;

[0010] S5. The feature layer after scale alignment and weighted attention is evenly divided along the channel dimension, and the backbone and the remaining feature layers are fused and spliced ​​into feature sub-blocks in a ratio of 2:1:1 according to the number of channels to obtain the fused feature F' i ;

[0011] S6. Generate the final feature enhancement representation P' i ;

[0012] S7. In the neck network, C2f_SAConv and SPDConv are used to enhance the model's adaptability to targets of different scales and guide multi-scale feature fusion.

[0013] S8. The feature is input to the detection head for detection;

[0014] S9. This paper selects two public datasets to test and evaluate the detection performance of the model. This paper uses precision, recall, map@0.5, map@0.5:0.95, and parameter count as quantitative indicators of object detection performance. The performance of the model on the two datasets is reported to demonstrate the detection results.

[0015] In summary, compared with the existing technology, the present invention has the following advantages:

[0016] (1) This paper designs a cross-scale feature reconstruction module (CAFR) based on channel attention guidance. Through the collaborative design of dynamic spatial transformation and cross-level feature interaction, a feature enhancement scheme adapted to the target scale is constructed, which achieves richer feature interaction and fusion, and significantly improves the multi-scale target detection capability.

[0017] (2) This paper redesigns and improves the C2f module in the backbone network using the DCNv3 operator, resulting in C2f-DCNv3, and replaces the C2f layer with the new module. This improvement enables the module to dynamically adjust the shape and position of the convolution kernel, significantly enhancing the network's ability to adapt to local deformations of multi-scale objects and improving overall detection performance.

[0018] (3) This paper introduces switchable atrous convolution (SAC) and combines it with the C2f module of the neck network to form the C2f SAConv module. This improvement optimizes the multi-scale feature fusion process through dynamic atrous rate switching and global context awareness, enhancing the model's adaptability and accuracy for objects of different scales.

[0019] (4) To further improve the accuracy of model detection, the present invention uses a new convolution method SPDConv to replace the traditional CNN architecture. This modification not only enhances the representation ability of the model, but also reduces the model complexity while improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a structural diagram of the target detection model of the present invention;

[0021] Figure 2 This is the schematic diagram of the DCNv3 module;

[0022] Figure 3 This is the workflow diagram of the SA module;

[0023] Figure 4 This is the workflow diagram of the CAFR module;

[0024] Figure 5 This is the schematic diagram of the SAConv module;

[0025] Figure 6 This is the schematic diagram of the SPDConv spatial reorganization principle;

[0026] Figure 7 This is the result of ablation experiment;

[0027] Figure 8 This is a comparison chart of the overall performance of the present invention and other models on the PASCAL VOC dataset; DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings and examples, which are for illustrative purposes only and are not to be construed as limiting the present invention.

[0029] The present invention discloses a multi-scale target detection method based on YOLOv8, and its model structure diagram is as follows: Figure 1 As shown. It includes the following steps:

[0030] S1. Preprocess the input image.

[0031] In specific implementation, the present invention scales the input image to a uniform size (e.g., 640×640), uses mosaic data enhancement, and performs input preprocessing;

[0032] S2. Input the preprocessed image into the backbone network, integrate DCNv3 variable convolution in C2f, dynamically adjust the sampling position, extract multi-scale features through multiple C2f_DCNv3 modules, and gradually downsample to generate three key feature layers;

[0033] When implementing it specifically, Figure 2 As shown, the present invention divides the input feature map into G groups, and each group independently predicts the offset and modulation scalar of the sampling point. The prediction results are then calculated, and the output feature map is finally obtained by calculation. The formula for feature calculation can be expressed as:

[0034]

[0035] Where G represents the total number of aggregation groups. For the g-th group, w g ∈R C×C′ represents the position-independent projection weight of the group, where C′=C / G represents the group dimension. mg,k∈ R represents the modulation scalar of the k-th sample point in the g-th group, normalized along dimension K by the softmax function. g ∈R C′×H×W Represents the slice input feature map. Δp g,k is the grid sampling position p in the gth group k The corresponding offset.

[0036] S3. Select target level P i As the fusion backbone, the scale alignment unit SA is used to align the resolution and number of channels;

[0037] When implementing it specifically, Figure 3 As shown, for the three feature layers of different scales obtained during the upsampling process, a target level P is first selected. i (3≤i≤5) as the fusion backbone, and the rest of the feature layers P are aligned by the scale alignment unit SA. t (t≠i, 3≤t≤5) unified to the target level P iResolution and number of channels. As Figure 4 shown, taking P4 as the fusion backbone as an example, use depthwise separable transposed convolution to replace the nearest neighbor interpolation method, retain the edge details through learnable parameters, upsample P5 by a factor of 2, and reduce the number of channels to the same as P4 through 1×1 convolution; use depthwise separable convolution to replace ordinary convolution to downsample P3 to align the resolution and avoid texture loss caused by pooling operations. Similarly, when P3 and P5 are used as the fusion backbone, the remaining feature layers are unified in resolution and number of channels through similar depthwise separable convolution and depthwise separable transposed convolution. For high-level feature P t (t>i), use depthwise separable transposed convolution for upsampling + 1×1 convolution for channel compression; for low-level feature P t (t<i): Use depthwise separable convolution for downsampling + zero padding. The specific formula is as follows:

[0038]

[0039] S4. Based on the fusion backbone P i Generate the channel attention weight M c (P i ), and apply it to the remaining feature layers to guide feature fusion;

[0040] Specifically, S4 includes the following steps:

[0041] S401. For the fusion backbone P i , perform global average pooling and global max pooling respectively, and pass the pooled results through a multi-layer perceptron with shared weights. Add the outputs of the two MLPs and pass through the Sigmoid function to obtain the channel attention weight. Its formula is expressed as follows:

[0042]

[0043] where, F represents the input feature map, σ represents the sigmoid function, W0∈R C / r×C , W1∈R C×C / r .

[0044] S402. Apply M c (P i ) to the remaining aligned feature layers to obtain P ti . The formula for this process is as follows:

[0045]

[0046] where, represents the scale alignment operation, P i represents the fusion backbone, and the output feature P ti of the SA operation has the same resolution and number of channels as Pi Consistent. M c (P i ) indicates that the i Generate channel weight vectors.

[0047] S5. The feature layer after scale alignment and weighted attention is evenly divided along the channel dimension, and the backbone and the remaining feature layers are fused and spliced ​​into feature sub-blocks in a ratio of 2:1:1 according to the number of channels to obtain the fused feature F' i ;

[0048] In specific implementation, S5 includes the following steps:

[0049] S501. Figure 4 As shown in the figure, the feature layer after scale alignment and weighted attention is evenly divided into 4 subsets along the channel dimension. The specific formula is as follows:

[0050]

[0051] S502. After a large number of experiments, it is proved that the reasonable ratio of the number of channels of the fusion backbone feature layer and the remaining feature layers is 2:1:1. Therefore, in the fusion backbone P i Select the two middle subsets from the feature layer, and select one edge subset from each of the other feature layers to perform concat operations to obtain F' i The splicing operation formula is as follows:

[0052]

[0053] Among them, P mi 、P ni They represent the remaining feature layers after unified scale and attention weighted fusion.

[0054] S6. Generate the final feature enhancement representation P' i ;

[0055] In specific implementation, the output F' after cross-level feature fusion i The input is sent to the convolution block for feature extraction and the add operation is performed with the corresponding original fusion backbone, which not only retains the underlying detail information but also strengthens the semantic association across levels to generate the final feature enhancement representation P' i ,Right now:

[0056]

[0057] S7. In the neck network, C2f_SAConv and SPDConv are used to enhance the model's adaptability to targets of different scales and guide multi-scale feature fusion.

[0058] In specific implementation, S7 includes the following steps:

[0059] S701. Figure 5 The input features are globally pooled to generate a compact feature map, and channel fusion is achieved through 1×1 convolution. They then enter the residual module for deep feature integration. In the switchable atrous convolution (SAC) module, an average pooling layer is used to generate a switching function, and two atrous convolutions with different expansion rates are combined to process the input features. The switching function dynamically adjusts the weights based on the spatial position of the feature map to achieve adaptive extraction of multi-scale features. Specifically, the gating network generates a weight vector S(x)∈[0,1] based on the local context of the input features. The calculation process is:

[0060] S(x)=σ(Conv 1×1 (AvgPool 5×5 (x)))

[0061] Where σ is the Sigmoid function. The final output feature is the weighted sum of the two sets of convolution results:

[0062] y=S(x)·Conv(x,w,1)+(1-S(x))·Conv(x,w+Δw,3)

[0063] Here, Conv(x,w,r) represents a convolution operation with weight w and hyperparameter r, and Δw is the trainable weight offset.

[0064] S702. The SPD layer generalizes the original image transformation technique to the feature map downsampling process within the CNN. The specific implementation is shown in the formula. Given an intermediate feature map X of arbitrary size S×S×C1, a sequence of sub-feature maps is generated as shown in the formula. Figure 6 When scale=2, four sizes are generated. Subgraph f 0，0 、f 1,0 、f 0，1 、f 1，1 , all achieve 2x downsampling. Subsequently, these sub-feature maps are concatenated along the channel dimension to obtain the feature map X′. Finally, SPD converts X(S, S, C1) into an intermediate feature map

[0065] f 0，0 =X[O:S:scale,O:S:scale],f 0，1 =X[1:S:scale,O:S:scale],...,

[0066] f scale-1,0 =X[scale-1:S:scale,O:S:scale];

[0067] f1，0 =X[0: S: scale, 1: S: scale], f 1，1 ,...,

[0068] f scale-1，1 =X[scale-1:S:scale,1:S:scale];

[0069] f 0，scale-1 =X[0: S: scale, scale-1: S: scale], f 1，scale-1 ,...,

[0070] f scale-1，scale-1 =X[scale-1: S: scale, scale-1: S: scale].

[0071] S8. The feature is input to the detection head for detection;

[0072] When implementing it specifically, Figure 1 As shown in the figure, the head is responsible for the final object detection and classification tasks and consists of a detection head and a classification head. The detection head consists of a series of convolutional and deconvolutional layers to generate detection results. These layers are responsible for predicting bounding box regression values ​​for each anchor box and the confidence level of the object's presence. The classification head uses global average pooling to classify each feature map, reducing the dimensionality of the feature map and outputting a probability distribution for each category.

[0073] S9. This paper selects two public datasets to test and evaluate the detection performance of the model. This paper uses precision, recall, map@0.5, map@0.5:0.95, and parameter count as quantitative indicators of object detection performance. The performance of the model on the two datasets is reported to demonstrate the detection results.

[0074] In specific implementation, S9 includes the following steps:

[0075] S901. First, in order to verify the effectiveness of the algorithm proposed in this invention, an ablation experiment is designed based on YOLOv8s. Each group of experiments uses the same hyperparameters and training strategy. Figure 7 As shown in the figure, the gradual introduction of each module increased mAP@0.5 to 85.8% and mAP@0.5:0.95 from 63.8% to 67.1%, significantly enhancing the model's overall detection performance. Furthermore, the computational complexity of the final model remained the same as the original model, demonstrating the balance between computational efficiency and detection accuracy achieved by this module design.

[0076] S902. The present invention demonstrates the model detection performance by comparing the target detection effect on the PASCAL VOC dataset with different models. Figure 8As shown, the proposed method is the best among all compared models. Compared to the original YOLOv8s, the proposed method improves mAP@0.5 and mAP@0.5:0.95 by 2.2% and 3.3%, respectively. While using fewer parameters and computationally less than the state-of-the-art FA-YOLO algorithm, the proposed method demonstrates higher detection accuracy. This demonstrates that the proposed method not only offers superior detection performance but also offers advantages in terms of model complexity.

[0077] This paper addresses the problems of low feature fusion efficiency and loss of detailed information in multi-scale object detection tasks in complex scenarios by proposing a series of innovative improvements based on the YOLOv8 model, significantly improving the model's detection performance. First, a channel-attention-guided cross-scale feature reconstruction module (CAFR) is designed. This module achieves adaptive fusion of multi-level features through the collaborative design of dynamic spatial transformation and channel recalibration. This module effectively addresses the problems of insufficient feature selection and limited cross-level interaction in traditional FPN / PAN architectures, significantly improving the model's detection capabilities for both small and large objects. Backbone Network Improvement - C2f DCNv3 Module: Deformable convolution (DCNv3) is introduced into the backbone network. By dynamically adjusting the sampling positions and weights of the convolution kernels, the model's geometric adaptability to irregular objects and multi-scale scenes is enhanced, further improving the robustness of feature extraction. The neck network, combined with the dynamic dilated rate switching mechanism of switchable atrous convolution (SAC), optimizes the multi-scale feature fusion process. This module significantly improves the model's adaptability and detection accuracy for objects of varying scales through the synergistic effect of global context perception and local feature enhancement. Finally, to alleviate the information loss problem associated with traditional downsampling, the SPDConv module uses a spatial-to-depth conversion strategy, which is particularly effective in improving small object detection performance.

[0078] Finally, the details of the above embodiments of the present invention are merely examples for explaining the present invention. For those skilled in the art, any modifications, improvements and replacements of the above embodiments should be included in the scope of protection of the claims of the present invention.

Claims

1. A multi-scale target detection method based on YOLOv8, characterized in that: The method comprises the following steps: S1. Preprocess the input image; S2. Input the preprocessed image into the backbone network, integrate DCNv3 variable convolution in C2f, dynamically adjust the sampling position, extract multi-scale features through multiple C2f_DCNv3 modules, and gradually downsample to generate three key feature layers; S3. Select target level P i As the fusion backbone, the scale alignment unit SA is used to align the resolution and number of channels; S4. Based on the fusion backbone P i Generate channel attention weight M c (P i ), which is applied to the remaining feature layers to guide feature fusion; S5. The feature layer after scale alignment and weighted attention is evenly divided along the channel dimension, and the backbone and the remaining feature layers are fused and spliced ​​into feature sub-blocks in a ratio of 2:1:1 according to the number of channels to obtain the fused feature F' i ; S6. Generate the final feature enhancement representation P' i ; S7. In the neck network, C2f_SAConv and SPDConv are used to enhance the model's adaptability to targets of different scales and guide multi-scale feature fusion. S8. The feature is input to the detection head for detection; S9. This paper selects two public datasets to test and evaluate the detection performance of the model. This paper uses precision, recall, map@0.5, map@0.5:0.95, and parameter count as quantitative indicators of object detection performance. The performance of the model on the two datasets is reported to demonstrate the detection results.

2. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that, The specific process of S2 is: The input feature map is divided into G groups, and each group independently predicts the offset and modulation scalar of the sampling point. The prediction results are then calculated, and the output feature map is finally obtained through calculation. The formula for feature calculation can be expressed as: Where G represents the total number of aggregation groups. For the g-th group, W g ∈R C×C‘ represents the position-independent projection weight of the group, where C′=C / G represents the group dimension. mg,k∈R represents the modulation scalar of the k-th sample point in the g-th group, normalized along dimension K by the softmax function. g ∈R C'×H×W Represents the slice input feature map. Δp g,k is the grid sampling position p in the gth group k The corresponding offset.

3. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that, The specific process of S3 is as follows: For the three feature layers with different scales obtained in the upsampling process, first select a target level P i (3 ≤ i ≤ 5) as the fusion backbone, and use the scale alignment unit SA to unify the remaining feature layers P t (t ≠ i, 3 ≤ t ≤ 5) to the resolution and number of channels of the target level P i . Taking P4 as the fusion backbone as an example, use depthwise separable transposed convolution to replace the nearest neighbor interpolation method, retain the edge details through learnable parameters, upsample P5 by 2 times, and reduce the number of channels to the same as P4 through 1×1 convolution; use depthwise separable convolution to replace ordinary convolution to downsample P3 to align the resolution and avoid texture loss caused by pooling operations. Similarly, when P3 and P5 are used as the fusion backbone, the remaining feature layers are unified in resolution and number of channels through similar depthwise separable convolution and depthwise separable transposed convolution. For high-level features P t (t > i), use depthwise separable transposed convolution for upsampling + 1×1 convolution for channel compression; for low-level features P t (t < i): use depthwise separable convolution for downsampling + zero padding, and the specific formula is as follows:

4. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that The specific process of S4 is: S401. For the fusion backbone P i , perform global average pooling and global maximum pooling respectively, and pass the pooled results through a shared weight multi-layer perceptron. Add the outputs of the two MLPs and pass them through the Sigmoid function to obtain the channel attention weight. Its formula is as follows: Among them, F represents the input feature map, σ represents the sigmoid function, W0∈R C / r×C ,W1∈R C×C / r . S402. M c (P i ) is applied to the remaining feature layers after alignment to obtain P ti The formula for this process is as follows: in, represents the scale alignment operation, P i Represents the fusion backbone, the output feature P of the SA operation ti Resolution and number of channels and P i Consistent. M c (P i ) indicates that the i Generate channel weight vectors.

5. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that, The specific process of S5 is as follows: S501. Divide the feature layer after scale alignment and weighted attention into 4 subsets along the channel dimension. The specific formula is as follows: S502. After a large number of experiments, it is proved that the reasonable ratio of the number of channels of the fusion backbone feature layer and the remaining feature layers is 2:1:

1. Therefore, in the fusion backbone P i Select the two middle subsets from the feature layer, and select one edge subset from each of the other feature layers to perform concat operations to obtain F' i The splicing operation formula is as follows: Among them, P mi 、P ni They represent the remaining feature layers after unified scale and attention weighted fusion.

6. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that, The specific process of S6 is as follows: The output F' after cross-level feature fusion i The input is sent to the convolution block for feature extraction and the add operation is performed with the corresponding original fusion backbone, which not only retains the underlying detail information but also strengthens the semantic association across levels to generate the final feature enhancement representation P' i ,Right now:

7. A multi-scale target detection method based on YOLOv8 according to claim 1, characterized in that, The specific process of S7 is as follows: S701. Input features are globally pooled to generate a compact feature map, and channel fusion is achieved through 1×1 convolution. They then enter the residual module for deep feature integration. In the switchable atrous convolution (SAC) module, an average pooling layer is used to generate a switching function, and two atrous convolutions with different expansion rates are combined to process the input features. The switching function dynamically adjusts the weights according to the spatial position of the feature map to achieve adaptive extraction of multi-scale features. Specifically, the gating network generates a weight vector S(x)∈[0,1] based on the local context of the input feature. The calculation process is: S(x)=σ(Conv 1×1 (AvgPool 5×5 (x))) Where σ is the Sigmoid function. The final output feature is the weighted sum of the two sets of convolution results: y=S(x)·Conv(x,w,1)+(1-S(x))·Conv(x,w+Δw,3) Here, Conv(x, w, r) represents a convolution operation with weight w and hyperparameter r, and Δw is the trainable weight offset. S702.SPD layer extends the original image transformation technology to the feature map downsampling process inside CNN. The specific implementation is shown in the formula. Given an intermediate feature map X of any size S×S×C1, a sub-feature map sequence is generated as shown in the formula. When scale=2, four sub-feature map sequences of size Subgraph f 0，0 、f 1,0 、f 0，1 、f 1，1 , all achieve 2x downsampling. Subsequently, these sub-feature maps are concatenated along the channel dimension to obtain the feature map X′. Finally, SPD converts X(S, S, C1) into an intermediate feature map f 0，0 =X[0:S:scale,0: S:scale],f 0，1 =X[1:S:scale,0: S:sCale], ..., f scale-1,0 =X[scale-1:S:scale,0: S: scale]; f 1，0 =X[0:S:scale,1:S:scale],f 1，1 ,..., f scale-1，1 =X[scale-1:S:scale,1: S: scale]; f 0，scale-1 =X[0:S:scale,scale-1:S:scale],f 1，scale-1 ,..., f scale-1，scale-1 =X[scale-1:S:scale,scale-1:S:scale]。

Images