A target detection method based on YOLOv8 improved DCN-BiFPN

Abstract

This invention discloses an improved DCN-BiFPN object detection method based on YOLOv8, and uses an improved BiFPN feature pyramid feature fusion in the network constructed using this method. First, improved deformable convolutions replace some ordinary convolutions in the original network, enhancing the modified network's ability to extract features from irregular targets. Second, an improved cascaded feature fusion module is used to enhance the neural network's ability to represent features at different scales and semantic levels. This improved cascaded feature module allows the network to extract features from feature maps at different levels or scales. Next, a weighted bidirectional feature pyramid network connection module is used to concatenate feature maps from different layers along the channel dimension, enriching the network's ability to represent targets. Finally, a hybrid attention mechanism is added to the network to reduce interference from background and non-target objects, improving the network's accuracy in target detection.

Description

Technical Field

[0001] This invention relates to improvements in target detection algorithms, specifically a DCN-BiFPN target detection method based on YOLOv8. Background Technology

[0002] Object detection, a crucial task in computer vision, aims to identify all objects of interest in an image or video, determining their category and location. It is one of the core problems in computer vision. Due to the different appearances and poses of various objects, coupled with interference from factors such as lighting and occlusion during imaging, object detection presents significant challenges in computer vision. Since the advent of deep learning, most object detectors have shared a similar structure: a backbone network for extracting features from images or videos, and a head network for object classification. Therefore, improving the feature extraction capabilities of the backbone network and enhancing the performance of the head network have become two critical issues in current object detection work.

[0003] Improving the feature extraction capability of the backbone network can be approached from three main aspects. First, a suitable convolution operator should be chosen. The convolution operator determines the network's receptive field to the input data and the feature representation it learns; therefore, choosing an appropriate convolution operator can significantly affect the network's performance. Second, the network structure should be designed reasonably based on the characteristics of the task and actual needs, avoiding over-design or over-simplification. A reasonable network structure can effectively improve the feature extraction capability of the feature extraction network. Finally, an appropriate receptive field should be selected to ensure that the network has a sufficiently large receptive field to capture global information in the input image. Most feature extraction networks achieve this by increasing the size of convolutional or pooling layers.

[0004] Developing an excellent object detection head presents various challenges, which can be summarized into several categories. First, the size and aspect ratio of target objects in an image can vary significantly. An effective object detection head needs to be adaptable to targets of multiple scales and aspect ratios; in short, the head should be scale-aware. Second, the object detection head should also possess spatial awareness, as target objects may appear in images at different poses and angles. The head needs to be able to perceive these spatial variations to accurately identify and locate objects. Finally, the head of an object detection system needs to be task-aware. Objects can be represented in different ways, such as bounding boxes, center points, and corner points. Different representations may involve different optimization objectives and constraints. Therefore, the head of an object detection system needs the ability to understand and process these different representations to ensure that the system can flexibly adapt to and correctly locate targets under different task settings, thereby improving the system's versatility and adaptability. Summary of the Invention

[0005] To address the problems of existing technologies, such as high equipment requirements for the target detection algorithm, high false negative rate for overlapping objects, and duplicate detection of the same target, this invention proposes a DCN-BiFPN target detection method based on YOLOv8 to solve the aforementioned problems.

[0006] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0007] A target detection method based on YOLOv8-improved DCN-BiFPN includes the following steps:

[0008] S1. Collect target scene image data and perform data augmentation, label classification, and build a paired dataset with classification labels;

[0009] S2. Establish a DCN-BiFPN object detection network and train iteratively to adapt to different application scenarios, including: 1) Backbone network: Based on the CSPDarkNet-53 backbone network, using CSP operation units to extract image features; the original ordinary convolution is set to an improved deformable convolution DConv module, which maps the feature vector to a higher-dimensional space, so that the offset has more learning space during calculation; a hybrid attention mechanism module is adopted to adaptively refine the feature map; an LWConcat module is added to introduce learnable weights and learn the input tensor during training; 2) Neck network: Introduce the BiFPN network structure and set the detection head to five inputs and four outputs; 3) In the detection stage, SoftNMS is used to improve the candidate boxes for object detection;

[0010] S3. Collect image data, input it into the optimized and trained target detection network, automatically locate the target location prediction box and output the predicted classification label.

[0011] The label classification is a manual categorization of the targets.

[0012] The backbone network specifically includes the DConv module, DC2f module, CBAM module, SPPF module, and LWConcat module. The weighted bidirectional feature pyramid network module is used to splice and extract feature maps from different layers along the channel dimension.

[0013] The DC2f module consists of two ordinary convolutions connected in sequence and a DConv module, with the output of the second ordinary convolution being residually concatenated with the output of the DConv module.

[0014] The improvements to the DConv module include:

[0015] Stage 1: After extracting feature maps from the input image, a 1*1 convolution is introduced to perform non-linear combination of features, enabling information interaction between different channels; a variable offset and a 1*1 convolution are added to dynamically adjust the convolution kernel position, enhancing the ability to process non-uniform shape objects.

[0016] Stage 2: The output tensor undergoes another 5x5 convolution, enabling the convolutional neural network to capture a larger receptive field;

[0017] Stage 3: After extracting features from the feature map through two convolutions, it is segmented into horizontal offset, vertical offset, and magnitude parameter.

[0018] Stage 4: Introduce two learnable offset parameters for training via the network's backpropagation algorithm.

[0019] The DConv module is used to learn offsets at all positions and channels, and outputs a feature map:

[0020]

[0021] Where i and j represent the coordinate positions on the feature map, c represents the channel of the feature map, Δx and Δy represent the learned offsets, and W... a,b,c The weights of the convolution kernel are used to convolve the input feature map X, and k is used to traverse the channels of the input feature map.

[0022] The improvements to the Neck network include:

[0023] Stage 1: Add convolution operations between BiFPN and backbone network features to extract local features from the input image;

[0024] Stage 2: Modify the number of layers in BiFPN to five inputs and four outputs; the feature connections adopt a learnable weight adjustment mechanism.

[0025] The present invention has the following beneficial effects and advantages:

[0026] First, we propose a novel deformable convolution DConv, which allows for more flexible learning of task-specific offsets by introducing 1x1 convolutions before calculating offsets.

[0027] Secondly, we propose a novel LBiFPN structure. By increasing convolutional operations and modifying the number of layers and output heads of BiFPN, we can extract richer local features and better adapt to the backbone network.

[0028] Finally, we introduce a learnable weight adjustment mechanism to make the model more flexible, better preserve feature information at each level, and thus improve its adaptability to targets of different scales. Attached Figure Description

[0029] Figure 1 Here is the flowchart of the proposed DConv calculation;

[0030] Figure 2 This shows the structure diagram of BiFPN and the BiFPN structure diagram commonly used in YOLO.

[0031] Figure 3 The LBiFPN structure proposed in this invention;

[0032] Figure 4 The proposed DCN-BiFPN model structure is shown in this invention. Detailed Implementation

[0033] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0035] This invention addresses the problems of traditional object detection algorithms, such as high requirements for model equipment, low detection accuracy, high false negative rate for overlapping targets, and repeated detection of the same target. It proposes an improved method using multi-layer deformable convolution and a fusion attention mechanism, and employs an improved BiFPN feature pyramid feature fusion in the network constructed using this method. First, the improved deformable convolution replaces some ordinary convolutions in the original network, enhancing the modified network's ability to extract features from irregular targets. Second, an improved cascaded feature fusion module is used to enhance the neural network's ability to represent features at different scales and semantic levels. This improved cascaded feature module allows the network to extract features from feature maps at different levels or scales. Next, a weighted bidirectional feature pyramid network connection module is used to concatenate feature maps from different layers along the channel dimension, enriching the network's ability to represent targets. Finally, a hybrid attention mechanism is added to the network to reduce interference from background and non-target objects, improving the network's target detection accuracy.

[0036] Figure 1 The flowchart for calculating the variable convolution DConv of this invention is shown below, and its specific theory and function are as follows:

[0037] This paper follows the work of previous researchers and also uses bilinear interpolation to calculate the offset. A 1x1 convolution operation is performed before the calculation. The application of the offset is to better represent the spatial information of the input feature map, thereby improving the accuracy of the convolution. The 1x1 convolution can perform a non-linear mapping on the input feature map, improving the feature representation capability of the offset. Specifically, performing a 1x1 convolution on the input feature map before calculating the offset maps the feature vectors to a higher-dimensional space, which allows the offset to have more learning space during calculation.

[0038] The offsets of the original deformable convolution are obtained directly on the same input feature map through convolution operations, with the spatial resolution and dilation of the convolution kernel consistent with the current convolutional layer. We propose introducing a 1x1 convolution before calculating the offsets. This additional convolution takes the output of the current convolutional layer as input and outputs the offsets used to adjust the deformable convolution. The 1x1 convolution does not introduce a spatial receptive field; it only performs a linear combination along the channel dimension, allowing for more flexible learning of task-specific offsets. Specifically, after adding the 1x1 convolutional layer, the offset calculation occurs on the output of the current convolutional layer, rather than on the input feature map.

[0039] First, the image undergoes a series of convolutions to extract features, resulting in an input feature map. Then, it passes through a deformable convolution module to obtain an output feature map. Simultaneously, the input feature map undergoes a 1x1 convolution and a regular convolution to calculate the offset and amplitude parameters. Finally, the obtained offset and amplitude parameters are used for adjustment and calculation to obtain the final output feature map.

[0040] 1x1 convolutions are used for non-linear combination of features, enabling information interaction between different channels. The input feature map tensor x1 is [H, W, C], where H represents the tensor height, W is the tensor width, and C is the number of input channels. The 1x1 convolution kernel K has dimensions [1, 1, c]. in ,x out Then, after a 1*1 convolution, the output y1 of x1 can be obtained. The formula is as follows:

[0041]

[0042] The tensor size of the output feature map y1 is [H, W, c]. out ], σ represents the activation function, b c This means that a 1x1 convolution corresponds to the output channel c. out The bias term. Then the output tensor y1 is subjected to another 5*5 convolution so that the convolutional neural network can capture a larger receptive field and process a wider range of information. The output tensor after the 5*5 convolution is y2.

[0043]

[0044] After extracting features from the feature map through two convolutions, more complex spatial features are learned. These features are divided into three parts: horizontal offset, vertical offset, and magnitude parameter. The resulting output feature map y2 serves as the input X for calculating the offset, where X corresponds to the offset ΔX. Simultaneously, two learnable offset parameters Δx and Δy are introduced and trained using the network's backpropagation algorithm. The first formula is used to calculate the offset.

[0045]

[0046] (i, j) represents a point at a certain location in the input feature map X, and u and v represent new sampling positions in the horizontal and vertical directions, respectively. Therefore, u and v can be expressed as u = i + Δx; v = j + Δy. The values ​​of a and b in the above equations are 0 and 1, respectively, and a and b are used to traverse the four points of the bilinear interpolation. Afterwards, the complete output feature map F2(X, ΔX) can be obtained. i，j，c The calculation formula.

[0047]

[0048] Where i and j represent the coordinate positions on the feature map, c represents the channel of the feature map, Δx and Δy represent the learned offsets, and W... a,b,c The weights of the convolution kernel are used to convolve the input feature map. This formula represents the entire calculation process of the output feature map, covering all positions, channels, and learned offsets. k is used to traverse the channels of the input feature map.

[0049] like Figure 2 As shown, we compare BiFPN with the general application of BiFPN in YOLO. The original BiFPN structure is shown in Figure (a). BiFPN removes nodes with only one edge at each end and introduces bidirectional connections in different layers. These bidirectional connections allow feature information to flow between different layers, which helps improve communication between different scales and layers, enhances the network's ability to process targets of different scales, and further improves the model's generalization ability. The structure in Figure (b) is the general usage of BiFPN in the YOLO series, that is, adding a skip connection in the connection of layer P4.

[0050] Figure 3 The LBiFPN structure diagram of this invention is shown in the figure:

[0051] This invention adds a convolution operation between BiFPN and the backbone network features. This added convolution operation extracts local features from the input image, thereby improving the model's ability to recognize targets, extracting richer local features, and further enhancing the feature fusion effect. Furthermore, the number of layers in BiFPN is modified from five inputs and five outputs to five inputs and four outputs, making it more compatible with our backbone network. More layers in BiFPN result in higher computational cost and reduced model performance; reducing the number of layers reduces computational cost and improves performance. In addition, the feature connections employ a weight adjustment mechanism, making the model more flexible, better preserving feature information at each level, and thus improving adaptability to targets of different scales.

[0052] Figure 4The diagram shows the DCN-BiFPN model structure of this invention, consisting of three parts from left to right: Backbone, Neck, and Detect. It illustrates the DCN-BiFPN model structure reconstructed based on YOLOv8. First, the backbone network for feature extraction uses an improved deformable convolutional layer (DConv) instead of ordinary convolutions. Furthermore, a hybrid channel attention mechanism (CBAM) is added. The difference between D_C2f and the C2f module used in YOLOv8 is that the ordinary convolutions are replaced with the improved deformable convolutional layer (DConv) described in this paper. At the end of the network, the original SPPF module is used to enhance feature extraction capabilities. Notably, the D_C2f module is repeated 3 times, 6 times, 6 times, and 3 times, and the number of channels is halved, further reducing the model's parameter count and computational cost. Before feature fusion in the Neck part, convolutional layers are used to adjust the number of channels, giving the model better advantages for deployment on smaller devices. Compared to the FPN used in YOLOv8, this paper adds two custom connection layers. Unlike the original Concat module, the improved LWConcat (Learnable weights) module introduces learnable weights, allowing the module to learn how to input tensors through training. This design allows the model to flexibly acquire higher-level feature information from the backbone network. Finally, SoftNMS is used in the detection stage to improve the candidate box retention rules. This technique effectively avoids suppressing intersecting but non-overlapping boxes, improving detection accuracy and stability. These improvements and innovations enable the DCN-BiFPN model to perform excellently in object detection tasks.

[0053] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. A target detection method based on YOLOv8-improved DCN-BiFPN, characterized in that, Includes the following steps: S1. Collect target scene image data and perform data augmentation, label classification, and build a paired dataset with classification labels; S2. Establish the DCN-BiFPN target detection network and train iteratively to adapt to different application scenarios, including: 1) Backbone network: based on the CSPDarkNet-53 backbone network, using CSP operation units to extract image features; The original ordinary convolution is replaced with an improved deformable convolution DConv module, which maps the feature vector to a higher-dimensional space, allowing the offset to have more learning space during computation; a hybrid attention mechanism module is used to adaptively refine the feature map; an LWConcat module is added to introduce learnable weights to learn the input tensor during training; 2) Neck network: a BiFPN network structure is introduced, and the detection head is set to five inputs and four outputs; 3) SoftNMS is used to improve the candidate boxes for object detection in the detection stage; S3. Collect image data, input it into the optimized and trained target detection network, automatically locate the target location prediction box and output the predicted classification label.

2. The target detection method based on YOLOv8-improved DCN-BiFPN according to claim 1, characterized in that, The label classification is a manual categorization of the targets.

3. The DCN-BiFPN target detection method based on YOLOv8 improved according to claim 1, characterized in that, The backbone network specifically includes the DConv module, DC2f module, CBAM module, SPPF module, and LWConcat module. The weighted bidirectional feature pyramid network module is used to splice and extract feature maps from different layers along the channel dimension.

4. The target detection method based on YOLOv8-improved DCN-BiFPN according to claim 3, characterized in that, The DC2f module consists of two ordinary convolutions connected in sequence and a DConv module, with the output of the second ordinary convolution being residually concatenated with the output of the DConv module.

5. The DCN-BiFPN target detection method based on YOLOv8 improved according to claim 3, characterized in that, The improvements to the DConv module include: Stage 1: After extracting feature maps from the input image, a 1*1 convolution is introduced to perform non-linear combination of features, enabling information interaction between different channels; a variable offset and a 1*1 convolution are added to dynamically adjust the convolution kernel position, enhancing the ability to process non-uniform shape objects. Stage 2: The output tensor undergoes another 5x5 convolution, enabling the convolutional neural network to capture a larger receptive field; Stage 3: After extracting features from the feature map through two convolutions, it is segmented into horizontal offset, vertical offset, and magnitude parameter. Stage 4: Introduce two learnable offset parameters for training via the network's backpropagation algorithm.

6. The DCN-BiFPN target detection method based on YOLOv8 improved according to claim 5, characterized in that, The DConv module is used to learn offsets at all positions and channels, and outputs a feature map: Where i and j represent the coordinate positions on the feature map, c represents the channel of the feature map, Δx and Δy represent the learned offsets, and W... a,b,c The weights of the convolution kernel are used to convolve the input feature map X, and k is used to traverse the channels of the input feature map.

7. The DCN-BiFPN target detection method based on YOLOv8 improvement according to claim 1, characterized in that, The improvements to the Neck network include: Stage 1: Add convolution operations between BiFPN and backbone network features to extract local features from the input image; Stage 2: Modify the number of layers in BiFPN to five inputs and four outputs; the feature connections adopt a learnable weight adjustment mechanism.

Images