A small target detection method based on improved YOLOv8n

Abstract

The present invention proposes a small target detection method based on improved YOLOv8n. The specific steps of the method are as follows: (1) constructing a small target dataset; (2) introducing a transition downsampling module to replace the shallow downsampling operation of the YOLOv8n model, thereby reducing the loss of small target information caused by continuous downsampling of the input image; (3) inserting an upsampling module and a downsampling module into the YOLOv8n model respectively, thereby reducing the loss of small target information caused by downsampling in the shallow network; (4) using an improved C3 module that integrates an improved SwinTransformer‑V2 module to replace part of the C2f module in the YOLOv8n model, thereby strengthening the attention of small target information in the deep part of the backbone feature extraction network; (5) using the dataset constructed in (1) to train, verify and evaluate the improved YOLOv8n model. The present invention increases the average precision of small target detection while reducing a small amount of model parameters and meeting the real-time detection requirements, and reduces the occurrence of false detection and missed detection.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning target detection and relates to a small target detection method based on improved YOLOv8n. Background Art

[0002] With the integration of deep learning theory and practice, object detection technology has achieved rapid development. The goal of object detection is to determine the location and type of objects in an image. It has been applied to various fields, including target tracking. However, small object detection faces significant difficulties and challenges due to its small number of pixels and weak feature representation. This leads to significant information loss during feature extraction.

[0003] In the rapidly developing field of deep learning target detection, from the two-stage RCNN series of algorithms to the single-stage YOLO series of algorithms and the SSD series of algorithms, related improved models for small target detection have been produced one after another. The literature "Guo Fan, Zhang Yongxiang, Tang Jin, et al. YOLOv3-A: Traffic Sign Detection Network Based on Attention Mechanism [J]. Journal of Communications, 2021, Vol. 42(1): 87-99." introduces a channel attention mechanism and a spatial attention mechanism guided by semantic segmentation in the YOLOv3 detection layer. On the TT100K traffic sign dataset, the mAP is improved by 3.5%. However, this algorithm introduces a large number of parameters and has a speed of only 1.3 frames / s. The paper "Zhang Chaoyang1,2, Zhang Shang2,3, Wang Hengtao2,3 et al. Multi-head attention detection of small remote sensing targets at multiple scales [J]. Computer Engineering and Applications, 2023, Vol. 59(8): 227-238." adds a small target detection layer to the YOLOv5 model, adds a bidirectional feature pyramid, adds a SwinTransformer multi-head attention mechanism module, and uses Varifocalloss. On the NWPUVHR-10 remote sensing target dataset, the mAP is improved by 3.05%. However, the algorithm does not consider that the shallow features have been lost during downsampling. The paper "Guo Lei1, Wang Qiulong2, Xue Wei2, et al. Small target detection algorithm based on improved YOLOv5 [J]. Journal of University of Electronic Science and Technology of China, 2022, Vol. 51(2): 251-258." Based on YOLOv5, the paper uses Mosaic-8 data enhancement, adds a shallow feature map, adjusts the loss function and the target box regression formula, and improves the mAP by 4.55% in the detection of protective mask wearing in dense scenarios. However, the algorithm ignores the effective extraction of deep semantic information in the backbone feature extraction network. Summary of the Invention

[0004] 1. Purpose of the invention:

[0005] The purpose of this invention is to propose a small target detection method based on improved YOLOv8n, which is used to enhance the feature expression ability of small targets in the backbone feature extraction network, alleviate the serious problem of small target information loss, strengthen the network's attention to small targets, and improve the average precision of small targets.

[0006] 2. Technical solution:

[0007] A small target detection method based on improved YOLOv8n is characterized by comprising the following steps:

[0008] Step 1: Filter out valid images containing small objects from the images of the packaging workshop captured by the camera, build a small object dataset, and divide the valid images into training and validation sets;

[0009] Step 2: Construct an improved YOLOv8n network architecture including a transition downsampling module and an improved SwinTransformer-V2 module. The improved YOLOv8n network includes the following structure:

[0010] Step 2.1: Replace all the first two layers of the Conv downsampling modules of the YOLOv8n backbone feature extraction network with transition downsampling modules. The structure of the transition downsampling module is constructed by adding a transition convolution Conv module with k = 3 × 3, s = 1, and p = 1 in front of the ordinary convolution downsampling Conv module, so that the number of input channels changes to the number of output channels in advance. On the basis of integrating more small object information, the ordinary convolution downsampling operation is performed on the feature map to alleviate the loss of small object information.

[0011] Step 2.2: Insert an upsampling module before the downsampling operation in the second layer of the YOLOv8n backbone feature extraction network, and insert a downsampling module after the C2f module in the fifth layer of the backbone feature extraction network; the upsampling module and the downsampling module use the nearest neighbor interpolation algorithm and the ordinary convolution downsampling Conv module respectively;

[0012] Step 2.3: Build the improved SwinTransformer-V2 module. The construction steps are:

[0013] Step 2.3.1: Add a 1×1 channel offset convolution to the front end of the SwinTransformer-V2 module, and perform normalization and SiLU activation to enable early interaction between windows. This also introduces the idea of implicit position encoding to make the module structure more robust.

[0014] Step 2.3.2: Add a window channel attention mechanism to the output of the SwinTransformer-V2 module. By multiplying the weights, the importance of all channels in the window is reflected, which indirectly increases the interaction between windows. The weight generation of the window channel attention mechanism comes from the output feature map of the self-attention mechanism Attention module in the SwinTransformer-V2 module. The weight generation module of the window channel attention can be expressed as follows:

[0015] z′ 2ij =Linear(z 2ij ) (1)

[0016] z″ 2i =SiLU(LN(z′ 2i )) (2)

[0017] z″′ 2i =Sigmoid(LN(Linear(z″ 2i ))) (3)

[0018]

[0019] Among them, z 2ij represents the input of the window channel attention weight generation module after window division, i represents the i-th window, j represents the j-th channel, Linear represents the full connection operation, LN represents the normalization operation, and z in formula (1) 2ij After the linear operation, the single-channel input of the window size is transformed into an output of 1×1 size. The z′ in formula (2) 2i After LN normalization and SiLU activation, we get z″ 2i ; The linear operation in formula (3) does not change z″ 2i The shape of z″′ is obtained after LN normalization and Sigmoid activation. 2i ; z″′ in formula (4) 2i After the upsample neighbor interpolation upsampling operation of the window size, the output of the window channel attention weight generation module after the window division is obtained

[0020] Step 2.3.3: Add a convolution module operation with a convolution kernel of 1×1 at the output of the SwinTransformer-V2 module to perform concat fusion with the subsequent convolution output;

[0021] Step 2.4: Replace the last two C2f modules in the deep layer of the YOLOv8n backbone feature extraction network with the improved C3 module. The improved C3 module is obtained by replacing all the Bottleneck modules in the original C3 module with the improved SwinTransformer-V2 module.

[0022] Step 3: Use the training set and validation set in step 1 to train, validate, and evaluate the improved YOLOv8n network model.

[0023] 3. Innovation:

[0024] Compared with the original YOLOv8n model, the present invention introduces a transition downsampling module in the shallow part of the network, inserts an upsampling module and a downsampling module in the shallower part of the network, and replaces part of the C2f module with an improved C3 module that integrates the improved SwinTransformer-V2 module.

[0025] In general, this patent introduces the following methods and ideas:

[0026] (1) To address the problem of small target information loss due to continuous downsampling of the input image, a transition downsampling module is introduced to replace the continuous downsampling operations in the first two layers of the model, so that the continuous downsampling is interrupted. The model can integrate more small target features before downsampling, thereby alleviating the loss of small target information;

[0027] (2) To address the problem of small target information loss caused by downsampling in the shallow layers of the model, an upsampling module and a downsampling module are inserted before and after the shallow layers of the model, respectively, so that the original small target feature information can be amplified and extracted, thereby alleviating the loss of small target information. After all the feature extraction operations are completed, the shape of the feature map is restored to its original size, so that subsequent feature extraction tasks are not affected.

[0028] (3) To address the problem that the model does not pay enough attention to small targets, resulting in the loss of small target information, the C2f module of the model is replaced with an improved C3 module that integrates the improved SwinTransformer-V2 module, so that the model pays more attention to small targets.

[0029] 4.Beneficial effects:

[0030] The present invention discloses a small target detection method based on improved YOLOv8n. The present invention effectively alleviates the information loss of small targets in shallow networks through the transition downsampling module and the insertion of upsampling modules and downsampling modules. The present invention also increases the attention paid to small targets through the improved C3 module that integrates the improved SwinTransformer-V2 module. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flowchart of a small target detection method based on the improved YOLOv8n. First, a small target dataset is constructed. Then, a transition downsampling module is introduced into the YOLOv8n model. Then, upsampling and downsampling modules are inserted into the YOLOv8n model. Finally, an improved C3 module that integrates the improved SwinTransformer-V2 module is introduced into the YOLOv8n model. Finally, the improved YOLOv8n model is trained, verified, and evaluated using the small target dataset.

[0032] Figure 2 This is the original YOLOv8n network structure diagram, which mainly includes the Backbone part of the backbone feature extraction network, the Neck part for feature fusion, and the Head part for task prediction;

[0033] Figure 3 To improve the YOLOv8n network structure, three improvements were made to the original YOLOv8n model. The first two layers of the model's ordinary convolutional downsampling Conv modules were replaced with transitional downsampling modules. An upsampling module and a downsampling module were inserted before the second layer downsampling module and after the fifth layer C2f module, respectively. The last two C2f modules of the backbone feature extraction network were replaced with an improved C3 module that incorporates an improved SwinTransformer-V2 module.

[0034] Figure 4 This is the structure diagram of the transition downsampling module. It mainly adds a transition convolution Conv module with k=3×3, s=1, and p=1 in front of the ordinary convolution downsampling Conv module, so that the number of channels changes to the number of output channels in advance;

[0035] Figure 5 The improved C3 module structure diagram is integrated with the improved SwinTransformer-V2 module. It mainly replaces all the Bottleneck modules in the original C3 module with the improved SwinTransformer-V2 module.

[0036] Figure 6 This is a structural comparison diagram of the original SwinTransformer-V2 module and the improved SwinTransformer-V2 module. Based on the original SwinTransformer-V2 module, three improvements are made: channel offset convolution, window channel attention, and post-transition fusion convolution.

[0037] Figure 7This figure compares the training loss curves of the original YOLOv8n model and the improved YOLOv8n model on the small target dataset of the Volume Packaging Workshop. Compared with the original YOLOv8n model, the improved YOLOv8n model has better loss convergence effect.

[0038] Figure 8 This is a comparison chart of the detection effects of the original YOLOv8n model and the improved YOLOv8n model. From the comparison of the detection effects of the first picture, it can be seen that the original YOLOv8n model has some false detections for mobile phones, and the improved YOLOv8n model has eliminated these false detections; from the comparison of the detection effects of the second picture, it can be seen that the original YOLOv8n model has some missed detections for open necklines, and the improved YOLOv8n model has eliminated these missed detections; from the comparison of the detection effects of the third picture, it can be seen that the original YOLOv8n model has multiple frames for one object when detecting dense material boxes, and the improved YOLOv8n model has eliminated the occurrence of multiple frames for one object. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions, innovations and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0040] See also Figure 1 , the present invention provides a small target detection method based on improved YOLOv8n, the specific implementation is as follows:

[0041] Step 1: Construct a small target dataset for the packaging workshop. The specific operations are as follows:

[0042] We defined ten small object categories in the wrapping workshop: bench (BD), material box (WLH), unknown collar (LKWZ), closed collar (LKB), open collar (LKK), unknown mask (KZWZ), mask worn (KZD), mask not worn (KZWD), long hair (CTF), and mobile phone (SJ). We screened 500 valid images containing these small object categories from the wrapping workshop images captured by the camera. We divided these valid images into training and validation sets with a 1:1 ratio and used the labelImg annotation tool to label the training and validation sets.

[0043] Step 2: Replace the first two layers of the Conv downsampling modules of the YOLOv8n backbone feature extraction network with transition downsampling modules. Figure 4 The construction method is as follows: add a transition convolution Conv module with k=3×3, s=1, and p=1 in front of the ordinary convolution downsampling Conv module, so that the number of input channels changes to the number of output channels in advance;

[0044] Step 3: Insert an upsampling module before the downsampling operation of the second layer of the YOLOv8n backbone feature extraction network, and insert a downsampling module after the C2f module of the fifth layer of the backbone feature extraction network. The upsampling module uses the nearest neighbor interpolation algorithm, and the downsampling module uses the ordinary convolution downsampling Conv module.

[0045] Step 4: Replace the last two C2f modules in the deep layer of the YOLOv8n backbone feature extraction network with the improved C3 module. The improved C3 module is obtained by replacing all the Bottleneck modules in the original C3 module with the improved SwinTransformer-V2 module. Figure 6 , the construction steps of the improved SwinTransformer-V2 module are:

[0046] Step 4.1: Add a 1×1 channel offset convolution to the front end of the SwinTransformer-V2 module, and perform normalization and SiLU activation. The channel offset convolution is performed by dividing the input feature map into four equal channels and offsetting them in four different directions of width and height. The offset is half of the width and height of the window in the corresponding SwinTransformer-V2 module. Finally, the feature map is subjected to the normal convolution module operation.

[0047] Step 4.2: Add a window channel attention mechanism to the output of the SwinTransformer-V2 module. By multiplying the weights, the importance of all channels in the window is reflected, which indirectly increases the interaction between windows. The weight generation of the window channel attention mechanism comes from the output feature map of the self-attention mechanism Attention module in the SwinTransformer-V2 module. The weight generation module of the window channel attention can be expressed as follows:

[0048] z′ 2ij =Linear(z 2ij ) (1)

[0049] z″ 2i =SiLU(LN(z′ 2i )) (2)

[0050] z″′ 2i =Sigmoid(LN(Linear(z″ 2i ))) (3)

[0051]

[0052] Among them, z 2ijrepresents the input of the window channel attention weight generation module after window division, i represents the i-th window, j represents the j-th channel, Linear represents the full connection operation, LN represents the normalization operation, and z in formula (1) 2ij After the linear operation, the single-channel input of the window size is transformed into an output of 1×1 size. The z′ in formula (2) 2i After LN normalization and SiLU activation, we get z″ 2i ; The linear operation in formula (3) does not change z″ 2i The shape of z″′ is obtained after LN normalization and Sigmoid activation. 2i ; z″′ in formula (4) 2i After the upsample neighbor interpolation upsampling operation of the window size, the output of the window channel attention weight generation module after the window division is obtained

[0053] Step 4.3: Add a convolution Conv module operation with a convolution kernel of 1×1 at the output of the SwinTransformer-V2 module, which does not change the number of output channels of the feature map;

[0054] Step 5: Use the training set and validation set in step 1 to train, validate, and evaluate the improved YOLOv8n model:

[0055] On the Linux platform Red Hat 4.8.5-28, using Python 3.6.6, the deep learning framework torch1.7.1+cu101 was used to build the network model. The training epochs were set to 600 and the batch size was set to 8. Other YOLOv8n parameters were set using the default settings. The original YOLOv8n model and the improved YOLOv8n model were trained and verified using the small target dataset of the roll-up workshop. Compared with the original YOLOv8n model, the improved YOLOv8n model achieved a mAP0.5:0.95 improvement of 4.6 percentage points and could meet the requirements of real-time detection.

Claims

1. A small target detection method based on improved YOLOv8n, characterized by The following steps are involved: Step 1: Filter out valid images containing small objects from the images of the packaging workshop captured by the camera, build a small object dataset, and divide the valid images into training and validation sets; Step 2: Construct an improved YOLOv8n network architecture including a transition downsampling module and an improved SwinTransformer-V2 module. The improved YOLOv8n network includes the following structure: Step 2.1: Replace all the first two layers of the Conv downsampling modules of the YOLOv8n backbone feature extraction network with transition downsampling modules. The structure of the transition downsampling module is constructed by adding a transition convolution Conv module with k = 3 × 3, s = 1, and p = 1 in front of the ordinary convolution downsampling Conv module, so that the number of input channels changes to the number of output channels in advance. On the basis of integrating more small object information, the ordinary convolution downsampling operation is performed on the feature map to alleviate the loss of small object information. Step 2.2: Insert an upsampling module before the downsampling operation of the second layer of the YOLOv8n backbone feature extraction network, and insert a downsampling module after the C2f module of the fifth layer of the backbone feature extraction network; The upsampling module and downsampling module use the nearest neighbor interpolation algorithm and the ordinary convolution downsampling Conv module respectively; Step 2.3: Build the improved SwinTransformer-V2 module. The construction steps are: Step 2.3.1: Add a 1×1 channel offset convolution to the front end of the SwinTransformer-V2 module, and perform normalization and SiLU activation to enable early interaction between windows. This also introduces the idea of implicit position encoding to make the module structure more robust. Step 2.3.2: Add a window channel attention mechanism to the output of the SwinTransformer-V2 module. By multiplying the weights, the importance of all channels in the window is reflected, which indirectly increases the interaction between windows. The weight generation of the window channel attention mechanism comes from the output feature map of the self-attention mechanism Attention module in the SwinTransformer-V2 module. The weight generation module of the window channel attention can be expressed as follows: With' 2ij =Linear(z 2ij ) (1) With" 2i =SiLU(LN(z′ 2i )) (2) z″′ 2i =Sigmoid(LN(Linear(z″ 2i ))) (3) Among them, z 2ij represents the input of the window channel attention weight generation module after window division, i represents the i-th window, j represents the j-th channel, Linear represents the full connection operation, LN represents the normalization operation, and z in formula (1) 2ij After the linear operation, the single-channel input of the window size is transformed into an output of 1×1 size. The z′ in formula (2) 2i After LN normalization and SiLU activation, we get z″ 2i ; The linear operation in formula (3) does not change z″ 2i The shape of z″′ is obtained after LN normalization and Sigmoid activation. 2i ; z″′ in formula (4) 2i After the upsample neighbor interpolation upsampling operation of the window size, the output of the window channel attention weight generation module after the window division is obtained Step 2.3.3: Add a convolution module operation with a convolution kernel of 1×1 at the output of the SwinTransformer-V2 module to perform concat fusion with the subsequent convolution output; Step 2.4: Replace the last two C2f modules in the deep layer of the YOLOv8n backbone feature extraction network with the improved C3 module. The improved C3 module is obtained by replacing all the Bottleneck modules in the original C3 module with the improved SwinTransformer-V2 module. Step 3: Use the training set and validation set in step 1 to train, validate, and evaluate the improved YOLOv8n network model.

Images