A small target detection method for improving YOLOX network structure
By introducing the CSPDarkNet network into the YOLOX network and embedding multi-scale spatial pyramid pooling, global self-attention, and multi-scale feature fusion modules, the problems of incomplete feature extraction and insufficient global modeling ability of the YOLOX network in small object detection are solved, and higher detection accuracy and precision are achieved.
Patent Information
- Application Number
- CN202211128912.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-16
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2042-09-16
AI Technical Summary
The existing YOLOX network structure suffers from problems such as incomplete feature extraction, lack of global modeling ability, and low detection accuracy due to single-level feature extraction in small object detection.
The CSPDarkNet network is used as the backbone network, and a multi-scale spatial pyramid pooling module, a global self-attention module, and a multi-scale feature fusion module are embedded in its feature extraction process. The receptive field is increased by multi-scale spatial pyramid pooling layer, and the integrity and accuracy of feature extraction are improved by global self-attention mechanism and multi-scale feature fusion.
It effectively improves the accuracy and completeness of small object detection, solves the problems of feature loss and insufficient global modeling ability of YOLOX network in small object detection, and improves detection accuracy.
Smart Images
Figure CN115424029B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, and specifically to a method for small target detection using an improved YOLOX network structure. Background Technology
[0002] YOLOX is an object detection method based on the YOLO algorithm, proposed in 2021. The development team has been continuously improving and maintaining the YOLOX network model, so object detection methods based on the YOLOX network structure have been widely used.
[0003] However, existing small target detection methods based on the YOLOX network structure have the following drawbacks:
[0004] 1. In the feature extraction stage, using max pooling with a finite kernel size can make high-level convolutions too sparse, reducing their ability to capture global feature information and causing the loss of local and long-range feature information, resulting in incomplete feature extraction. For object detection, the appearance of objects is complex and varied. When convolutional networks extract features, if the network cannot extract target features from a global perspective, the final prediction result will lose a lot of detailed information, thus reducing the detection accuracy.
[0005] 2. The YOLOX backbone feature extraction network is a CNN network. CNN has translation invariance and locality, but lacks the ability to model globally and over long distances.
[0006] 3. Low-level features have high resolution and contain more location and detail information, but their semantics are lower and they are noisier. High-level features have strong semantic information, but their resolution is very low and their perception of details is poor. For small object detection, objects are complex and varied, such as deformed objects, occluded objects, and overlapping objects. Moreover, the number of small objects in the actual captured image accounts for a small percentage of the total image pixels. Therefore, using only a single-level convolution for feature extraction will result in a final prediction that differs greatly from the reality. Summary of the Invention
[0007] The purpose of this invention is to provide an improved method for small target detection using the YOLOX network structure, which aims to enable the YOLOX network to extract small target features from complex datasets and accurately detect the localization and effective detection of small targets.
[0008] To achieve the above objectives, this invention provides an improved method for small target detection using a YOLOX network structure, comprising the following steps:
[0009] The CSPDarkNet network was selected as the backbone network for initialization.
[0010] A multi-scale spatial pyramid pooling module is embedded in the fifth layer of the CSPDarkNet network;
[0011] A global self-attention module is embedded between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid.
[0012] A global self-attention module is embedded between C3_P3 and C3_n3 of the original path aggregation feature pyramid to form a multi-scale fusion module;
[0013] Train and evaluate the improved CSPDarkNet network model for small object detection.
[0014] Specifically, the process of embedding a multi-scale spatial pyramid pooling module in the fifth layer of the CSPDarkNet network involves inputting the location and semantic information extracted from the second-layer convolutional features into the multi-scale feature fusion stage, and replacing the fifth-layer spatial pyramid pooling layer with the multi-scale spatial pyramid pooling module.
[0015] The multi-scale spatial pyramid pooling module increases the amount of feature information extracted from the receptive field by using pooling kernel sizes of 3×3, 5×5, 9×9, 13×13, and 17×17. The difference from the original spatial pyramid pooling layer is that it adds small kernel pooling convolution of 3×3 for local information and large kernel pooling convolution of 17×17 for global information extraction.
[0016] Specifically, the process of embedding a global self-attention module between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid involves embedding a global self-attention module in the second and fifth layers respectively, which changes the original ordinary 3×3 spatial convolution of the Bottleneck block.
[0017] The process involves embedding a global self-attention module between C3_P3 and C3_n3 of the original path aggregation feature pyramid to form a multi-scale fusion module. In the feature fusion layer of the path aggregation feature pyramid, positional and semantic information of C2 feature extraction is added between C2 and C3_P4. A global self-attention module is then embedded between C3_P3 and C3_n3 of the path aggregation feature pyramid to complete the fusion of multi-scale feature information.
[0018] The multi-scale spatial pyramid pooling modules are arranged in an incremental pattern, that is, a set of small-sized pooling layers and a set of large-kernel-sized pooling layers are added. The receptive fields of the five branches increase in an incremental manner, with small receptive fields extracting neighboring information and large receptive fields extracting global information.
[0019] The global self-attention module includes spatial attention mechanism and channel attention mechanism.
[0020] The multi-scale fusion module first processes the features output by the CSPDarkNet network using a 1×1 convolution, then upsamples the resulting feature map to the same size as the input image and puts it into the feature fusion layer of the path aggregation feature pyramid. Finally, the fused features are processed using a 1×1 convolution to obtain the final prediction result.
[0021] This invention provides an improved small target detection method using the YOLOX network structure. By introducing and improving the CSPDarkNet network, it integrates modules such as multi-scale spatial pyramid pooling layers, global self-attention, and multi-scale feature fusion into the network model. This enables the extraction of small target features from complex datasets, accurately locating and effectively detecting small targets. It primarily addresses three technical problems: first, the problem of excessively sparse top-level convolutions due to limited use of max-pooling convolutions, leading to incomplete feature extraction; second, the lack of global and long-range modeling capabilities in CNNs; and third, the problem that features extracted from a single level can cause the final prediction results to deviate significantly from the actual situation. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart illustrating a small target detection method based on an improved YOLOX network structure according to the present invention.
[0024] Figure 2 This is a schematic diagram of the structure of the multi-scale spatial pyramid module of the present invention.
[0025] Figure 3 This is a schematic diagram of the global self-attention module of the present invention.
[0026] Figure 4 This is a schematic diagram of the structure of the multi-scale fusion module of the present invention. Detailed Implementation
[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0028] The following explanations of the Chinese and English terms used in this invention serve as a reference for the English descriptions that follow:
[0029] Multi-Spatial Pyramid Pooling (M-SPP) module;
[0030] Path Aggregation Feature Pyramid Networks (PAFPN);
[0031] Global self-attention block (GAB);
[0032] Spatial Pyramid Pooling (SPP);
[0033] Feature Pyramid Networks (FPN);
[0034] Path Aggregation Network (PANet).
[0035] Please see Figure 1 This invention provides an improved method for small target detection using a YOLOX network structure, comprising the following steps:
[0036] S1: Select the CSPDarkNet network as the backbone network for initialization;
[0037] S2: A multi-scale spatial pyramid pooling module (M-SPP module) is embedded in the fifth layer of the CSPDarkNet network. By setting different receptive fields in each layer, local or global spatial information is extracted. Local information is beneficial for extracting semantic detail features of small targets, while global information is beneficial for extracting location information of small targets.
[0038] S3: A global self-attention module is embedded between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid to capture the salient features of small targets from three dimensions (channel, space, tensor).
[0039] S4: A global self-attention module is embedded between C3_P3 and C3_n3 of the original path aggregation feature pyramid to form a multi-scale fusion module. The rich semantic information of the second layer of the CSPDarkNet network is supplemented to PAFPN through global self-attention, which facilitates the fusion of detailed features of small targets.
[0040] S5: Train and evaluate the improved CSPDarkNet network model for small object detection.
[0041] The process of embedding a multi-scale spatial pyramid pooling module in the fifth layer of the CSPDarkNet network involves inputting the location and semantic information extracted from the second-layer convolutional features into the multi-scale feature fusion stage, and replacing the fifth-layer spatial pyramid pooling layer with the multi-scale spatial pyramid pooling module.
[0042] The multi-scale spatial pyramid pooling module increases the amount of feature information extracted from the receptive field by using pooling kernel sizes of 3×3, 5×5, 9×9, 13×13, and 17×17. The difference from the original spatial pyramid pooling layer is that it adds small kernel pooling convolution of 3×3 for local information and large kernel pooling convolution of 17×17 for global information extraction.
[0043] The process of embedding a global self-attention module between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid is specifically to embed a global self-attention module in the second and fifth layers respectively, which changes the ordinary 3×3 spatial convolution of the original Bottleneck block.
[0044] The process of embedding a global self-attention module between C3_P3 and C3_n3 of the original path aggregation feature pyramid to form a multi-scale fusion module involves adding positional and semantic information of C2 feature extraction between C2 and C3_P4 in the feature fusion layer of the path aggregation feature pyramid, and embedding a global self-attention module between C3_P3 and C3_n3 of the path aggregation feature pyramid to complete the fusion of multi-scale feature information.
[0045] The following description, in conjunction with the improved module functions, further illustrates the present invention:
[0046] (1) Multi-scale spatial pyramid pooling module (M-SPP)
[0047] In convolutional neural networks, it's common to see training models with fixed inputs. However, when the input image cannot be of a fixed size, the object detection field typically employs the following methods:
[0048] 1) Reset the input by manually changing the output feature size of each layer to the input specifications required by other layers. However, this is too brute-force and may result in the loss of a lot of information or the addition of a lot of unnecessary information, such as image distortion, which will ultimately affect the output results.
[0049] 2) Replace the fully connected layers in the network and use global average pooling on the last convolutional layer. Global average pooling is only related to the number of channels and has nothing to do with the size of the feature map.
[0050] 3) It adopts the classic SPPNet structure layer, which can transform feature maps of arbitrary size into feature vectors of fixed size.
[0051] In the YOLO series, most implementations use 3 or 4 max pooling layers as intermediate layers of the SPP (Single Pooling Module), with a CBL (Cellular Pooling Module) convolutional layer added before and after to form the SPP module. If all targets use a limited number of pooling layers, as the network depth increases, the top-level convolutions will blur the feature information of small targets, resulting in the loss of some positional information of small targets. To solve this problem, this invention embeds an M-SPP module in the fifth layer of CSPDarkNet feature extraction to replace the original SPP. This can expand the network's receptive field while reducing the loss of local information, ensuring that the network can extract features from a global perspective.
[0052] like Figure 2 As shown, the M-SPP arrangement uses an incremental pattern, that is, small-sized pooling layers are grouped together, and the receptive field of each group increases in size from small to large. Small receptive fields extract neighboring information, while large receptive fields extract global information. In this way, the top convolutional layers can extract features from a wider range of pixels, ensuring that the network can acquire global feature information.
[0053] (2) Global self-attention block (GAB)
[0054] Global self-attention combines spatial and channel attention mechanisms. It replaces the standard 3×3 spatial convolution with Transformer global multi-head self-attention (MHSA) only in the third layer of CSPDarkNet. This significantly improves the accuracy of small object detection on the YOLOX baseline while reducing parameters, thus minimizing latency. For small object detection, factors such as complex backgrounds, variable object shapes, and small image size result in limited information extraction and low accuracy. Figure 3 As shown, the global self-attention module can better capture feature information, suppress non-significant features, and extract more small target features.
[0055] (3) Multi-scale feature fusion
[0056] In many object detection studies, fusing features at different scales is a crucial method for improving detection performance. Classic examples include Feature Pyramid Networks (FPNs), Path Aggregation Networks (PANet), and PAFPN. For small objects, accurate location information and good semantic information are essential for enhancing detection performance. Low-level features have high resolution and contain more location and detail information, but their semantics are lower and they are noisier. High-level features have strong semantic information, but their resolution is very low, resulting in poor detail perception.
[0057] To address these issues, this invention employs a multi-scale feature fusion approach to improve the accuracy of target detection, such as... Figure 4 As shown, the image output from the side edges of CSPDarkNet is first processed using a 1×1 convolution. A GAB module is then embedded in the second (C2) and fifth (C5) layers of CSPDarkNet. Next, the outputs from layers 2 (C2) to 5 (C5) are upsampled to the same size as the output image. Finally, the feature maps from these four layers are input into a multi-scale fusion layer for information fusion. The multi-scale feature fusion module integrates multi-scale and multi-level feature maps, making the final prediction result closer to the sampled true label.
[0058] Furthermore, the present invention provides a specific embodiment for verifying the execution steps:
[0059] Step 1: Pre-select Linux=3.10.0-1127.e17.x86_64, CentOS=7.8, PyTorch=1.7.1, TorchVision=0.8.2, GPU=GTX 2080TiPyTorch as the experimental platform;
[0060] Step 2: Set the improved YOLOX model parameters. The input image size is 640×640, the learning rate is le-4, embed and modify the module to be added, change the parameter configuration at the time of initialization, set the weight of the fusion layer to 1, reduce the learning rate by 10 times every 100 epochs, the weight decay is 2e-4, the number of training epochs is 100, and record the change data of model parameters once every epoch.
[0061] Step 3: Edit the M-SPP module in the fifth layer of CSPDarkNet. The difference from the original SPP is the addition of small kernel pooling convolutions (3×3) and large kernel pooling convolutions (17×17). Small kernel pooling convolutions focus on capturing local details of the feature map, while large kernel pooling convolutions focus on extracting global information from the feature map.
[0062] Step 4: Embed a GAB module into the second layer (C2) and the fifth layer (C5) of the backbone network CSPDarkNet, respectively. This changes the original spatial ordinary 3×3 convolution of the Bottleneck block, captures more salient location information at the lower level and salient semantic information at the higher level, and processes this information by feature fusion.
[0063] Step 5: Add positional and semantic information of C2 feature extraction between C2 and C3_P4 in the PAFPN feature fusion layer, and embed a GAB module between C3_P3 and C3_n3 of PAFPN to complete the fusion of multi-scale feature information. Since a large number of channels has a significant impact on weight allocation, GAB (mainly considering the information of the feature map and the number of feature channels) is used to assign larger weight factors to neighboring parameters through self-attention weight allocation, highlighting the global information that needs attention, and performing weighted summation to replace the ordinary spatial convolution 3×3 (3×3 convolution only focuses on the weighted summation of the surrounding 8 pixel values);
[0064] Step 6: Adjust the parameters of the network model based on the experimental results to achieve the best object detection verification results and maintain the stability of the training process. After training, test the training weight file using the test set.
[0065] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A method for small target detection using an improved YOLOX network structure, characterized in that, Includes the following steps: The CSPDarkNet network was selected as the backbone network for initialization. In the fifth layer of the CSPDarkNet network, a multi-scale spatial pyramid pooling module is embedded. Specifically, the positional and semantic information extracted from the second-layer convolutional features is input into the multi-scale feature fusion stage, and the spatial pyramid pooling layer of the fifth layer is replaced with the multi-scale spatial pyramid pooling module. The multi-scale spatial pyramid pooling module increases the amount of feature information extracted from the receptive field by using pooling kernel sizes of 3×3, 5×5, 9×9, 13×13, and 17×17. The difference from the original spatial pyramid pooling layer is that it adds small kernel pooling convolution of 3×3 for local information and large kernel pooling convolution of 17×17 for global information extraction. A global self-attention module is embedded between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid. A global self-attention module is embedded between C3_P3 and C3_n3 of the original path aggregation feature pyramid to form a multi-scale fusion module; In the feature fusion layer of the path aggregation feature pyramid, positional and semantic information of C2 feature extraction is added between C2 and C3_P4, and a global self-attention module is embedded between C3_P3 and C3_n3 of the path aggregation feature pyramid to complete the fusion of multi-scale feature information. The multi-scale spatial pyramid pooling modules are arranged in an incremental pattern, that is, a set of small-sized pooling layers and a set of large-kernel-sized pooling layers are added. The receptive fields of the five branches are increased in an incremental manner, with small receptive fields extracting neighboring information and large receptive fields extracting global information. The multi-scale fusion module first processes the features output by the CSPDarkNet network using a 1×1 convolution, then upsamples the resulting feature map to the same size as the input image, and puts it into the feature fusion layer of the path aggregation feature pyramid. Finally, the fused features are processed using a 1×1 convolution to obtain the final prediction result. Train and evaluate the improved CSPDarkNet network model for small object detection.
2. The small target detection method with improved YOLOX network structure as described in claim 1, characterized in that, The process of embedding a global self-attention module between the second and fifth layers of the CSPDarkNet network and the path aggregation feature pyramid is specifically to embed a global self-attention module in the second and fifth layers respectively, which changes the ordinary 3×3 spatial convolution of the original Bottleneck block.
3. The small target detection method with improved YOLOX network structure as described in claim 1, characterized in that, The global self-attention module includes spatial attention mechanism and channel attention mechanism.