A lightweight small target detection method based on stride-free convolution and context

Abstract

The present application provides a kind of lightweight small target detection method based on stride-free convolution and context, with YOLOv8s as baseline network, introduce stride-free convolution multiple feature fusion module NSC-MFFM, C-Cot module and GhostConv module to optimize YOLOv8s: replace the conventional convolution in YOLOv8s with NSC-MFFM;Add a shallower detection branch, and remove the deepest detection branch, use C-Cot module as the detection layer of YOLOv8s;Introduce GhostConv module to reduce the number of model parameters.It can improve the performance of small target detection while lightening YOLOv8s to meet the end-to-end industrial deployment.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a lightweight small target detection method based on non-stride convolution and context. Background Art

[0002] Object detection is a research hotspot in computer vision. Its main tasks include identifying object categories and locating their locations in images. Small object detection, a key challenge in object detection, involves detecting relatively small objects in an image. Small object detection has broad application value and significant research significance in fields such as national defense security, intelligent transportation, autonomous driving, and industrial automation. However, existing deep learning-based object detection algorithms perform poorly for small object detection due to factors such as their small image size, insufficient texture detail, and complex backgrounds.

[0003] Traditional object detection algorithms use hand-crafted features and machine learning methods for object detection. With the introduction of convolutional neural networks (CNNs), deep learning-based object detection algorithms have become the mainstream direction of object detection research due to their simple structure, ease of training, and excellent detection performance. Deep learning-based object detection models are mainly divided into two categories: one is a two-stage detection algorithm based on candidate regions, which includes region extraction and classification prediction based on candidate regions, such as R-CNN and Fast R-CNN; the other is a one-stage detection algorithm, represented by YOLO and SSD, which transforms object detection into a regression problem and predicts regression and classification probabilities to determine location and category.

[0004] To address the problem of missed detection and false detection of small targets. Oriented Bounding Boxes and BoxBoundary-Aware Vectors improve the detection of small targets by rotating the prediction box and the rotation detector, but this is only applicable to remote sensing scenarios. Replication enhancement technology is also used to enhance the training phase, thereby improving detection results. TPH-TOLOv5 effectively enhances the detection capability of small targets by introducing a target detection layer, combining the transformer prediction head and the CBAM attention mechanism. However, in dense target scenarios, the model may have missed detection issues. In addition, the number of parameters of the model has also increased, and there is still room for improvement to meet end-to-end industrial deployment. Summary of the Invention

[0005] In view of this, and in response to the defects and shortcomings of the existing technology, as well as the room for improvement, the present invention provides a lightweight small target detection method based on unstride convolution and context. Based on YOLOv8s, it can improve the small target detection performance while performing lightweight improvements on YOLOv8s to meet end-to-end industrial deployment. By improving the YOLOv8s structure, a balance between model lightweighting and performance is achieved, thereby improving the accuracy of small target detection results.

[0006] In this approach, the publicly available small object detection dataset, VisDrone2019-DET, was converted to the YOLO training format and divided into training, validation, and test sets. Using YOLOv8s as the baseline network, the conventional convolutions in YOLOv8s were first replaced with the NSC-MFFM, based on a strided convolutional model. This effectively prevents the neglect of important feature information of small objects and improves the model's detection accuracy. A shallower detection branch was then added, while the deepest detection branch was removed. The C-Cot module, which contains rich contextual information, was used as the detection layer in YOLOv8s. This better separated small-scale objects from noisy backgrounds, making small objects easier to identify. Finally, the model was made even more lightweight by introducing the lightweight and efficient GhostConv module, significantly reducing the number of model parameters without significantly reducing model accuracy. The small object detection model was trained on the training set and hyperparameters were adjusted using the validation set to obtain the optimal small object detection model. The test set was then fed into the optimal small object detection model, and the small object detection results were output.

[0007] The technical solution specifically adopted by the present invention to solve the technical problem is:

[0008] A lightweight small object detection model based on non-strided convolution and context is proposed. Taking YOLOv8s as the baseline network, the non-strided convolution multiple feature fusion module (NSC-MFFM), the C-Cot module, and the GhostConv module are introduced to optimize YOLOv8s. NSC-MFFM replaces the conventional convolution in YOLOv8s; a shallower detection branch is added, and the deepest detection branch is removed. The C-Cot module is used as the detection layer of YOLOv8s; and the GhostConv module is introduced to reduce the number of model parameters.

[0009] Furthermore, the improved YOLOv8s small target detection model includes: backbone network, neck network and detection head;

[0010] The backbone network includes a first NSC-MFFM module, a second NSC-MFFM module, a first Ghost-C2f module, a third NSC-MFFM module, a second Ghost-C2f module, a fourth NSC-MFFM module, a third Ghost-C2f module, a fifth NSC-MFFM module, a fourth Ghost-C2f module and an SPPF module connected in sequence;

[0011] The neck network includes a first upsampling module, a first aggregation module, a fifth Ghost-C2f module, a second upsampling module, a second aggregation module, a sixth Ghost-C2f module, a third upsampling module, a third aggregation module, a seventh Ghost-C2f module, a first C-Cot module, a sixth NSC-MFFM module, a fourth aggregation module, an eighth Ghost-C2f module, a second C-Cot module, a seventh NSC-MFFM module, a fifth aggregation module, a ninth Ghost-C2f module, and a third C-Cot module, which are connected in sequence;

[0012] The first aggregation module is connected to the third Ghost-C2f module, the second aggregation module is connected to the second Ghost-C2f module, the third aggregation module is connected to the first Ghost-C2f module, the SPPF module is connected to the first upsampling module, the fifth aggregation module is connected to the fifth Ghost-C2f module, and the fourth aggregation module is connected to the sixth Ghost-C2f module;

[0013] The detection head includes three identical decoupling heads, and the first C-Cot module, the second C-Cot module, and the third C-Cot module are respectively connected to the three decoupling heads.

[0014] Furthermore, the input of the NSC-MFFM module is a feature map of C×H`×W`, which is first obtained by a conventional convolution module with a step size of 1; then divided into 4 sub-feature maps of size C×H / 2×W / 2; feature mapping is performed on the sub-feature maps of the four branches respectively, and finally the output feature maps of the four branches are channel-wise spliced ​​to output the spliced ​​feature map.

[0015] Furthermore, the Ghost-C2f module performs conventional convolution on the input feature map, and then splits the convolved feature map into two branches, the right branch, which is continuously subjected to two GhostConv modules to extract features; then, the feature map obtained by the two GhostConv modules and the left branch feature map are spliced ​​together, and finally, the spliced ​​feature map is output after conventional convolution.

[0016] Furthermore, the GhostConv module first extracts feature information through conventional convolution to obtain an intermediate feature map; then the intermediate feature map is filtered to extract features, and finally the output intermediate feature maps obtained by filtering are spliced ​​together.

[0017] Furthermore, the C-Cot module passes the input feature map through two conventional convolutions with a convolution kernel of 1×1, dividing it into left and right branches to achieve channel dimensionality reduction respectively; then the left branch passes through the residual module of Dcot-block and is superimposed with the right branch; finally, a conventional convolution with a convolution kernel of 1×1 is performed to interact information between channels, and then the feature map is output.

[0018] Furthermore, the Dcot-block module obtains a static contextual expression by performing conventional convolution on the key of the input feature map. Subsequently, the encoded key feature is connected to the input query, and a dynamic multi-head attention matrix is ​​learned through continuous conventional convolution with a convolution kernel of 1×1. The dynamic contextual expression of the input is realized by multiplying the learned attention matrix with the input value. Then, the static and dynamic contextual expressions are fused. Deep convolution is used to aggregate local information on the input, and this branch is used as a shared information branch. Finally, its output is merged with the fused information of static and dynamic context to obtain the final output.

[0019] The final lightweight small object detection method based on unstride convolution and context is obtained. Based on the above model, it includes the following steps:

[0020] Convert the small object detection dataset VisDrone2019-DET into the YOLO training format and divide it into training set, validation set and test set;

[0021] Train the small target detection model based on the training set, optimize the parameters based on the validation set, and finally obtain the optimal small target detection model;

[0022] The test set is input into the optimal small target detection model, and the small target detection result is output.

[0023] Compared to the existing technology, the present invention and its preferred embodiment first adopt a multi-scale feature fusion module based on NSC-MFFM (non-strided convolution) to replace large-stride convolution for feature extraction. At the same time, a C-Cot module with rich contextual information is introduced as a detection layer to accurately locate small targets, and the detection branch of the network is optimized. Specifically, while adding a shallow branch, the deepest detection branch mainly used for large target detection is carefully removed. Finally, GhostConv is introduced to achieve lightweight network model. While ensuring that the model performance is not negatively affected, the number of model parameters is effectively reduced, which greatly improves the number of model parameters and detection accuracy. Compared with the original model, it achieves a balance between lightweight and high performance, ensuring the accuracy of small target detection while also laying a solid foundation for its porting to terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0025] Figure 1 This is a flowchart of a lightweight small target detection method based on unstride convolution and context proposed in an embodiment of the present invention;

[0026] Figure 2 This is a diagram of the YOLOv8s network structure used as the baseline network in an embodiment of the present invention;

[0027] Figure 3 This is a diagram of the improved YOLOv8s (NSCOT-YOLO) network structure according to an embodiment of the present invention;

[0028] Figure 4 This is a structural diagram of the NSC-MFFM module based on multi-scale feature fusion without strided convolution in an embodiment of the present invention;

[0029] Figure 5 This is a structural diagram of the GhostConv module according to an embodiment of the present invention;

[0030] Figure 6 This is a structural diagram of the Ghost-C2f module according to an embodiment of the present invention;

[0031] Figure 7 This is a structural diagram of the Dcot-block module in an embodiment of the present invention.

[0032] Figure 8 This is a structural diagram of the C-Cot module in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the features and advantages of this patent more clearly understood, the following embodiments are specifically described in detail as follows:

[0034] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the art to which this application belongs.

[0035] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0036] The embodiment of the present invention discloses a lightweight small target detection method based on non-strided convolution and context, such as Figure 1 As shown, the following steps are included:

[0037] 1. Convert the public small object detection dataset into the YOLO training format and divide it into training set, validation set, and test set;

[0038] In this example, the VisDrone2019-DET public dataset was used for target detection image data. This dataset was converted to the YOLO training format using custom Python code. The training set consists of 6,471 images, used for training the small target detection model based on the improved YOLOv8s; the validation set consists of 548 images, used for parameter optimization of the small target detection model based on the improved YOLOv8s; and the test set consists of 1,610 images, used for performance evaluation of the small target detection model based on the improved YOLOv8s.

[0039] 2. We introduce the NSC-MFFM module based on non-stride convolution, the C-Cot module containing rich contextual information, and the GhostConv module to optimize the YOLOv8s object detection algorithm. We also optimize the detection layer of YOLOv8s and build a lightweight small object detection model based on the improved YOLOv8s.

[0040] The improvement process of constructing the small target detection model based on the YOLOv8s improvement in this embodiment can be divided into three steps:

[0041] The first step is to replace the large-stride convolution in YOLOv8s with the NSC-MFFM module. This module can effectively extract the features of small targets, suppress interference information, and improve the detection effect at the cost of slightly increasing the number of parameters and computational complexity.

[0042] Because small objects occupy only a small portion of the feature map, the feature information they contain is very limited. In this case, the feature information skipped by large-stride convolutions is often crucial. Furthermore, large-stride convolutions weaken the distinction between target pixels and surrounding pixels in the feature map, making the features of the target object more difficult to distinguish. This reduces the efficiency and quality of feature extraction by the network, ultimately leading to poor detection of small objects. To address this issue, the NSC-MFFM module is proposed.

[0043] The workflow of the NSC-MFFM module is as follows Figure 4 The specific operations are as follows:

[0044] (1) The feature map input to the NSC-MFFM module is C×H`×W`;

[0045] (2) First, a conventional convolution module with a step size of 1 is used to obtain the feature map C×H×W;

[0046] (3) Divide it into four sub-feature maps of size C×H / 2×W / 2, which are respectively used as B1 branch, B2 branch, B3 branch, and B4 branch;

[0047] (4) The B1 branch and the B2 branch are fused, and coordinate attention is introduced into the B2 branch to extract position information and enhance the detection capability of small targets.

[0048] (5) The feature map obtained by the attention module is added to the B3 branch for fusion, and the insignificant weights are suppressed by the NAM method.

[0049] (6) The output of branch B3 will be added to branch B4 for fusion, and a conventional convolution with a convolution kernel of 1×1 will be used to enhance the information exchange between channels.

[0050] (7) The output feature maps of the four branches B1, B2, B3, and B4 are spliced ​​together, and the convolutional nonlinear expression is enhanced through the shuffling module to improve the feature extraction capability.

[0051] To further explore where the proposed NSC-MFFM module should be placed in the YOLOv8s network to achieve the best detection performance, we designed an experiment with four scenarios:

[0052] (1) Case 1 is when the NSC-MFFM module is not used;

[0053] (2) Case 2 is to replace the conventional convolution with a stride of 2 in the neck network (Neck) structure with the NSC-MFFM module;

[0054] (3) Case 3 is that the conventional convolution with a stride of 2 in the backbone network structure is replaced by the NSC-MFFM module;

[0055] (4) Case 4 is to replace all conventional convolutions with a stride of 2 in the network with NSC-MFFM modules.

[0056] Table 1 Comparison of experimental results under different conditions

[0057] Network Model Precision / % mAP@0.5 / % mAP@0.5:0.95 / % Parameter quantity / M Case 1 50.6 40.0 24.1 11.1 Case 2 51.5 40.1 24.1 11.7 Case 3 54.2 42.7 26.1 12.6 Case 4 54.4 43.3 26.2 13.2

[0058] The experimental results in Table 1 show that the network performs best in Case 4, extracting more effective feature information than the original YOLOV8s network. Therefore, the final solution chosen is to replace all conventional convolutions with a stride of 2 in the network with NSC-MFFM modules.

[0059] The second step is to use the C-Cot module to replace the three detection layers in YOLOv8s and optimize the detection branch of YOLOv8s.

[0060] For small object detection, rich contextual information is crucial for improving performance. This is because small objects are often tiny, sometimes just a few pixels, and are easily lost in the surrounding noise, making their shape difficult to determine. Contextual information helps the network model separate the object from the background noise, better understand the object's shape and surroundings, and provide more semantic information.

[0061] The C-CoT module, with its rich contextual information, can better help the detection head locate small targets. However, since small targets occupy a very small portion of the feature map, their features can be easily lost when the downsampling stride is larger than the target's size. Because small targets occupy a larger portion of shallower feature maps with higher resolution, we decided to optimize feature prediction and positioning by adding a shallower C-COT module as a detection layer and a corresponding detection head. This detection layer is more targeted at small targets and effectively increases the probability of small object detection.

[0062] The workflow of the C-Cot module is as follows Figure 8 The specific operations are as follows:

[0063] (1) The C-Cot module passes the input feature map through two conventional convolutions with a convolution kernel of 1×1, which are divided into two branches, the left and the right, to achieve channel dimensionality reduction respectively;

[0064] (2) The left branch then passes through the residual structure of the Dcot-block module and is superimposed with the right branch;

[0065] (3) After the convolution kernel is 1×1, the information interaction between channels is carried out, and then the feature map is output.

[0066] The C-Cot module uses a dimensionality reduction followed by a dimensionality increase to reduce the computational burden and number of parameters while maintaining the network's expressive power. This multi-scale feature processing improves the network's perception and adaptability to features at different scales.

[0067] The workflow of the Dcot-block module is as follows Figure 7 The specific operations are as follows:

[0068] (1) The Dcot-block module sequentially passes the key of the input feature map through the conventional convolutional coding with a convolution kernel of 1×3 and 3×1 to obtain a static context representation. Compared with the conventional convolutional coding with a convolution kernel of 3×3, this coding method not only reduces the amount of computation and the number of parameters, but also captures horizontal and vertical features;

[0069] (2) The encoded key features are connected to the input query, and a dynamic multi-head attention matrix is ​​learned through continuous conventional convolution with a convolution kernel of 1×1. The learned attention matrix is ​​multiplied by the input value to achieve dynamic contextual expression of the input;

[0070] (3) Fusion of static and dynamic contextual expressions;

[0071] (4) Use simple depthwise convolution to aggregate local information on the input, and this branch is used as the shared information branch;

[0072] (5) Finally, the output is merged with the information of static and dynamic context fusion to obtain the final output;

[0073] Considering the design goal of this invention is to provide a lightweight small target detection solution based on unstride convolution and context, the detection target is small objects. In order to achieve generalization, the YOLOv8s model has three detection layers corresponding to the classification and location of large, medium and small objects respectively. Among them, the detection layer with the deepest depth and lowest resolution is mainly used to deal with large targets, so we try to remove the detection layer and detection branch with the deepest depth and lowest resolution.

[0074] To further explore the optimal solution for YOLOv8s detection branch, we designed experiments, which are divided into the following four cases:

[0075] (1) Case 1 is the YOLOv8s model;

[0076] (2) Case 2 is to use the C-Cot module to replace the three detection layers in YOLOv8s;

[0077] (3) Case 3 adds a shallower C-Cot module on the basis of Case 2;

[0078] (4) Case 4 is based on Case 3 but without the deepest detection branch.

[0079] Table 2 is a comparison of experimental results of different addition methods of C-Cot modules

[0080] Network Model Precision / % mAP@0.5 / % mAP@0.5:0.95 / % Parameter quantity / M Case 1 50.6 40.0 24.1 11.1 Case 2 51.6 40.9 24.6 12.6 Case 3 56.2 45.5 28.1 12.1 Case 4 54.7 45.4 28.1 7.8

[0081] From the experimental results in Table 2, it can be seen that the detection ability of using the C-Cot module as the detection layer is significantly stronger than that of the YOLOv8s model. At the same time, when a shallow detection layer is added on the original basis, the detection ability of the network is greatly increased. This shows that the newly added shallow detection layer has greatly improved the positioning and classification capabilities of small objects. After removing the deepest detection branch, the mAP of the network has hardly changed, but the number of parameters has dropped from 12.1M to 7.8M. This shows that the deepest branch was originally responsible for detecting larger objects, and in the present invention, all detection targets are small objects. Therefore, this detection branch contributes less to the detection of small targets and can be removed. At the same time, the number of parameters can also be greatly reduced, which helps to make the model lightweight. Therefore, the final solution for optimizing the detection layer is to add a shallower detection branch, use the C-Cot module as the detection layer, and finally remove the deepest detection branch.

[0082] Making network models lightweight is a prerequisite for deploying them. During this lightweighting process, it is usually necessary to reduce the number of model parameters. However, reducing the number of parameters often leads to a decrease in model accuracy, which is unacceptable.

[0083] The third step is to use the Ghost-C2f module to replace the C2f module in YOLOv8s. The workflow of the Ghost-C2f module is as follows Figure 6 The specific operations are as follows:

[0084] (1) The input feature map undergoes conventional convolution;

[0085] (2) Split the convolutional feature map into two branches: left and right;

[0086] (3) The right branch passes through two GhostConv modules to extract features;

[0087] (4) The feature maps obtained by the two GhostConv modules and the left branch feature map are spliced ​​together, and finally the spliced ​​feature map is output after conventional convolution;

[0088] The workflow of the GhostConv module is as follows Figure 5As shown in the figure, feature information is first extracted through conventional convolution to obtain an intermediate feature map. The intermediate feature map is then filtered to extract features, and finally the output intermediate feature maps obtained by filtering are spliced ​​together;

[0089] Through the above three steps Figure 2 The YOLOv8s model shown in the figure is optimized to obtain a lightweight small target detection model. The improved YOLOv8s (i.e. NSCOT-YOLO in the following text) network structure is referenced Figure 3 , including the backbone network Backbone, neck network Neck and detection head Head.

[0090] Specifically, the backbone network, represented by the first 10 layers in the network structure diagram, is primarily used for feature extraction. It includes the first NSC-MFFM module, the second NSC-MFFM module, the first Ghost-C2f module, the third NSC-MFFM module, the second Ghost-C2f module, the fourth NSC-MFFM module, the third Ghost-C2f module, the fifth NSC-MFFM module, the fourth Ghost-C2f module, and the SPPF module, which are connected in sequence.

[0091] Neck network Neck: It is connected in sequence: the first upsampling module, the first aggregation module, the fifth Ghost-C2f module, the second upsampling module, the second aggregation module, the sixth Ghost-C2f module, the third upsampling module, the third aggregation module, the seventh Ghost-C2f module, the first C-Cot module, the sixth NSC-MFFM module, the fourth aggregation module, the eighth Ghost-C2f module, the second C-Cot module, the seventh NSC-MFFM module, the fifth aggregation module, the ninth Ghost-C2f module, and the third C-Cot module; the first aggregation module is connected to the third Ghost-C2f module, the second aggregation module is connected to the second Ghost-C2f module, the third aggregation module is connected to the first Ghost-C2f module, the SPPF module is connected to the first upsampling module, the fifth aggregation module is connected to the fifth Ghost-C2f module, and the fourth aggregation module is connected to the sixth Ghost-C2f module.

[0092] The detection head includes three identical decoupling heads, which are used to predict the type and location of the target. The original YOLOv8s detection head is retained in the algorithm without modification. The first C-Cot module, the second C-Cot module, and the third C-Cot module are connected to the three decoupling heads respectively.

[0093] 3. Train the small object detection model based on the improved YOLOv8s based on the training set, and optimize the hyperparameters based on the validation set to obtain the optimal small object detection model. The training parameters for this implementation are as follows:

[0094] The experimental environment was Ubuntu 18.04, using a GeForce RTX 2080 Ti GPU for accelerated computing. The experiments were conducted using the Python programming language and the PyTorch 1.12.1 deep learning framework. The input image resolution was set to 640 x 640, the number of epochs was 300, the batch size was 4, the initial learning rate was 0.01, and the SGD optimizer was used. All experiments in this example used the same training parameters.

[0095] 4. Input the test set into the optimal small target detection model and output the small target detection results.

[0096] 5. Validity Verification:

[0097] To verify the effectiveness of the lightweight optimization method used in this experiment, the NSC-MFFM module, the C-Cot module containing rich contextual information, and the detection branch optimization, this example set up ablation experiments and comparative experiments to explore the impact of the proposed improvement method and some other improvement methods on the performance of the YOLOv8s model.

[0098] 5.1 Performance Indicators

[0099] First, let's talk about performance indicators. In the YOLO series of models, the main indicators for evaluating its network performance are the following: Precision (P), Recall (R), and Mean Average Precision (mAP). This experiment uses mAP50 and mAP50:95 as performance reference indicators. mAP@0.5 and mAP@0.5:0.95 represent the mAP value when the IoU threshold is 0.5 and the average mAP value when the IoU starts from 50% and increases to 95% with a step size of 0.05. The larger the mean average precision mAP, the higher the overall accuracy of the model. The calculation formulas for each indicator are as follows:

[0100] (1)

[0101] (2)

[0102] (3)

[0103] (4)

[0104] Among them, TP is the number of correctly predicted positive samples, FN is the number of incorrectly predicted negative samples, and FP is the number of incorrectly predicted positive samples.

[0105] When discussing the balance between lightweight and performance, the number of model parameters must also be considered. Therefore, it is also necessary to introduce the parameter amount (Params), a detailed parameter of the model architecture.

[0106] 5.2 Method Comparison

[0107] This embodiment uses ablation experiments and comparative experiments to partially verify the effectiveness of the improved algorithm.

[0108] Ablation experiments are divided into the following five cases:

[0109] (1) Case 1 is the baseline model (YOLOv8s);

[0110] (2) Case 2 is to use the C-Cot module to replace the detection layer of the baseline model (YOLOv8s) based on Case 1;

[0111] (3) Case 3 is based on Case 2 and adds a shallower C-Cot module as the detection layer and removes the deepest detection branch;

[0112] (4) Case 4 is based on Case 3 and uses the NSC-MFFM module to replace the conventional convolution with a step size of 2 in the model;

[0113] (5) Case 5 is based on Case 4 and replaces the C2f module in the model with the Ghost-C2f module;

[0114] Table 3. Comparison of ablation experiments

[0115] Network Model Precision / % mAP@0.5 / % mAP@0.5:0.95 / % Parameter quantity / M Case 1 50.6 40.0 24.1 11.1 Case 2 51.6 40.9 24.6 12.6 Case 3 54.7 45.5 28.1 7.8 Case 4 57.9 47.8 29.8 9.4 Case 5 54.8 46.8 28.7 6.8

[0116] Table 3 shows that the baseline model (YOLOv8s) has 11.1 million parameters. Based on this, the three detection layers in YOLOv8s were replaced with the C-Cot module. Compared to the original model, the precision increased by 1 percentage point, the mAP@0.5 metric increased by 0.9 percentage points, and the mAP@0.5:0.95 metric increased by 0.5 percentage points, while the parameter count slightly increased by 1.5 million. This demonstrates that the C-Cot module effectively utilizes rich contextual information to enhance the perception of small objects. By adding a shallower C-Cot module as the detection layer and removing the deepest detection branch in the YOLOv8s model, the precision improved by 3.1 percentage points, the mAP@0.5 metric increased by 4.6 percentage points, and the mAP@0.5:0.95 metric increased by 3.5 percentage points. The parameter count decreased by 4.8 million. This indicates that in shallow feature maps with higher resolution, small objects are easier to detect because they account for a large proportion. Furthermore, the deepest detection branch primarily targets large objects and contributes little to small object detection. Removing this branch not only does not affect network accuracy but also significantly reduces the number of model parameters. Building on the above improvements, replacing the conventional convolutional layer with a stride of 2 with the NSC-MFFM module in the model increases the number of parameters by 1.6 million compared to the original model, but improves precision by 3.2 percentage points, mAP@0.5 by 2.3 percentage points, and mAP@0.5:0.95 by 1.7 percentage points. Therefore, the NSC-MFFM module effectively extracts more precise object features and avoids the loss of small object feature information. Finally, replacing the C2f module in the improved network model with the Ghost-C2f module based on the GhostConv module reduces the number of model parameters to 6.8 million compared to the original model, while only decreasing the mAP@0.5 by 1 percentage point and the mAP@0.5:0.95 by 1.1 percentage points. As can be seen, the Ghost-C2f module significantly reduces the number of parameters and computational complexity, while slightly reducing detection accuracy, which is acceptable. This shows that the improved YOLOv8s is effective in both performance improvement and lightweighting.

[0117] While keeping the experimental environment unchanged, the performance of the improved YOLOv8s (NSCOT-YOLO) was compared with the currently commonly used target detection algorithms: YOLOv8s, YOLOv5s, YOLOv3-tiny, YOLOv6s, and YOLOv7-tiny.

[0118] Table 4 Comparison of some target detection algorithms

[0119] Network Model Precision / % mAP@0.5 / % mAP@0.5:0.95 / % Parameter quantity / M YOLOv8s 50.6 40.0 24.1 11.1 YOLOv3-tiny 38.9 23.2 13.1 12.1 YOLOv5s 47.4 34.4 19.1 7.0 YOLOv7-tiny 47.3 36.7 19.1 6.1 YOLOv6s 48.0 35.6 21.2 16.2 NSCOT-YOLO (this example) 54.8 46.8 28.7 6.8

[0120] The comparison results are shown in Table 4. Compared to YOLOv8s, NSCOT-YOLO achieves a 4.2 percentage point improvement in Precision, a 6.8 percentage point increase in mAP@0.5, and a 4.6 percentage point increase in mAP@0.5:0.95, using images of the same input size. This improves accuracy on small objects while reducing the number of parameters by 39% compared to YOLOv8s, achieving lightweight performance. NSCOT-YOLO's mAP surpasses that of other classic network models. In terms of parameter count, only YOLOv7-tiny, with a 0.7M smaller parameter count, achieves a 10.1 percentage point increase in mAP@0.5 and a 9.6 percentage point increase in mAP@0.5:0.95 over YOLOv7-tiny.

[0121] Experimental results show that compared with YOLOv8s, NSCOT-YOLO has better accuracy performance on small targets and overall performance improvement, and has excellent performance in balancing lightweight and algorithm performance.

[0122] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0123] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0128] It should be noted that, unless otherwise defined, the technical or scientific terms used in the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0129] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

[0130] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive various other forms of lightweight small target detection methods based on non-strided convolution and context under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of this invention should be covered by this patent.

Claims

1. A lightweight small target detection model based on non-strided convolution and context, characterized by: Using YOLOv8s as the baseline network, we introduce the non-stride convolutional multi-feature fusion module (NSC-MFFM), the C-Cot module, and the GhostConv module to optimize YOLOv8s. We replace the conventional convolution in YOLOv8s with NSC-MFFM, add a shallower detection branch, remove the deepest detection branch, and use the C-Cot module as the detection layer of YOLOv8s. The GhostConv module is introduced to reduce the number of model parameters; The improved YOLOv8s small target detection model includes: backbone network, neck network and detection head; The backbone network includes a first NSC-MFFM module, a second NSC-MFFM module, a first Ghost-C2f module, a third NSC-MFFM module, a second Ghost-C2f module, a fourth NSC-MFFM module, a third Ghost-C2f module, a fifth NSC-MFFM module, a fourth Ghost-C2f module and an SPPF module connected in sequence; The neck network includes a first upsampling module, a first aggregation module, a fifth Ghost-C2f module, a second upsampling module, a second aggregation module, a sixth Ghost-C2f module, a third upsampling module, a third aggregation module, a seventh Ghost-C2f module, a first C-Cot module, a sixth NSC-MFFM module, a fourth aggregation module, an eighth Ghost-C2f module, a second C-Cot module, a seventh NSC-MFFM module, a fifth aggregation module, a ninth Ghost-C2f module, and a third C-Cot module, which are connected in sequence; The first aggregation module is connected to the third Ghost-C2f module, the second aggregation module is connected to the second Ghost-C2f module, the third aggregation module is connected to the first Ghost-C2f module, the SPPF module is connected to the first upsampling module, the fifth aggregation module is connected to the fifth Ghost-C2f module, and the fourth aggregation module is connected to the sixth Ghost-C2f module; The detection head includes three identical decoupling heads, and the first C-Cot module, the second C-Cot module, and the third C-Cot module are respectively connected to the three decoupling heads; The input of the NSC-MFFM module is a feature map of size C×H`×W`, which is first passed through a conventional convolution module with a step size of 1 to obtain a feature map of size C×H×W; then it is divided into four sub-feature maps of size C×H / 2×W / 2; feature mapping is performed on the sub-feature maps of the four branches respectively, and finally the output feature maps of the four branches are channel-wise spliced ​​to output the spliced ​​feature map; The Ghost-C2f module performs conventional convolution on the input feature map, then splits the convolved feature map into two branches, the right branch, which is passed through two GhostConv modules to extract features. The feature map obtained by the two GhostConv modules is then concatenated with the feature map of the left branch, and the concatenated feature map is output after conventional convolution. The C-Cot module passes the input feature map through two conventional convolutions with a convolution kernel of 1×1, which is divided into left and right branches to achieve channel dimensionality reduction respectively; then the left branch passes through the residual module of Dcot-block and is superimposed with the right branch; finally, a conventional convolution with a convolution kernel of 1×1 is performed to interact information between channels, and then the feature map is output.

2. The lightweight small target detection model based on unstride convolution and context according to claim 1, characterized in that: The GhostConv module first extracts feature information through conventional convolution to obtain an intermediate feature map; then the intermediate feature map is filtered to extract features, and finally the output intermediate feature maps obtained by filtering are spliced ​​together.

3. The lightweight small target detection model based on unstride convolution and context according to claim 1, characterized in that: The Dcot-block module obtains a static contextual expression by performing conventional convolution on the key of the input feature map. Subsequently, the encoded key feature is connected to the input query, and a dynamic multi-head attention matrix is ​​learned through continuous conventional convolution with a 1×1 convolution kernel. The dynamic contextual expression of the input is realized by multiplying the learned attention matrix with the input value. Then, the static and dynamic contextual expressions are fused. Deep convolution is used to aggregate local information on the input, and this branch is used as a shared information branch. Finally, its output is merged with the fused information of static and dynamic context to obtain the final output.

4. A lightweight small target detection method based on non-strided convolution and context, based on a lightweight small target detection model based on non-strided convolution and context according to any one of claims 1-3, characterized in that: The following steps are involved: Convert the small object detection dataset VisDrone2019-DET into the YOLO training format and divide it into training set, validation set and test set; Train the small target detection model based on the training set, optimize the parameters based on the validation set, and finally obtain the optimal small target detection model; The test set is input into the optimal small target detection model, and the small target detection result is output.

Images