Small target detection optimization method based on self-rejection convolution

By designing self-immune convolution module and feature extraction selector in small object detection, the problems of background information interference and feature loss in small object detection are solved, the detection accuracy and recall rate are improved, and the smaller object detection is achieved more efficient Effect.

CN118711039BActive Publication Date: 2024-11-15XIAN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411178429.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-15
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

The existing small object detection methods are difficult to extract multi-scale features under complex backgrounds, and the target features are easily disturbed by background information, resulting in low detection accuracy and loss of features.

Method used

Design the self-immune convolution module, use the secondary center difference to extract small target features, and combine the vanilla convolution to characterize background information, and balance the target features and background information in the YOLOv8 backbone network through feature extraction selector, and replace the YOLOv8 neck structure. The vanilla convolution is an autoimmune convolution module.

Benefits of technology

The accuracy, recall and average detection accuracy of small target detection are improved, the detection effect of small targets is significantly improved, feature loss and noise are reduced, and the detection performance of the model is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118711039B_ABST
    Figure CN118711039B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of target detection methods, and discloses a small target detection optimization method based on self-disturbance rejection convolution, including: step 1, designing a self-disturbance rejection convolution module, extracting small target features by using secondary central difference, and combining vanilla convolution to characterize effective background information; step 2, constructing a feature extraction selector in the YOLOv8 backbone network, activating a strategy function to constrain the differential operation, thereby balancing the target features and background information; step 3, using the feature extraction selector and self-disturbance rejection convolution to respectively improve the feature fusion network of the YOLOv8 backbone network and the neck structure, and obtaining an optimization method. The method of the present invention solves the problems of existing small target detection methods that the target is greatly disturbed by background information and the features are easily lost during the sampling process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of target detection methods and relates to a small target detection optimization method based on self-anti-disturbance convolution. Background Art

[0002] Object detection is an important research direction in the field of computer vision, which aims to detect given objects in images to obtain information such as location and size. With the rapid development of deep learning, the efficiency of object detection has been significantly improved, and it is widely used in many fields such as infrared detection, intelligent transportation and medical imaging. According to the size of the target, the object detection task can be divided into general object detection and small object detection. General object detection is mainly aimed at objects with large and medium sizes and rich features, and small object detection is faced with objects that account for less than 0.12% of the image or have a size less than 32:32 pixels.

[0003] Object detection methods can be divided into traditional object detection and deep learning-based object detection. Traditional object detection methods use underlying features such as the edge and shape of the object for detection. This type of method has weak feature representation capabilities and is greatly affected by background contrast, so it performs poorly in terms of accuracy and robustness. Object detection methods based on deep learning can generally be divided into candidate region-based detection algorithms and regression-based detection algorithms. Among them, the candidate region-based detection algorithm divides the object detection problem into two stages. First, a large number of target candidate regions are generated, and the features of the effective target regions are extracted through a convolutional neural network to generate candidate regions; secondly, the candidate regions are placed in a classifier for classification and position correction. Therefore, this type of method is also called a two-stage detection method. Typical models include R-CNN, Fast-RCNN, Faster-RCNN, etc. Since the candidate region selection of this type of method takes a long time, its detection speed is slow. Regression-based detection methods do not need to generate a priori boxes, but directly determine the target area and generate the predicted box position through regression analysis. This type of method only regresses the predicted target object in a single stage, and is also called a single-stage detection method. The fast convergence feature makes this type of method more efficient in target detection tasks and has become the research focus of target detection in recent years. Its representative algorithms include SSD and YOLO series. However, although regression-based detection methods have significant effects on general target detection, they are difficult to extract features for small-sized targets, lack context information, and are interfered by background information, resulting in the fact that the detection effect of this type of method on small targets still needs to be improved.

[0004] To improve the performance of small object detection, Kim et al. (Kim M, Jeong J, Kim S. ECAP-YOLO: Efficient channel attention pyramid YOLO for small object detection in aerial image [J]. Remote Sensing, 2021, 13 (23): 4851. https: / / doi.org / 10.3390 / rs13234851) proposed an efficient channel attention pyramid, which uses transposed convolution for upsampling to avoid greater information loss, but it is difficult to extract multi-scale features in complex backgrounds. Cao et al. (Cao J, Zhuang Y, Wang M, et al. Pedestrian detection algorithm based on ViBe and YOLO [C] / / Proceedings of the 2021 5th International Conference on Video and Image Processing. 2021: 92-97.) combined the ViBe algorithm with YOLO. Although it improved the effect of small object detection to a certain extent, the number of parameters of the model increased significantly. Betti et al. (Betti A, Tucci M. YOLO-S: A lightweight and accurate YOLO-like network for small target detection in aerial imagery[J]. Sensors, 2023, 23(4): 1865. https: / / doi.org / 10.3390 / s23041865) proposed YOLO-S, which uses a lightweight feature extraction network based on Darknet20. Although it improves the speed of small target detection to a certain extent, the detection accuracy of the model is not high due to the limited feature extraction capability of its backbone network. Liu et al. (Liu Z, Gao X, Wan Y, et al. Animproved YOLOv5 method for small object detection in UAV capture scenes[J]. IEEE Access, 2023, 11: 14365-14374.) proposed a feature enhancement module to generate different receptive field features and combine it with spatial pyramid pooling to improve the feature extraction capability of small targets. However, the feature extraction network of this method is too deep, resulting in the loss of its own feature information.Wang et al. (Wang G, Chen Y, An P, et al. UAV-YOLOv8: asmall-object-detection model based on improved YOLOv8 for UAV aerialphotography scenarios[J]. Sensors, 2023, 23(16): 7190-7217.) proposed a feature processing module FFNB that integrates shallow and deep features, which improves the recall rate of small target detection, but the detection accuracy is still not high.

[0005] In summary, the existing research on small target detection methods still faces difficulties such as small target size, background information interference and few effective features, which makes it difficult to accurately extract target features and there is a problem of target feature loss during the sampling process. Summary of the invention

[0006] The purpose of the present invention is to provide a small target detection optimization method based on self-anti-disturbance convolution, which solves the problems of existing small target detection methods that the target is greatly disturbed by background information and the characteristics of the sampling process are easily lost.

[0007] The technical solution adopted by the present invention is a small target detection optimization method based on self-rejection convolution, which is specifically implemented according to the following steps:

[0008] Step 1: Design an anti-disturbance convolution module, use the secondary central difference to extract small target features, and combine the vanilla convolution to represent the effective background information;

[0009] Step 2: Build a feature extraction selector in the YOLOv8 backbone network and constrain the differential operation by activating the strategy function to balance the target features and background information.

[0010] Step 3: Use feature extraction selector and anti-disturbance convolution to improve the feature fusion network of YOLOv8 backbone network and neck structure respectively to obtain the optimization method.

[0011] The present invention is also characterized in that:

[0012] In step 1, the anti-disturbance convolution module divides the interference of the small target feature extraction process into internal and external factors. The internal interference is: the small size of the small target makes it difficult to accurately extract the features in the sampling; the external interference is: the small target is easily disturbed by background information, which causes the loss of the extracted features in the aggregation.

[0013] In step 1, the working principle of the self-disturbance rejection convolution module is:

[0014] First, a central difference operation is used to extract the central difference information feature map fpcd , the extracted f pcd The local receptive field of the first sampling R Feature map obtained by vanilla convolution f 0 fusion, perform secondary central difference operation; secondly, use vanilla convolution to extract global features and compare them with the enhanced target features of secondary central difference; finally, output the feature map representing small target features and global information f qcd .

[0015] In step 2, the feature extraction selector uses the vanilla convolution in the YOLOv8 benchmark model to perform convolution operations, and judges the operation results based on the activation decision. If it is higher than the IoU value set by the activation decision, all operations of this layer are terminated; otherwise, the self-anti-disturbance convolution operation is performed.

[0016] Feature Extraction Selector Selects Hyperparameters in Centered Difference Convolution θ As the decision quantifier, the IoU set by the non-maximum suppression function in the YOLO model is selected as the decision quantifier θ The decision boundary θ IoU .

[0017] The specific improvements in step 3 are:

[0018] The CSP residual structure and SPPF pooling layer in the YOLOv8 backbone network use vanilla convolution, and the two-layer convolution of the feature pyramid structure in the YOLOv8 neck network is replaced with an anti-disturbance convolution module, that is, the two vanilla convolution operations in the feature fusion downsampling are replaced with anti-disturbance convolution.

[0019] The beneficial effects of the present invention are:

[0020] (1) The method of the present invention proposes a new auto-disturbance rejection convolution (ADRConv) module, which uses the secondary central difference to extract the key features of small targets to avoid insufficient features obtained by sampling; it aggregates the target features with the global information obtained by the vanilla convolution to solve the problem of background submergence of small targets in feature extraction;

[0021] (2) The method of the present invention designs a novel feature extraction selector in the YOLOv8 backbone network, activates the self-disturbance rejection convolution or vanilla convolution through the decision boundary of the selector to avoid excessive prominence of the target features, and adds a small-scale convolution kernel to construct a convolution layer to further extract small target features; on this basis, the constructed self-disturbance rejection convolution module is used to optimize the sampling convolution in the YOLOv8 neck structure, effectively eliminating the noise generated by the fusion of feature maps of different scales, and avoiding the problem of feature loss during model training;

[0022] (3) To verify the performance of the proposed method, experiments were conducted on the SIRST dataset and the Cracktree dataset. The experimental results show that compared with the YOLOv8 benchmark algorithm and the mainstream methods in recent years, the precision of this method is improved by 4.3% and 2.6%, the recall rate is improved by 7.5% and 0.9%, and the average detection accuracy is improved by 3.9% and 1.3%, which effectively improves the detection effect of small objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall framework diagram of the method of the present invention;

[0024] Figure 2 It is a structural diagram of the auto-disturbance rejection convolution module in the method of the present invention;

[0025] Figure 3 is a network structure diagram of a feature extraction selector in the method of the present invention;

[0026] Figure 4 It is the overall structure diagram of the method of the present invention;

[0027] Figure 5 It is a visualization effect diagram of the auto-disturbance rejection convolution module in the model test in an embodiment of the present invention;

[0028] Figure 6 is the decision parameter in the feature extraction selector of the embodiment of the present invention θ IoU The corresponding indicator when the value range is from 0 to 1;

[0029] Figure 7 is the decision parameter in the feature extraction selector of the embodiment of the present invention θ IoU The corresponding index when the value range is 0.6 to 0.8;

[0030] Figure 8 is the box_loss change curve during SIRST dataset training in an embodiment of the present invention;

[0031] Fig. 9 is the box_loss variation curve during Cracktree dataset training in an embodiment of the present invention;

[0032] Fig.10 This is a comparison of the detection effects of YOLOv8 and ESOD-YOLOv8 on the SIRST and Cracktree datasets in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] The present invention is based on the small target detection optimization method of self-rejection convolution, such as Figure 1 As shown in the figure, it mainly includes target feature representation based on self-anti-disturbance convolution, feature extraction selector design and structure optimization based on YOLOv8 model, which is implemented in the following steps:

[0035] Step 1: Design an anti-disturbance convolution module, use the secondary central difference to extract small target features, and combine the vanilla convolution to represent the effective background information;

[0036] The combination of center difference convolution and ordinary convolution (such as vanilla convolution) can effectively reduce the sensitivity of face anti-spoofing (FAS) tasks to feature scale differences and contrast. Inspired by this, the present invention designs a self-anti-disturbance convolution module to effectively alleviate the interference of convolution in small target detection tasks.

[0037] The anti-disturbance convolution module of the present invention combines the central difference idea to improve the sampling and aggregation process of ordinary convolution (such as vanilla convolution). The structural design is as follows: Figure 2 shown. Figure 2 In the self-anti-disturbance convolution module, it is proposed to set the small size of small targets, which makes it difficult to accurately extract features in sampling, as internal interference; and the small targets are easily disturbed by background information, which leads to the loss of extracted features in aggregation, as external interference. The interference of small target detection is divided into the above two categories, and targeted self-anti-disturbance convolution design is carried out.

[0038] like Figure 2 As shown, the anti-disturbance convolution of the present invention first uses a central difference operation to extract the central difference information feature map f pcd , highlight the sampling center information and strengthen the small target feature information; then extract f pcd The local receptive field of the first sampling R Feature map obtained by vanilla convolution f0 fusion, a secondary central difference operation is performed to enhance the central target information twice, thereby highlighting the small target features; secondly, the vanilla convolution is used to extract the global features, which are compared with the enhanced target features of the secondary central difference to avoid ignoring the background information; finally, the feature map representing the small target features and the global information is output f qcd .

[0039] The calculation process of the secondary central difference processing in the self-anti-disturbance convolution and the fusion of the original feature map is described as formula (1):

[0040] (1)

[0041] in, t 0 represents the current position on the input and output feature maps, t n Used to enumerate local receptive fields R The current position in x ( t 0) represents the aggregate information of the initial position of t0, which is used to make a difference with other positions traversed in the receptive field to obtain the central difference information. ω ( t n )and x ( t 0+ t n ) represent the current n-th sampling information and aggregation information respectively, ω ( t n-1 )and x ( t 0+ t n-1 ) represent the sampling information and aggregation information of the n-1th time, respectively, which represent the two central difference operations and vanilla convolution operations designed in the self-disturbance-resistant convolution. For example, for a single-layer non-expanded convolution with a convolution kernel of 4×4 and a stride of 1, its local receptive field is R is 4. Except for the experimental parameter settings, the parameters involved in the method are all set using the vanilla convolution parameter settings in YOLOv8, where the n value is updated according to the specific training rounds set in the experiment.

[0042] In order to avoid increasing the amount of convolution operations, formula (1) does not introduce new hyperparameters, and after decomposition and merging, it can form two separate terms, which are Figure 2 The corresponding expression is shown in formula (2).

[0043] (2)

[0044] The first term in formula (2) can be described as the secondary central difference operation of the self-disturbance rejection convolution module, and the second term can be described as the representation of the global feature by the vanilla convolution.

[0045] Compared with the center difference convolution, the self-disturbance rejection convolution module of the present invention also introduces the center difference idea in sampling, but according to the characteristics of small targets with few features, the interference in the small target feature extraction process is divided into internal and external factors. A secondary center difference operation is used in sampling to accurately sample small target information; in the aggregation process, the global information obtained by the vanilla convolution is combined as the final output, and the interference of internal and external factors is eliminated in the two stages of sampling and aggregation. Therefore, while focusing on strengthening the features of small targets, the self-disturbance rejection convolution module avoids weakening the extraction effect of global information.

[0046] Step 2: Build a feature extraction selector in the YOLOv8 backbone network and constrain the differential operation by activating the strategy function to balance the target features and background information.

[0047] The self-disturbance rejection convolution module is mainly used to strengthen the features of small targets, and its degree of strengthening needs to be considered in the iterative training of the model to avoid excessive strengthening of the target features. Therefore, the present invention further designs a feature extraction selector to measure the degree of use of self-disturbance rejection convolution in feature extraction by the YOLOv8 backbone network.

[0048] The feature extraction selector uses the vanilla convolution in the YOLOv8 benchmark model to perform convolution operations, and judges the operation results based on the activation decision. If the IoU value is higher than the IoU value set by the activation decision, all operations of this layer are terminated. Otherwise, the self-disturbance rejection convolution module is executed. The feature extraction selector makes full use of the hyperparameters in the central difference convolution. θ Set as the decision quantity, make effective decisions on the training process without adding additional parameters, and the feature extraction selector structure is as follows Figure 3 shown.

[0049] The feature extraction selector uses the IoU set by the non-maximum suppression (NMS) function in the YOLO model as the decision metric θ The decision boundary is denoted as θ IoU In the self-anti-disturbance convolution module, the θ and the decision boundary value to determine whether additional processing is required. θ Greater than θ IoU , then the small target feature confidence has met the set target feature decision boundary, and the vanilla convolution result (Vanilla_Conv) is directly returned; otherwise, the weighted sum of the convolution kernel is calculated Center_diff , expressed as formula (3),

[0050] (3)

[0051] in, N is the number of convolution kernels, ω i represents the convolution kernel, X i It is i The output value of the convolution kernel. Then, the input feature map is convolved in two dimensions to obtain the output tensor Diff_out, which is expressed as formula (4):

[0052] (4)

[0053] in, F.conv2d (·) represents a two-dimensional convolution function; I is the input tensor, whose shape is ( n , C in , h , w ),in n is the batch size, C in is the number of input channels, h and w is the size of the input feature map; K is the convolution kernel tensor, whose shape is ( N , C in , H , W ),in, N is the batch size, H and W are the sizes of the convolution kernels respectively.

[0054] Center_diff It is the sum of all weights in the convolution kernel multiplied by the value of the corresponding region in the input tensor I, corresponding to a single element in the two-dimensional convolution output Diff_out. Finally, the output of the self-disturbance rejection convolution module is obtained D conf , as shown in formula (5),

[0055] (5)

[0056] The feature extraction selector judges the verification results after each training according to the activation decision, and determines whether it is necessary to activate the self-anti-disturbance convolution module to enhance the small target features according to the judgment result. Its selection function is as shown in formula (6):

[0057] (6)

[0058] The backbone network of YOLOv8 extracts fewer small target features in the early stage of training. θThe value is lower than the decision parameter, so under the activation decision selection, the anti-disturbance convolution module is continuously used to strengthen the target feature representation. As the number of running iterations increases, the small target features are increasingly strengthened by the anti-disturbance convolution module. If the iteration effect does not meet the decision parameter, the anti-disturbance convolution module is activated. Otherwise, the convolution operation of this layer is terminated. After that, the decision amount will continue to be higher than the decision boundary, and the feature extraction selector tends to extract global information with vanilla convolution, which is conducive to the improvement of the final detection performance.

[0059] Step 3: Use feature extraction selector and anti-disturbance convolution to improve the feature fusion network of YOLOv8 backbone network and neck structure respectively to solve the problem of feature loss caused by small target size during sampling.

[0060] The backbone network and neck network of YOLOv8 can obtain richer gradient flow information while ensuring lightweight, realize end-to-end target detection, and can also adjust different numbers of channels for different scale models to improve model performance by fine-tuning the structure. The backbone network convolution layer of YOLOv8 adopts convolution operation, BN layer normalization and function activation neuron convolution layer construction method, which can be described as formula (7):

[0061] (7)

[0062] in, ξ (·) is the activation function, bias is the offset, BN (·) is the batch normalization function, which is used to improve the stability and convergence speed of network training. Conv (·) is the convolution operation.

[0063] The method of the present invention replaces the feature extraction selector into each convolutional layer of the backbone network. The specific parameter design still uses the original convolutional layer structure and adopts the SiLU activation function. The convolutional layer of the backbone network is shown in formula (8):

[0064] (8)

[0065] In addition to replacing the convolutions of the original five convolutional layers of the backbone network with feature extraction selectors, a new convolutional layer with a convolution kernel size of 2×2 is added as the first layer to improve the extraction effect of small pixel-level objects. The vanilla convolution module is replaced with a feature extraction selector to activate the corresponding convolution operation. Among them, the parameters such as the convolution kernel, number of channels, and stride remain the same as before; based on the structural optimization of the YOLOv8 model, the specific parameters are shown in Table 1.

[0066] Table 1 Convolutional layer parameters of the backbone network

[0067]

[0068] The CSP residual structure and SPPF pooling layer in the YOLOv8 backbone network still use vanilla convolution to avoid excessive feature extraction and reduce the amount of model calculation. The convolution of the feature pyramid structure in the neck network of YOLOv8 can realize the scale transformation and feature map generation of multi-scale feature fusion. Among them, the two layers of convolution are used to adapt to the anchor frames of different sizes of the detection head. In order to avoid the problem of target feature loss in feature fusion, the two layers of convolution are replaced by the self-anti-disturbance convolution module to maintain a high degree of attention to the target information in feature fusion. Figure 1 The neck structure of describes the convolution position, that is, the two vanilla convolution operations in the feature fusion downsampling are replaced with anti-disturbance convolution. Because the detection layer continues to use the decoupled head structure of YOLOv8, the convolution parameters in the replaced anti-disturbance convolution module are consistent with the original convolution parameters. The detection layer of YOLOv8 continues to use the anchor-free detection head, and the model does not rely on predefined anchor boxes during the detection process, thereby improving the accuracy and efficiency of the detection process. Figure 4 It is the overall structure of the method of the present invention.

[0069] Embodiment 1:

[0070] The method of the present invention is a small target detection optimization method based on self-disturbance rejection convolution (Enhanced smallobject detection YOLOv8, ESOD-YOLOv8), and the method of the present invention is referred to as ESOD-YOLOv8 in the embodiments.

[0071] In order to evaluate the performance of the proposed method under different types of data sets, experiments were conducted on two widely used small target data sets, the SIRST data set and the CrackTree data set. The SIRST data set is one of the few publicly released and widely used infrared small target data sets. It is shot in a real IR scene, where half of the objects only contain 0.1% of the pixels of the entire image, challenging objects and complex backgrounds. The training set and test set in the SIRST data set are divided into a ratio of 8:2. The CrackTree data set consists of real road images, including road cracks of various sizes. Due to the small amount of available data, the algorithm only uses 120 images for training, 36 images for the verification step, and 50 images for testing. In order to make more target sizes fit the small targets, the data set is re-annotated by using the crack bifurcation nodes of the road cracks as single cracks. The experiment adopts the pre-trained weights of YOLOv8n. In order to ensure the comparative experimental effect with the mainstream method, the data set is not expanded by data augmentation. The model is retrained using the above data set based on the pre-trained model.

[0072] The proposed method was trained and tested in the environment of Intel i9-12900H CPU 2.50 GHz and NVIDIA GeForce RTX 3060 Laptop GPU16G, using GPU operation mode and CUDA version 11.1; the code environment uses Pytorch 3.8.16 and Python 3.8; the Ultralytics dependency library uses version 8.0.117. The initial learning rate lr0 of the model is 0.01, and the weight decay coefficient weight_decay is 0.0005.

[0073] The comparative experiments conducted on the two datasets were evaluated using three key indicators: precision, recall, and average precision (AP50). In order to measure the complexity of the YOLOv8 model before and after the improvement, the model parameter quantity indicator (Params) was added in the ablation experiment. The experiment used the hyperparameters configured in YOLOv8, the input image size was uniformly adjusted to 640×640, and the batch size was set to 4. According to the training results, the SIRST dataset and CrackTree training rounds were finally adjusted to 300 epochs and 500 epochs respectively.

[0074] In order to verify the effectiveness of the self-disturbance rejection convolution module and the feature extraction selector in the method of the present invention, an ablation experiment was conducted on the SIRST dataset with extremely small target size, and the results are shown in Table 2. The self-disturbance rejection convolution module focuses on strengthening the features of small targets, so it has a good improvement in the comprehensiveness of small target detection, but the continuous strengthening process of target features will introduce interference information, resulting in poor improvement in detection accuracy. The feature extraction selector determines the strengthening scale of target features through the activation strategy, which can reduce the impact of interference information and feature loss on detection accuracy. Therefore, the final improved YOLO algorithm has significant improvements in both detection accuracy and recall rate of small targets.

[0075] Table 2 Comparison of CDC, ADRConv and Feature Extraction Selector (FES) in YOLOv8 baseline

[0076]

[0077] As can be seen from Table 2, after the central difference convolution replaces the vanilla convolution, the accuracy is significantly improved due to its prominent enhancement of the target features. The self-anti-disturbance convolution module of the present invention not only strengthens the target features, but also focuses on the screening of background information, so it can detect the target information in the image more comprehensively. However, during the multi-round iterative training process, the vanilla convolution focuses on extracting global information, while the central difference highlights the strengthening of target features, so that the two have different effects on improving accuracy and recall, but the amount of parameters generated by both is reduced. In order to improve the detection performance more balanced, the introduction of FES (feature extraction selector) has achieved a more comprehensive optimization effect in the evaluation of detection performance indicators while maintaining the parameter amount of the YOLOv8 model at the level of the benchmark algorithm.

[0078] Embodiment 2:

[0079] On the basis of Example 1, in order to intuitively understand the role of each stage of the self-disturbance rejection convolution module in the feature extraction process, the trained ESOD-YOLOv8 is tested on the SIRST dataset to visualize the feature extraction effect of the self-disturbance rejection convolution module in the small target detection process. The results are as follows: Figure 5 As shown. Figure a is the original image of the test set; Figure b and Figure c are the feature maps obtained by the self-disturbance rejection convolution module before and after the first central difference. It can be seen that when the trained model is tested on the test set, the small target feature information of the feature map extracted by the first central difference is not complete; Figure d outputs the features after the fusion of Figure b and Figure c, and Figure e is a separate vanilla convolution extraction. From the visual saliency distribution, it can be seen that the target information and the background cannot be effectively distinguished; after performing a second central difference on Figure d and fusing the background information in Figure e, the final output feature map of the self-disturbance rejection convolution module is Figure f. From the effect of Figure f, it can be seen that the visualization of small target features among all features shows an obvious effect, and the best confidence score is obtained among all features.

[0080] In the design of feature extraction selectors, appropriate decision boundary values ​​can prevent the model from over-enhancing small target features while still focusing on global information. Therefore, to verify the effect of decision boundaries on the model, we use the Cracktree dataset to θ IoU When different values ​​are taken, the detection efficiency of the method of the present invention is compared, and the experimental results are as follows: Figure 6 and Figure 7 As shown in the figure θ IoU The models trained under different values ​​are tested, and the models are evaluated using two performance indicators: precision and recall. Figure 6 is the decision parameter θ IoU The corresponding indicator when the value range is 0 to 1, θ IoUWhen the value is 0, the model does not activate the anti-disturbance convolution module; θ IoU As the value increases, the anti-disturbance convolution module plays a role in model training, continuously strengthening the target features to improve the model's detection ability for small targets; but when the value is too large, the model loses the ability to judge interference information due to excessive emphasis on the target and ignoring other information.

[0081] Depend on Figure 6 visible, θ IoU When the value is around 0.7, the performance of the model is improved most significantly. Therefore, in order to further obtain a more effective decision boundary, a more detailed experiment is conducted on the performance of the model in the range of 0.6 to 0.8. The experimental results are as follows: Figure 7 As shown. θ IoU When the value is set on both sides of 0.7, although the recall rate is improved to a certain extent, the insufficient enhancement of target features and excessive introduction of background disturbances ultimately cause the detection accuracy of the model to drop significantly. θ IoU =0.7, the feature extraction selector, under the action of the decision boundary, makes the small target detection capability of the method of the present invention more efficient and balanced.

[0082] Embodiment 3:

[0083] In order to measure the difference between the predicted value and the true value of the model in the small target detection task, the bounding box loss (box_loss) is used to test the training effect of the model. The box_loss of YOLOv8 and ESOD-YOLOv8 under different data sets is compared with the change of training rounds. The experimental results are as follows: Figure 8 and Fig. 9 As shown in the figure, the box_loss change curves during training of SIRST and Cracktree datasets are shown respectively.

[0084] After the ESOD-YOLOv8 model training is stable, the box_loss is lower than that of the YOLOv8 benchmark model, so the method of the present invention has a better detection effect. Figure 8 The target features of the SIRST dataset are small. As the number of training rounds increases, the box_loss of ESOD-YOLOv8 begins to surpass the original YOLOv8. This is because the dataset has few small target features. In the selection of the feature extraction selector activation strategy, the target features are continuously strengthened in the early stage, so that they can obtain higher confidence compared with the initial image features such as background information during the extraction process, and show better prediction results in subsequent training rounds. The small target features of cracks in road cracks are relatively obvious, so Fig. 9The box_loss effect in , from the beginning of training ESOD-YOLOv8 is better than the YOLOv8 baseline model.

[0085] In order to directly reflect the optimization effect of the method of the present invention, Fig.10 The comparison results of YOLOv8 and the detection method of the present invention on two data sets are shown. Fig.10 As shown in the figure, the targets in the SIRST dataset are extremely small, so the confidence scores of small target detection are locally enlarged for comparison. It can be seen that the confidence scores for small targets are higher after improvement. In the CrackTree dataset, road crack targets are re-labeled according to nodes. In addition to making the road crack targets more consistent with the characteristics of small targets, higher requirements are placed on the accuracy of target detection. According to the comparison results, the effective prediction box of the road cracks in the method of the present invention can cover the target more completely.

[0086] In order to more comprehensively verify the effect of ESOD-YOLOv8, it is compared with the current mainstream methods of DNANet, UCFNet, UIU-Net and ResUNet-NFA using the SIRST dataset. The comparison results of each indicator are shown in Table 3 below.

[0087] Table 3 Comparative experiments on SIRST dataset

[0088]

[0089] As a baseline model, YOLOv8 has achieved relatively ideal results in small target detection of this dataset, but the improved YOLOv8 by the method of the present invention has achieved 4%, 7.5% and 3.9% improvement in precision, recall and average precision respectively. For the SOTA model for small target detection, the feature extraction effect of small targets has achieved relatively high performance, but the feature loss caused by the network structure itself causes the extracted features to be lost again during the training process. Under the interference of this factor, it is difficult for the algorithm for small target detection to achieve balanced improvement in both precision and recall. Therefore, it is slightly lower than UIUNet by 0.4% in precision, but the method of the present invention can meet more balanced detection efficiency. Regarding the performance indicator of recall, the method of the present invention has improved by 6.6% compared with the methods in recent years.

[0090] On the CrackTree dataset, the LF-YOLO, Topo-LIOT and BGF-YOLO high-efficiency models are compared, and the comparison results are shown in Table 4.

[0091] Table 4 Comparative experiments on CrackTree dataset

[0092]

[0093] The target features of the CrackTree dataset are relatively obvious, but the contrast between its road background and target is small. Therefore, in the research based on this dataset, the main focus is on reducing the interference of background information on the target. Therefore, in the YOLOv8 baseline model with better generalization ability, the detection effect achieved is slightly lower than that of BGF-YOLO. The introduction of the feature extraction selector in the method of the present invention, although slightly different from BGF-YOLO in recall rate, improves the detection accuracy by 2.6% compared with the accuracy of the YOLOv8 baseline network. This is due to the interference of background information during training, and the self-anti-disturbance convolution module is triggered multiple times in the feature extraction selector to perform secondary central difference operations, which reduces the sensitivity of the model to target features, and the improvement of target recall rate is slightly lower than that of BGF-YOLO. Therefore, the strengthening of small target features by the self-anti-disturbance convolution module in the method of the present invention, and the balance of target features and background information by the feature extraction selector are conducive to comprehensively improving the detection performance of small targets.

[0094] The present invention proposes an optimization method for small target detection based on self-rejection convolution, ESOD-YOLOv8, which further improves the small target detection performance of the model on the basis of YOLOv8. The method of the present invention designs ADRConv to further visualize the small target features in the original image; constructs a feature extraction selector to make decisions and adjust the training process of the model, reduce the feature loss of the model during the training process, and further improve the accuracy of small target detection. The experimental results show that the self-rejection convolution module and feature extraction selector in ESOD-YOLOv8 can effectively reduce the interference of background information and make the target features more prominent. The method of the present invention shows good detection effects on the SIRST dataset and CrackTree dataset with large differences in target features, and improves the precision and recall rate by 4.3%, 2.6% and 7.5%, 0.9% respectively compared with the existing models. However, the generalization effect of the method of the present invention has not been verified. In the future, we will continue to explore and improve to reduce the computational complexity of the algorithm, and verify it with datasets of target features in different fields, so as to improve the generalization of the model.

Claims

1. Small target detection optimization method based on self-rejection convolution, It is characterized in that Follow the steps below to implement it: Step 1: Design an anti-disturbance convolution module, use the secondary central difference to extract small target features, and combine the vanilla convolution to represent the effective background information; Step 2: Build a feature extraction selector in the YOLOv8 backbone network and constrain the differential operation by activating the strategy function to balance the target features and background information. Step 3, using feature extraction selector and anti-disturbance convolution to improve the feature fusion network of YOLOv8 backbone network and neck structure respectively, and obtain the optimization method; In step 1, the anti-disturbance convolution module divides the interference of the small target feature extraction process into internal and external factors, wherein the internal interference is: the small size of the small target causes the feature in the sampling to be difficult to extract accurately; the external interference is: the small target is easily disturbed by the background information, causing the loss of the extracted features in the aggregation; In step 1, the working principle of the self-disturbance rejection convolution module is: First, a central difference operation is used to extract the central difference information feature map f pcd , the extracted f pcd The local receptive field of the first sampling R Feature map obtained by vanilla convolution f 0 After fusion, a secondary central difference operation is performed; secondly, the global features are extracted using vanilla convolution and compared with the enhanced target features of the secondary central difference; Finally, the feature map representing the small target features and global information is output f qcd .

2. According to claim 1, the small target detection optimization method based on self-rejection convolution, It is characterized in that In step 2, the feature extraction selector uses the vanilla convolution in the YOLOv8 benchmark model to perform convolution operations, and judges the operation results according to the activation decision. If it is higher than the IoU value set by the activation decision, the self-disturbance rejection convolution operation is terminated; otherwise, the self-disturbance rejection convolution operation is performed.

3. According to claim 2, the small target detection optimization method based on self-rejection convolution, It is characterized in that The feature extraction selector selects the hyperparameters in the central difference convolution θ As the decision quantifier, the IoU set by the non-maximum suppression function in the YOLO model is selected as the decision quantifier θ The decision boundary θ IoU .

4. According to claim 1, the small target detection optimization method based on self-disturbance rejection convolution, It is characterized in that The specific improvements in step 3 are: The CSP residual structure and SPPF pooling layer in the YOLOv8 backbone network use vanilla convolution, and the two-layer convolution of the feature pyramid structure in the YOLOv8 neck network is replaced with an anti-disturbance convolution module, that is, the two vanilla convolution operations in the feature fusion downsampling are replaced with anti-disturbance convolution.