Small target detection method used in foreground and background confusion scene

By introducing the global context attention mechanism, separating the global context attention mechanism and residual context aggregation module in the YOLOv8 model, and replacing the step convolution module, the small object detection algorithm SRN_YOLO is improved, and small objects in the foreground background confusion scenario are solved. The problem of low detection accuracy achieves higher detection accuracy and lower error detection rate.

CN119068309APending Publication Date: 2024-12-03CHONGQING UNIV OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411167971.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing object detection algorithms are difficult to accurately detect small targets in foreground background confusing scenarios, especially because the target features are similar to background features, resulting in low detection accuracy and high error detection rate.

Method used

A modified YOLOv8 model, called SRN_YOLO, is proposed to enhance the model's attention to small goals by introducing global context attention mechanisms and separating global context attention mechanisms in the backbone network, and adding residual contexts to the neck network Aggregation module reduces background information interference, and at the same time replaces the standard step-step convolution module with a step-free step-step convolution module to reduce information loss.

Benefits of technology

While ensuring the detection speed, the accuracy of small object detection is significantly improved. Especially in the foreground background confusion scenario, the model's ability to extract small object features is enhanced, and the error detection rate and missed detection rate are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068309A_ABST
    Figure CN119068309A_ABST
Patent Text Reader

Abstract

The invention provides a small target detection method used in a foreground and background confusion scene. The method comprises the following steps: inputting a foreground and background confusion small target picture to be detected into an SRNYOLO model to obtain a small target detection result; the SRNYOLO model comprises an input network, a backbone network, a neck network and a head network, and is an improved model of a YOLOv8 model; compared with a YOLOv8 model, the improvement point of the SRNYOLO model is that a global context attention separation mechanism is added in a C2f1 module in a backbone network, and the purpose of highlighting the position of a small target in a highly similar background is achieved; a residual context aggregation module is integrated in the neck network, so that the interference of irrelevant background noise in small target feature fusion is effectively weakened; non-stride convolution is deployed in the backbone network and the neck network at the same time, and the small target detection precision is improved by minimizing feature loss in the convolution process. Small target detection of foreground and background confusion scenes can be realized, and the detection precision is improved while the model detection speed is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a small target detection method for a scene with foreground and background confusion. Background Art

[0002] Object detection is a core area of ​​current computer vision applications, locating and classifying objects of interest in images or videos. In recent years, with the rise of convolutional neural networks and improvements in computing power, convolutional neural network-based object detection technology has made significant progress. This technology has been widely applied in areas such as product defect detection in industrial manufacturing, obstacle detection in autonomous driving, and object detection in remote sensing imagery. Deep learning has achieved promising results for detecting conventional objects, but existing object detection algorithms still face challenges with small objects, particularly those in scenes with foreground and background confusion. Small objects can be defined using two methods: relative and absolute scale. The International Society for Optical Engineering (IOE) proposed a relative scale definition, which considers objects smaller than 0.12% of the original image size to be small. The concept of absolute scale, defined in the MS COCO dataset, considers objects smaller than 32*32 pixels to be small. Small objects in scenes with foreground and background confusion are characterized by small size, limited contextual information, and similarity between target and background features. This makes detection more challenging and prone to missed and false detections.

[0003] Deep learning-based object detection algorithms can be divided into two categories: single-stage and two-stage. Two-stage algorithms first generate candidate object regions and then classify and locate them using a convolutional neural network. Common two-stage algorithms include R-CNN, Fast-RCNN, and Faster-RCNN. These algorithms offer high detection accuracy but suffer from high computational overhead and slow performance. Compared to two-stage algorithms, single-stage algorithms omit the candidate bounding box generation and directly predict the object type and location through regression analysis. Common single-stage algorithms include SSD, YOLOv3, YOLOv5, and YOLOv8. Single-stage algorithms are widely used in scenarios requiring high real-time performance due to their low computational overhead and high detection speed. However, both of these algorithms perform poorly for small objects in scenes with mixed foreground and background.

[0004] In short, in industrial and remote sensing images, when the features of the small target to be detected (foreground) are similar to the features of some objects in the background of the image, it is still a great challenge to accurately detect the position of the small target. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a small target detection method for scenes with foreground and background confusion.

[0006] In order to achieve the above-mentioned object of the present invention, the present invention provides a small target detection method for a scene with foreground and background confusion, comprising:

[0007] The image of a small target to be detected with a mixed foreground and background is input into the SRN_YOLO model to obtain the detection result of the small target. The SRN_YOLO model consists of four parts: input network, backbone network, neck network, and head network. It is an improved model of the YOLOv8 model. The improvements of the SRN_YOLO model compared to the YOLOv8 model are as follows:

[0008] Add an attention mechanism in the C2f1 module in the backbone network, where the attention mechanism is either a global context (GC) attention mechanism or a separated global context (SGC) attention mechanism;

[0009] The global context attention mechanism includes the following step 2, and the separated global context attention mechanism includes the following steps 1 and 2;

[0010] Step 1: Feature map separation operation;

[0011] The purpose of the separation operation is to distinguish the parts of the feature map that contain more information from those that contain less information. The scaling factor in the group normalization (GN) layer is used to distinguish the information content of different parts of the feature map.

[0012] Given a feature map Where N represents the batch number axis, C represents the number of channels of the feature map, H and W represent the height axis and width axis of the feature map;

[0013] First, the features Figure X Perform standardized operations:

[0014]

[0015] Where μ and σ are input features Figure X The mean and standard deviation of

[0016] ε is a very small positive constant added to ensure stable operation of division;

[0017] α and β are both trainable affine transformation parameters; we mainly use the trainable parameters in the GN layer (α is a vector of length C, and each value in the vector is a real number ) is a method for measuring the spatial pixel variance of each batch and channel. The richer the spatial information of a feature map, the greater the variation in its spatial pixels, resulting in a larger value for α. We consider α to be the set of weights for each channel of a feature map, that is, the weight value of a single feature map.

[0018] Normalize α to get the normalized weight value W α :

[0019]

[0020] Among them, C represents the number of channels of the feature map;

[0021] α i represents the trainable affine transformation parameters of the i-th channel;

[0022] α j represents the trainable affine transformation parameters of the j-th channel;

[0023] Next, the input feature X is input to the GN layer for normalization; then the output of the GN layer is combined with the weight value W α Multiply, and finally, W α The product result with the output of the GN layer is mapped to (0,1) through the Sigmoid activation function; then, through the Gate gate containing the threshold, the threshold is set to 0.5, the weight above the threshold is set to 1, and the weight below the threshold is set to 0, and the information weight value ω1 is obtained; at this time, the weight above the threshold indicates that the channel in the feature map corresponding to the weight is a channel with a lot of information, so the weight is set to 1 to retain it, which helps to obtain a feature map with a lot of information in the subsequent separation; the weight above the threshold is set to 0, and the weight below the threshold is set to 1 to obtain a non-information weight value ω2; at this time, the weight below the threshold indicates that the channel in the feature map corresponding to the weight is a channel with little information, so the weight is set to 1 to retain it, which helps to obtain a feature map with a little information in the subsequent separation; ω1 and ω2 are obtained according to the following formulas:

[0024] ω i,i=1、2 = Gate(Sigmoid(W α (GN(X)))) (3)

[0025] Finally, the input feature X is multiplied by the information weight value ω1 and the non-information weight value ω2 respectively to obtain the feature with more information Figure X ω1 and less informative features Figure X ω2 ;

[0026] Step 2: Capture global contextual attention;

[0027] Since the feature map with less information contains almost no target information, this part can be regarded as redundant information of the original feature map. Figure X ω1 The global context attention mechanism is introduced to avoid the interference of redundant information in the original feature map and improve the attention of the global context attention mechanism to the target information. The global context attention mechanism mainly consists of three parts: context modeling, bottleneck transformation and broadcast element addition; the expression of the global context attention mechanism is:

[0028]

[0029] Among them, γ i Represents the output result of the i-th position in the feature map after the above formula;

[0030] X i Represents the input feature value of the i-th position of the feature map;

[0031] W k1 、W k2 Respectively represent the weight matrices used for linear transformation;

[0032] ReLU() represents the activation function;

[0033] LN() represents layer normalization;

[0034] W k represents the linear transformation matrix;

[0035] is the weight of the global attention pool;

[0036] It represents the context modeling part;

[0037] W k2 *ReLU(LN(W k1 (.))) represents the bottleneck conversion operation;

[0038] H and W represent the height and width of the input feature map;

[0039] Finally, we will go through the GC attention mechanism X ω1 Feature map and X with less information ω2 The feature maps are added together to achieve a more effective use of the global context attention mechanism to focus on the local features of the feature map without losing information.

[0040] Adding the Separated Global Context (SGC) attention mechanism to the model backbone network enhances the model backbone's attention to small objects, thereby improving its feature extraction capability.

[0041] Furthermore, the SRN_YOLO model improves upon the YOLOv8 model by adding a residual context aggregation module, i.e., an RCA module, to the neck network. The neck network takes as input the three output feature maps of the backbone network at different levels, i.e., the first-level feature map, the second-level feature map, and the third-level feature map. The neck network structure is:

[0042] The first-level feature map P3 is connected to the input end of the first Concat module of the neck network, the output end of the first Concat module is connected to the input end of the first C2f2 module, the output end of the first C2f2 module is connected to the input end of the first RCA module, the output end of the first RCA module outputs the first feature map and is connected to the input end of the first Conv module, the output end of the first Conv module is connected to the input end of the second Concat module, the output end of the second Concat module is connected to the input end of the second C2f2 module, the output end of the second C2f2 module outputs the input end of the second RCA module, the output end of the second RCA module outputs the second feature map and is connected to the input end of the second Conv module, the output end of the second Conv module and the third-level feature map P5 are connected to the input end of the third Concat module, the output end of the third Concat module is connected to the input end of the third C2f2 module, the output end of the third C2f2 module is connected to the input end of the third RCA module, and the output end of the third RCA module outputs the third feature map;

[0043] The second-level feature map P4 is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the fourth C2f2 module, the output of the fourth C2f2 module is connected to the input of the first Upsample module, and the output of the Upsample module is connected to the input of the first Concat module;

[0044] The third-level feature map P5 is connected to the input of the second Upsample module and the input of the third Concat module, and the output of the second Upsample module is connected to the input of the fourth Concat module;

[0045] The first Conv module and the second Conv module are standard strided convolution modules;

[0046] The residual context aggregation module enhances the input features by calculating attention scores, weighted feature aggregation, normalization and activation functions, and finally combines the enhanced feature map with the original feature map.

[0047] The purpose of adding layer normalization and activation functions is to enable the CABlock block to capture channel dependencies while reducing background information interference in the feature map, thereby reducing redundancy after pixel feature aggregation. The purpose of adding a residual structure outside the CABlock module is to avoid gradient vanishing.

[0048] The expression of the residual context aggregation module structure is:

[0049]

[0050] Among them, Q i Represents the input feature, which represents the feature of the i-th position;

[0051] P i Represents the output features after the input features pass through the residual context aggregation module;

[0052] δ i Represents the weighting coefficient, which is used to amplify or reduce the enhanced features;

[0053] W v1 Represents a weight matrix used to perform linear transformation on the weighted feature map;

[0054] W v2 Represents a weight matrix used to linearly transform the feature map after the activation operation;

[0055] W v Represents a weight matrix used to linearly transform the input feature map;

[0056] N i Indicates the number of features in the i-th layer, indicating how many features in the current layer need to be processed;

[0057] Q m Indicates the value corresponding to a specific feature in the i-th layer;

[0058] Q and P represent the input and output feature maps of the residual context aggregation module, respectively. Each feature map contains N i pixels;

[0059] δ i The role of is to balance the degree of context aggregation for each pixel, δ i The formula is expressed as:

[0060]

[0061] Among them, W a Represents a weight matrix, which is mainly used to perform linear transformation on the input feature Qi;

[0062] exp() is the exponential function of the natural base;

[0063] m∈{1,N i}, m represents the index of each pixel in the feature map.

[0064] Introducing the residual context aggregation module at the neck of the model can effectively reduce the impact of useless background features on the final target detection.

[0065] Furthermore, the data processing process of the RCA module is as follows:

[0066] The input feature map first undergoes three convolution operations with a convolution kernel size of 1 to obtain the first convolution feature map, the second convolution feature map, and the third convolution feature map; the weight value of the first convolution is W a , the weight values ​​of the second and third convolutions are W v ;

[0067] The first convolution feature map is normalized by sigmoid to obtain the normalized first convolution feature map, which corresponds to the parameter δ in formula 6. i ;

[0068] The third convolution feature map is normalized by softmax to obtain a weight value; the second convolution feature map is then multiplied by the weight value to obtain a context feature; the context feature is then sequentially subjected to a convolution operation, a layer normalization (LN) operation, and a RELU activation function; the activated feature map is further subjected to a convolution operation to generate a final context feature representation;

[0069] The normalized first convolution feature map is point-multiplied with the final context feature representation, and then added to the original input feature map to form a residual connection, thereby outputting the final feature map.

[0070] Furthermore, the SRN_YOLO model has the following improvements compared to the YOLOv8 model: replacing the standard strided convolution modules in the backbone network and the neck network with the non-stride convolution module (NSConv module);

[0071] The non-stride convolution module includes:

[0072] The input feature map first passes through the SPD module to halve the width and height of the feature map, and then changes the number of channels of the feature map through a 1*1 convolution operation (standard convolution); then the feature map is divided into two parts according to the number of channels, one part is operated and the other part is not operated. The operation is: first perform the Addcoords operation, then pass a 3*3 convolution operation, and then perform group convolution (GConv) and point-by-point convolution (PConv) on the convolved feature map respectively, and then combine the results of group convolution (GConv) and point-by-point convolution (PConv);

[0073] Then the feature maps of the operated part and the non-operated part are concat-connected, and finally the Channel-Shuffle operation is used to enhance the flow of information between channels and improve the feature expression capability.

[0074] Using non-stride convolution modules in the model trunk and model neck can avoid the loss of target feature information caused by traditional stride convolution modules and improve the accuracy of model detection.

[0075] Furthermore, the Addcoords operation includes:

[0076] Two coordinate channels are added to the feature map, representing the X and Y coordinates respectively. This allows the convolution process to have spatial awareness, allowing for more efficient feature acquisition.

[0077] Furthermore, the SPD module includes:

[0078] The SPD operation mainly splits the input feature map according to a certain scale value, and then connects the split feature maps; Figure X The output feature map obtained after inputting the SPD method is Where scale represents the specified ratio, scale 2 *C represents the number of channels of the output feature map, represents the height of the output feature map, Represents the width of the output feature map; C represents the number of channels of the feature map, and H and W represent the height axis of the feature map;

[0079] The SPD operation avoids the information loss caused by the stride operation in traditional convolution, helping to improve model detection accuracy. Since small objects themselves contain little information, traditional strided convolution operations will further lose this limited information. However, the SPD operation avoids the stride operation and effectively reduces the information loss of small objects.

[0080] Furthermore, the parameter quantity P of the non-stride convolution module is NSThe calculation formula is:

[0081]

[0082] Where Δ represents the split ratio;

[0083] g represents the group size in the group convolution (we set g to the number of feature channels after passing through the group convolution module, i.e. );

[0084] k represents the convolution kernel size;

[0085] C1 represents the number of channels of the input feature map;

[0086] C2 represents the number of channels of the output feature map.

[0087] In summary, due to the adoption of the above technical solution, the present invention can realize small target detection in scenes with mixed foreground and background, and improve detection accuracy while ensuring model detection speed. The specific advantages are as follows:

[0088] (1) A separated global context (SGC) attention module is designed and introduced into the backbone of the model. The SGC module first divides the incoming feature map into feature maps with high information content and feature maps with low information content, then passes the feature map with high information content into the global context attention module, and finally adds the feature map with low information content to the feature map processed by the GC attention mechanism. In this way, the GC attention mechanism can be better utilized without losing feature map information, maximizing the acquisition of small target features, thereby improving detection performance.

[0089] (2) The context aggregation module is improved to obtain the residual context aggregation module, which is added to the neck network of the model. The residual context aggregation module can effectively reduce the background information in the feature map after feature fusion, avoiding the influence of background features on subsequent small target detection.

[0090] (3) To address the information loss problem of the standard strided convolution module, a strided-free convolution module was constructed. It adds coordinate information in the X and Y axes. This allows the convolution process to avoid information loss while also maintaining spatial awareness, resulting in more efficient feature extraction. When the number of input and output channels is the same, the strided-free convolution module reduces the number of parameters by approximately 27% compared to the standard strided convolution module.

[0091] (4) We conducted multiple experiments using two datasets of small objects with realistic foreground-background confusion. These images come from different domains, including an industrial PCB dataset and a remote sensing dataset. To validate the innovative features of the SRN_YOLO model, we conducted sufficient ablation experiments. The SRN_YOLO model demonstrated superior generalization capabilities across different domains.

[0092] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0094] Figure 1 It is a schematic diagram of the network structure of YOLOv8s of the present invention.

[0095] Figure 2 It is a structural diagram of the separation of the global context attention mechanism and the global context attention mechanism of the present invention; Figure 2 (a) is the separation of global context attention mechanism; Figure 2 (b) is the global context attention mechanism.

[0096] Figure 3 It is the improved backbone network and C2f1-SGC module structure of the present invention; Figure 3 (a) is the improved backbone network. Figure 3 (b) is the improved C2f module.

[0097] Figure 4 Comparison of the original model and the heat map after adding attention (taking the leak defect as an example); Figure 4 (a) is the original image; Figure 4 (b) is the YOLOv8s heat map; Figure 4 (c) is the YOLOv8s+C2f1-GC heat map; Figure 4 (d) is the YOLOv8s+C2f1-SGC heat map.

[0098] Figure 5 It is the improved context aggregation module of the present invention; Figure 5 (a) is the original context aggregation module, Figure 5 (b) is the residual context aggregation module.

[0099] Figure 6 This is the neck network structure in which the residual context aggregation module is added in the present invention.

[0100] Figure 7It is the non-stride convolution module of the present invention; Figure 7 (a) is the structure diagram of the non-stride convolution module; Figure 7 (b) is the spath to depth operation when scale = 2; Figure 7 (c) is the Addcoords operation.

[0101] Figure 8 This is the SRN_YOLO network structure diagram of the present invention.

[0102] Figure 9 These are the six defects of the PCB dataset of the present invention: Figure 9 (a) is a leak, (b) is a rat bite, (c) is an open circuit, (d) is a short circuit, (e) is a stray, and (f) is false copper.

[0103] Figure 10 This is the PCB picture after data enhancement of the present invention: Figure 10 (a) is the original image, (b) is brightness changed, (c) is salt and pepper noise, (d) is blur, (e) is random scaling, and (f) is random rotation.

[0104] Figure 11 Here is the mAP@0.5:0.95 loss comparison between YOLOv8s and SRN_YOLO. Figure 11 (a) is mAP@0.5:0.95, Figure 11 (b) is a loss.

[0105] Figure 12 This is the test result of the SRN_YOLO model of the present invention on the PCB dataset.

[0106] Figure 13 This is the test result of SRN_YOLO on the Dior dataset. DETAILED DESCRIPTION

[0107] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0108] 1. Related technologies and their existing problems

[0109] 1.1 Traditional target detection algorithm

[0110] Traditional target detection algorithms are mainly divided into three steps: first, a sliding window is used to traverse the image to obtain candidate target regions; second, a manually designed method is used to extract target features; and finally, a classifier is used to classify the target features and output the detection results. For example, the VJ detector (Viola Jones) proposed by Viola et al. in 2001 is used for face recognition tasks. The principle of this method is to use a sliding window to scan the entire image, then use Haar features to describe the light and dark relationship of each candidate region, and finally use Adaboost to classify the extracted target features and output the face recognition results. Dalal et al. proposed a feature description method for pedestrian detection in 2005. The basic idea of ​​this method is to first divide the image into different units and calculate the directional gradient histogram of different units. Finally, the directional gradient histogram is combined into a feature map to continue target detection. Traditional target detection has two key problems: (1) The sliding window will produce a large number of areas that do not contain targets, resulting in a lot of useless calculations; (2) The model has poor robustness when facing different targets.

[0111] 1.2 Object Detection Algorithm Based on Deep Learning

[0112] Deep learning-based object detection algorithms offer significant advantages over traditional object detection. They utilize convolutional neural networks (CNNs) to extract higher-level features, improving detection accuracy while maintaining speed. Deep learning-based object detection algorithms can be categorized into two types: two-stage object detection networks and single-stage object detection networks.

[0113] The two-stage object detection algorithm first generates a series of candidate regions in the input image, uses classification and regression operations to obtain the category and coordinate information of the bounding boxes from these candidate regions, and finally uses the non-maximum suppression (NMS) method to remove redundant bounding boxes to predict the category and coordinates of each target to be detected. Zhao Z et al. proposed a remote sensing image target detection algorithm based on the Faster R-CNN architecture. This method achieved good results on two optical remote sensing datasets. To detect small defects inside 3D-printed lattice structure samples, Zhang Y et al. proposed an adaptive defect detection method based on the Faster R-CNN architecture. Experimental results show that small defects in lattice structures can be effectively detected. Although the two-stage object detection network has the advantage of high detection accuracy, it also has problems such as difficult model training, slow detection speed, and large number of model parameters.

[0114] To address the challenges of two-stage object detection algorithms, researchers have proposed single-stage object detection algorithms. Compared to two-stage object detection algorithms, single-stage object detection algorithms eliminate the time-consuming step of generating candidate regions using a region proposal network and integrate classification and localization into a single network, thereby simplifying the network architecture and improving detection speed. To address the difficulties of industrial defect detection, Qian X et al. proposed a lightweight feature fusion network based on the YOLO model architecture. This network achieved good detection results on a dataset of small industrial objects. To address the complex backgrounds and dense objects in remote sensing images, Han H et al. proposed a single-stage object detection algorithm based on an improved YOLOv5. Experimental results show that this algorithm can effectively improve object detection accuracy in remote sensing images. To achieve a balance between detection accuracy and speed, Lin J et al. proposed a fast detection algorithm based on YOLO and a decoupling head, which effectively improved the model's detection performance. Single-stage object detection algorithms have the advantages of simple model structure and fast inference speed, but their detection accuracy is slightly lower than that of two-stage object detection algorithms.

[0115] 1.3 Small Target Detection Algorithm

[0116] Due to the limited semantic information inherent in small objects, previous object detection algorithms have performed poorly. Therefore, improving the accuracy of small object detection remains a major challenge in the current field of object detection. Yang C, Huang Z, Wang N et al. proposed a simple and effective cascaded sparse query (CSQ) mechanism, which reduces the computational cost of all feature pyramid-based object detectors and transfers feature information from deep layers (small-resolution feature maps) to shallow layers, facilitating the detection of small objects. Law H et al. proposed the CornerNet algorithm, which, based on the keypoint concept, determines the position and size of an object by detecting its corners, thereby enabling the detection of small objects. The advantage of the CornerNet algorithm lies in its ability to accurately represent the position and size of an object through corners, enabling precise detection of small objects. Duan K et al. proposed the CenterNet algorithm, which directly outputs the coordinates, width, and height of the object's center point, eliminating the need for bounding box regression to calculate the object's position. However, these models perform poorly when detecting small objects in scenes with mixed foreground and background.

[0117] In summary, to improve the accuracy of small object detection in scenarios with mixed foreground and background, this paper uses a single-stage YOLOv8s as the baseline model. First, an improved global contextual attention mechanism is added to the model's backbone network, improving the network's ability to capture small object features and facilitating better object localization. Second, an improved context aggregation module is added to the model's neck network to mitigate the impact of background information introduced by the backbone network on small object localization. Finally, a custom non-strided convolution module is added to replace the standard strided convolution modules in the backbone and neck networks to avoid feature information loss caused by strided convolution.

[0118] 2. Proposed algorithm SRN_YOLO

[0119] YOLOv8 is an object detection algorithm that combines convolutional neural networks with deep learning. YOLOv8 is divided into five types, in ascending order, based on the depth and width of its network structure: YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x. This article uses YOLOv8s as the baseline model and, without using pre-trained weights, develops a small object detection algorithm suitable for scenarios with foreground-background confusion.

[0120] The network structure of YOLOv8 is as follows Figure 1 As shown in Figure 2, the YOLOv8 model consists of four main components: input, backbone, neck, and head. The input is augmented using the Mosaic method, where four images are cropped and spliced ​​at a time to increase the diversity of the dataset. The backbone network, which is responsible for acquiring image features, consists of a Conv module, a C2f module, and a SPPF module. The C2f module can adjust its channel count based on the model scale to fine-tune the model structure. The SPPF module performs max pooling on the input image using pooling kernels of sizes 5, 9, and 13, respectively, to achieve a larger receptive field. The neck network is primarily responsible for feature fusion and consists of a feature pyramid network and a pyramid attention network. The head network predicts objects, primarily using anchor-free detection methods. Compared to anchor-based detection methods, anchor-free detection methods reduce the number of anchors and speed up detection.

[0121] Although YOLOv8 is a widely used target detection algorithm, it still has problems in detecting small targets in scenes where the foreground and background are mixed. To this end, this paper proposes an improved algorithm based on YOLOv8s, which is named SRN_YOLO. The specific improvements of SRN_YOLO include: adding an attention mechanism to the C2f1 module in the backbone network, adding an improved context aggregation module to the neck network, replacing the standard strided convolution module in the backbone network and the neck network with a self-made convolution module (NSConv module) Figure 1 conv module).

[0122] 2.1 Separated Global Context (SGC) Attention Mechanism

[0123] This study introduces the SGC attention mechanism to focus on the feature information of small objects, thereby improving the detection accuracy of the model. Under the influence of the human visual perception system, the attention mechanism allows the neural network to selectively focus on the target area of ​​the input data, thereby improving the model's performance in identifying key features.

[0124] Small targets in foreground and background confusion scenarios have the characteristics of small target size, easy confusion between target features and background features, and a large number of noise points in the image. Due to the small size of the target, the conventional backbone network is very likely to lose the feature information of the small target during the multi-level transmission process. If it is directly input into the neck network for feature fusion without processing, the accuracy of small target detection will decrease. Because the attention mechanism can effectively improve the model's detection performance for small targets, this paper attempts to introduce the attention mechanism into the backbone network. According to the scaling factor in the group normalization, we can divide the feature maps extracted from the backbone network into feature maps containing more information and less information. The feature maps with less information contain almost no target information and can be considered as redundant information in the feature map. Compared with inputting the entire feature map into the attention mechanism, only inputting the feature map with more information into the attention mechanism can avoid the problem of resource waste caused by redundant information in the feature map. Removing the interference of redundant information in the feature map can also make more effective use of the attention mechanism and improve the extraction of target features by the model backbone network. To this end, this paper reconstructs the global context attention module (GC) to obtain a separate global context attention module (SGC), such as Figure 2 (a) shows that the Separated Global Context (SGC) attention mechanism specifically includes two steps: separating feature maps and capturing global context attention.

[0125] 1) Feature map separation operation

[0126] In the SGC module, the purpose of the separation operation is to separate feature maps with more information from feature maps with less information. We mainly use the scaling factor in the group normalization (GN) layer to distinguish the information content in different feature maps. For example, given a feature map Where N represents the batch number axis, C represents the number of channels of the feature map, H and W represent the height axis and width axis of the feature map. Figure X Perform the standardization operation as follows:

[0127]

[0128] Where μ and σ are input features Figure X , ε is a very small positive constant added to ensure stable operation of division, and α and β are trainable affine transformation parameters.

[0129] We mainly use the trainable parameters in the GN layer (α is a vector of length C, and each value in the vector is a real number ) is a method for measuring the spatial pixel variance of each batch and channel. The richer the spatial information of the feature map, the more changes in its spatial pixels, which makes the value of α larger. Normalized correlation weight W α ∈R C It is obtained by formula 2, which specifically represents the importance of different feature maps.

[0130]

[0131] Among them, C represents the number of channels of the feature map;

[0132] α i Represents the trainable parameters of the i-th channel

[0133] α j represents the trainable parameters of the j-th channel;

[0134] Next, we will go through W α The weight values ​​of the reweighted feature maps are input into the sigmoid function to map them between (0, 1). The weights above the threshold are set to 1, and the weights below the threshold are set to 0, thereby obtaining the information weight value ω1; the weights below the threshold are set to 1, and the weights above the threshold are set to 0, thereby obtaining the non-information weight value ω2 (the threshold is set to 0.5 in the experiment). According to Formula 3, ω1 and ω2 are obtained:

[0135] ω i,i=1、2 = Gate(Sigmoid(W α (GN(X)))) (3)

[0136] Finally, the input feature X is multiplied by the information weight value ω1 and the non-information weight value ω2 respectively, so as to obtain two weighted feature maps: the feature with more information Figure X ω1 and less informative features Figure X ω2 .

[0137] 2) Capturing global contextual attention

[0138] After the feature map separation operation, we get features with a lot of information Figure X ω1 Features with less information Figure X ω2 . Since the feature map with less information contains almost no target information, this part can be regarded as redundant information of the original feature map. Only inputting the feature map with more information into the global context attention mechanism can avoid the interference of redundant information in the original feature map and improve the attention of the global context attention mechanism to the target information. The global context attention mechanism mainly consists of three parts: context modeling, bottleneck conversion and broadcast element addition. The structure of the global context attention mechanism is as follows Figure 2 (b), the formula is expressed as:

[0139]

[0140] Among them, γ i Represents the output result of the i-th position in the feature map after the above formula;

[0141] X i Represents the input feature value of the i-th position of the feature map;

[0142] W k1 、W k2 Respectively represent the weight matrices used for linear transformation;

[0143] ReLU() represents the activation function;

[0144] LN() represents layer normalization;

[0145] W k represents the linear transformation matrix, is the weight of the global attention pool, It represents the context modeling part, W k2 *ReLU(LN(W k1 (.))) represents the bottleneck transformation operation, and H and W represent the height and width of the input feature map.

[0146] Finally, we will go through the GC attention mechanism X ω1 Feature map and X with less information ω2The feature maps are added together, which constitutes the entire process of the SGC attention mechanism. This allows for more effective use of the global context attention mechanism to focus on local features of the feature map without losing information.

[0147] Since the YOLOv8 backbone network is mostly composed of stacked convolutional modules, it cannot distinguish small targets in scenes with mixed foreground and background. In order to enhance the backbone network's ability to extract features of small targets, we use the SGC attention mechanism to enhance the C2f1 module in the backbone network. Figure 3 As shown in (a), for Bottleneck1 in the C2f1 module, we replace it with Bottleneck1-SGC that incorporates the SGC attention mechanism. Because only the feature maps output by the 2nd, 3rd, and 4th C2f1 modules in the backbone network are fed into the neck network, we only replace these three C2f1 modules with C2f-SGC modules, as shown in Figure 3 (b) We integrated the original global context attention mechanism and the separated global context attention mechanism into the YOLOv8s algorithm and compared their respective accuracies, as shown in Table 1. We found that the SGC attention module can effectively improve the mAP@50.95 indicator. Figure 4 Heatmap results for the original model, the model with global contextual attention, and the model with separate global contextual attention are shown. In the heatmap, the higher the overlap between darker areas and small objects, the higher the attention paid to them. The results show that heatmaps using the SGC module have wider coverage of small objects and higher attention to them than heatmaps using the GC module.

[0148] Table 1 Comparison of YOLOv8s with GC attention mechanism and SGC attention mechanism

[0149]

[0150] 2.2 Residual Context Aggregation Module

[0151] Previous studies have found that in images where the target only covers a small area, the backbone network will extract useless background information while extracting the features of the small target. In the YOLOv8s model, the main function of the neck network is to fuse the features of the shallow feature layer extracted by the backbone network with the features of the deep feature layer, but the feature map after the fusion of the neck network contains a lot of background information. When the target features are similar to the background features, these background information will affect the accuracy of the final target detection. In this regard, this paper is inspired by the context aggregation module and constructs a new context aggregation module, named the residual context aggregation module. The context aggregation module is as follows: Figure 5(a). The CABlock module can effectively fuse local and global features while reducing the influence of background information. Compared with the CABlock module, the residual context aggregation module adds layer normalization, activation function, and residual structure. The residual context aggregation module enhances the input features by calculating attention scores, weighted feature aggregation, normalization and activation functions, and finally combines the enhanced feature map with the original feature map. The purpose of adding layer normalization and activation functions is to enable the CABlock block to capture channel dependencies while reducing the interference of background information in the feature map, thereby reducing the redundancy after pixel feature aggregation. The purpose of adding a residual structure outside the CABlock module is to avoid the disappearance of gradients. The structure of the residual context aggregation module is as follows Figure 5 (b), the formula is expressed as:

[0152]

[0153] Among them, Q i Represents the input feature, which represents the feature of the i-th position;

[0154] P i Represents the output features after the input features pass through the residual context aggregation module;

[0155] δ i Represents the weighting coefficient, which is used to amplify or reduce the enhanced features;

[0156] N i Indicates the number of features in the i-th layer, indicating how many features in the current layer need to be processed;

[0157] Q m Indicates the value corresponding to a specific feature in the i-th layer.

[0158] Q and P represent the input and output feature maps of the residual context aggregation module, respectively. Each feature map contains N i pixels. i,m∈{1,N i} represents the index of each pixel in the feature map (index i mainly represents the feature position currently being processed; index m is used to traverse all other positions associated with position i to calculate the attention weight). v , W v1 and W v2 Represents the linear transformation matrix (with W in Formula 4 v1 and W v2 Both represent linear transformation matrices, but the specific values ​​are different). In the experiment, we use 1*1 convolution to represent the linear transformation matrix. LN represents layer normalization, and ReLU is the activation function. δ i The role of is to balance the degree of context aggregation for each pixel.i The formula is expressed as:

[0159]

[0160] Among them, W a Represents a weight matrix, which is mainly used to perform linear transformation on the input feature Qi;

[0161] exp() is the exponential function of the natural base;

[0162] m∈{1,2,…,N i}, m represents the index of each pixel in the feature map.

[0163] like Figure 6 As shown in Figure 1, a residual context aggregation module is embedded in the neck network of the model. This module reduces the impact of background information while also reducing the redundancy of pixel feature aggregation in the neck, effectively improving the model's detection accuracy for small objects. We integrated the original and residual context aggregation modules into the YOLOv8s algorithm and compared their respective accuracy, as shown in Table 1. We found that the residual context aggregation module is more effective in improving the model's detection accuracy than the original context aggregation module.

[0164] Table 1 Comparison of YOLOv8s with the context aggregation (CA) module and the residual context (RCA) aggregation module

[0165]

[0166] 2.3 Unstride Convolution Module

[0167] In convolutional neural networks, both the backbone network and the neck network are constructed through standard strided convolution modules ( Figure 1 Conv in ) to obtain local features of the image. However, we found that the extensive use of standard strided convolution modules not only increases the number of parameters and computational complexity of the model as a whole, but also causes information loss due to the striding operation. To this end, this paper creates a new convolution module to solve the information loss problem caused by standard strided convolution. The convolution module created in this paper is named No Stride Convolution. Figure 7(a) is the structural diagram of the NSConv module. The specific process of NSConv is as follows: the input feature map first passes through the SPD module to halve its width and height. Then, a 1x1 convolution operation (standard convolution) changes the number of channels in the feature map. The feature map is then split into two parts according to the number of channels. One part undergoes operations such as Addcoords, while the other part does not. The two feature maps are then concatenated. Finally, a Channel-Shuffle operation is performed to enhance the flow of information between channels and improve feature expression capabilities.

[0168] The structure of the Space-to-depth (SPD) method in the NSConv module is as follows Figure 7 (b) shows this method. This method is to split the feature map according to the specified ratio, and then connect these sub-feature maps according to the channel dimension. For example, a feature map is Where C represents the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map. The SPD operation mainly splits the input feature map according to a certain scale value and then connects the split feature maps. The output feature map obtained by inputting X into the SPD method is Where scale represents the specified ratio, scale 2 *C represents the number of channels of the output feature map, represents the height of the output feature map, Indicates the width of the output feature map. The SPD operation can avoid the information loss caused by the stride operation of the traditional convolution, which helps to improve the detection accuracy of the model. Since the small target itself contains a small amount of information, the traditional stride convolution operation will cause the already small amount of information to be further lost, while the SPD operation avoids the stride operation and effectively reduces the information loss of small targets. The NSConv module also adopts the idea of ​​Pconv in FasterNet, which believes that there is a high degree of similarity between different channels of the feature map, so only some channels in the input feature map can be convolved. The NSConv module divides the feature map into two sub-feature maps, and the number of channels of each sub-feature map is only half of the number of channels of the original output feature map. The Addcoords operation in the NSConv module mainly adds two coordinate channels to the feature map. The two channels represent the X coordinate information and the Y coordinate information respectively, such as Figure 7(c). The purpose of this is to make the convolution process spatially aware, thereby acquiring features more efficiently. Due to the influence of sparse connections, although grouped convolution has fewer parameters and less computation, it loses information exchange between channel groups. Since point-by-point convolution can compensate for the information loss problem caused by grouped convolution, this paper combines grouped convolution (GConv) and point-by-point convolution (PConv) to reduce computational costs while obtaining better feature information. Channel-Shuffle is mainly used to enhance feature exchange and information transfer, thereby improving model performance and generalization capabilities.

[0169] Standard strided convolution can be used with Y = C k X represents the input feature map, Y represents the output feature map, and C k Represents the convolution operation, k is the size of the convolution kernel. The parameter calculation formula for standard convolution is:

[0170] P s =k*k*C1*C2=k 2 *C1*C2 (7)

[0171] Where k is the convolution kernel size, C1 is the number of input feature map channels, and C2 is the number of output feature map channels.

[0172] The parameter calculation formula of the NSConv module is:

[0173]

[0174] Where Δ represents the split ratio, g represents the group size in the group convolution (we set g to the number of feature channels after passing through the group convolution module, i.e. ), k represents the convolution kernel size, C1 represents the number of channels of the input feature map, and C2 represents the number of channels of the output feature map. We show the advantages of NSConv by comparing it with the labeled convolution. In the experiment, when When k=3 and C1=C2, This indicates that NSConv uses 28% fewer parameters than standard convolution. We integrated the strided convolution module into the YOLOv8s algorithm and compared it with the original YOLOv8s algorithm, as shown in Table 1. We found that the introduction of the strided convolution module reduced both the number of parameters and the computational complexity of the model, while also improving model detection accuracy.

[0175] Table 1 Comparison of YOLOv8s and non-strided convolution modules

[0176]

[0177] 2.4SRN_YOLO Network Structure

[0178] Figure 8 This is our improved model architecture. We introduced the SGC attention mechanism into the YOLOv8s backbone network, enhancing its ability to capture small object features. Adding a residual context aggregation module to the neck network not only reduces the impact of background information introduced by the backbone network but also reduces redundancy after pixel feature fusion. Replacing the standard strided convolution modules in the backbone and neck networks with NSConv convolution modules avoids information loss caused by standard strided convolution.

[0179] 3. Experimental Analysis and Discussion

[0180] In order to further illustrate the technical effects of this application, experiments and analyses were conducted, as follows.

[0181] 3.1 Dataset

[0182] The effectiveness of SRN_YOLO in detecting small targets in the presence of foreground and background confusion is evaluated based on two real datasets, including the Peking University PCB defect dataset and the Dior optical remote sensing dataset.

[0183] The defect types of the PCB defect dataset include leaky holes, rat bites, open circuits, short circuits, burrs and fake copper, with a total of 693 images. Figure 9 The images corresponding to these six defect types are shown. In order to better reflect the feature of foreground and background confusion in the data, this paper uses the PCB defect dataset that has been enhanced by Chongqing University of Science and Technology. The data enhancement methods include motion blur, random scaling, cropping, random brightness and darkness, salt and pepper noise, and random rotation. The enhanced images are shown in the figure below. Figure 10 As shown in the figure, there are 4158 images in total. The training set, validation set, and test set are divided into 8:1:1 ratios. The final training set contains 3326 images, and the validation set and test set each contain 416 images.

[0184] The Dior dataset is an optical remote sensing image dataset that also suffers from foreground-background confusion. It contains 23,463 images covering 20 different objects and a total of 192,472 instances. The training, validation, and test sets are divided in a 6:2:2 ratio, resulting in a training set of 14,077 images, a validation set of 4,694 images, and a test set of 4,692 images.

[0185] 3.2 Experimental Setup and Evaluation Indicators

[0186] This article uses the deep learning framework PyTorch 1.13.0 on Linux, using an NVIDIA GeForce RTX 3060 12GB GPU server and the CUDA 11.7 toolkit for GPU acceleration. Python version is 3.7.13.

[0187] To train the model in this article, we used the stochastic gradient descent (SGD) optimizer with momentum and weight set to 0.937 and 0.0005, respectively. The learning rate of the model was set to 0.0105. To obtain the best hyperparameter values, we set the input image size to 640*640 and the batch size to 32. Without using pre-trained weights, we set the training epoch number to 1000, and to conserve computational resources during training, we set the number of early stopping rounds to 50.

[0188] This article uses precision (p), recall (R) and mean average precision (mAP) as key indicators for evaluating model performance. Precision is mainly used to measure the accuracy of the detection model (can also be defined as the positive predictive value), and recall is mainly used to measure the comprehensiveness of the detection model (can also be defined as sensitivity). Single-class precision is mainly calculated using the integral method, considering the area enclosed by the precision-recall curve and the axis. mAp is calculated by summing the individual class accuracies and dividing it by the number of categories. mAP@0.5 means calculating at the IOU intersection of 0.5. The calculation formulas for each parameter indicator are as follows:

[0189]

[0190] Ap=∫0 1 P(r)dr (12)

[0191]

[0192] Where A and B represent the set of predicted bounding boxes and true bounding boxes. TP (true positive), FP (false positive) and FN (false negative) represent the number of correct predictions, incorrect predictions and missed targets respectively. P(r) represents the precision-recall curve after smoothing operation, and the area enclosed by the curve and the axis is obtained by integrating the curve. N represents the number of categories to be detected. Ap i represents the accuracy of the i-th class, where i is the index. IOU represents the closeness of the predicted bounding box to the true bounding box. mAP@0.5:0.95 represents the average mAP across different IOU thresholds (thresholds range from 0.5 to 0.95, with a step size of 0.05).

[0193] 3.3 Model Training

[0194] In order to verify the improved SRN_YOLO algorithm, this paper uses the PCB dataset to train YOLOv8s and SRN_YOLO models in the above experimental environment. Figure 11 (a) shows that the mAP@0.5:0.95 of the YOLOv8s model stabilizes at 0.751 after 540 rounds and reaches 0.762 after the final model converges. In contrast, the mAP@0.5:0.95 of SRN_YOLO reaches 0.751 after 320 rounds and reaches 0.796 after the model finally converges.

[0195] The overall loss comparison between YOLOv8s and SRN_YOLO is as follows Figure 11 As shown in (b), YOLOv8s dropped to 8.66 in 304 rounds and finally converged to 8.03. SRN_YOLO dropped to 8.66 in 222 rounds and finally converged to 7.30.

[0196] Compared with YOLOv8s, SRN_YOLO not only has a significant improvement in accuracy, but also requires fewer training rounds with the same loss value.

[0197] 3.4 Ablation Experiment

[0198] To demonstrate the effectiveness of the improved approach proposed in this paper, we designed the ablation experiments shown in Table 1. Scheme 1 represents the original YOLOv8s model; Scheme 2 adds the SGC attention mechanism to the C2f module of the backbone network; Scheme 3 builds on Scheme 2 by adding a residual context aggregation module to the model neck; and Scheme 4 uses a custom unstride convolution module based on Scheme 3. The experimental results of these four ablation experiments are shown in Table 2.

[0199] Table 1 Ablation experimental plan

[0200]

[0201]

[0202] Table 2 Ablation experiment results

[0203]

[0204] The results of Solution 2 in Table 2 show that adding SGC attention to the C2f module in the model backbone improves mAP@0.5:0.95 by 2.4%. This demonstrates that the C2f module with the SGC attention mechanism can more accurately extract the detailed features of small objects, bringing the predicted box closer to the ground-truth box, thereby improving the accuracy of the model detection.

[0205] The results of Solution 3 in Table 2 show that adding a residual context aggregation module to the neck of the model can effectively reduce the background features introduced by the backbone network, allowing the model to focus more on the effective area. Compared with Solution 2, Solution 3 improves all indicators.

[0206] The results of Solution 4 in Table 2 show that by converting the strided convolutions in the model backbone and neck to custom non-strided convolutions, the feature information loss caused by strided convolutions can be avoided. Compared with Solution 3, Solution 4 achieves a 2% improvement in mAP@0.5:0.95.

[0207] Ultimately, we found that the SRN_YOLO model achieved improvements of 0.5%, 0.7%, 0.2%, 3%, and 4.6% in precision (P), recall (R), mAP@0.5, mAP@0.75, and mAP@0.5:0.95, respectively, compared to the YOLOv8s model. These experimental results demonstrate that the improved model can better detect small objects in scenes with mixed foreground and background.

[0208] 3.5 Comparison with SOTA Methods on Various Datasets

[0209] In order to evaluate the detection performance of the SRN_YOLO model for small targets in foreground-background confusion scenarios, we conducted experiments on the PCB dataset and the Dior dataset respectively.

[0210] On the PCB dataset, we first selected two models that performed well on the PCB dataset, one model that performed well on a remote sensing satellite dataset, and TPH-YOLO, which achieved excellent results on a small drone target dataset, for comparative experiments. The experimental results of the five algorithms on the data-augmented PCB dataset are shown in Table 3. Next, we compared SRN_YOLO with advanced object detection algorithms such as YOLOv3, YOLOv5, and YOLOv8. The comparative experimental results are shown in Table 4.

[0211] Table 3 Comparison of SRN_YOLO with other advanced small target detection algorithms

[0212]

[0213]

[0214] Table 4 Comparison of advanced target detection algorithms

[0215]

[0216] We used the same model to conduct comparative experiments on the Dior dataset. The experimental results are shown in Table 5.

[0217] Table 5 Comparison of advanced target detection algorithms on the Dior dataset

[0218]

[0219] 3.6 Experimental Results Analysis

[0220] According to the experimental results in Table 3, we can find that compared with other small target detection models in foreground-background confusion scenarios, all evaluation indicators of SRN_YOLO reach the highest level in the PCB dataset after data augmentation. This shows that the SRN_YOLO model can more effectively locate small targets in foreground-background confusion scenarios and achieve the best detection accuracy in all defect types, which further demonstrates that SRN_YOLO has excellent detection performance for small targets in foreground-background confusion scenarios.

[0221] Table 4 shows the comparison results of SRN_YOLO with several other mainstream models. Experimental results show that SRN_YOLO achieves precision (P), recall (R), mAP@0.5, and mAP@0.5:0.95 values ​​of 99.3%, 96.7%, 98.7%, and 81.5%, respectively, achieving the highest performance compared to other models. Compared to the original YOLOv8s, these evaluation metrics increased by 0.5%, 0.7%, 0.2%, and 4.6%, respectively.

[0222] To demonstrate the generalization capability of the SRN_YOLO model, we also conducted comparative experiments using the same model on the Dior dataset. The results show that the SRN_YOLO model also achieves the best detection results on the Dior dataset. Comparing Table 3 with Table 5, we find that some models only achieve good detection results on one dataset and lack strong generalization capabilities. Comparative experiments on the PCB and Dior datasets show that SRN_YOLO achieves excellent detection performance for small targets in scenarios with mixed foreground and background, and demonstrates strong generalization capabilities.

[0223] 3.7 Visualization of detection results

[0224] Figure 12 and Figure 13 The detection results of SRN_YOLO on PCB and Dior datasets are shown. Figure 12As shown in the figure, SRN_YOLO can accurately locate and classify defects in PCB images. Compared to the entire image, the defects in the PCB image are relatively small, and the defect features are similar to the background features. However, SRN_YOLO still accurately predicts the defect locations. This is mainly because the SGC attention mechanism enables the model backbone to better capture the feature information of small objects, and the residual context aggregation module reduces the influence of background information on detection. Figure 13 This paper mainly demonstrates the detection effect of SRN_YOLO on the Dior dataset. The targets in the Dior dataset and the targets in the PCB dataset are both small in size and easily confused with foreground and background features. Therefore, SRN_YOLO shows the same excellent detection effect.

[0225] In summary, the proposed SRN_YOLO model is used to improve small target detection in scenarios with foreground-background confusion. By introducing three novel modules, including SGC, RCA, and NSConv, the proposed SRN_YOLO can effectively improve the accuracy of small target detection in scenarios with foreground-background confusion. Based on the PCB and Dior datasets, the SRN_YOLO model demonstrates its strong generalization ability for detecting small targets in scenarios with foreground-background confusion.

[0226] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A small target detection method for scenes with mixed foreground and background. It is characterized in that include: The image of a small target to be detected with a confused foreground and background is input into the SRN_YOLO model to obtain the detection result of the small target; the SRN_YOLO model includes four parts: input network, backbone network, neck network and head network, which is an improved model of the YOLOv8 model; the improvements of the SRN_YOLO model compared with the YOLOv8 model are as follows: Add an attention mechanism in the C2f1 module in the backbone network, wherein the attention mechanism is a global context attention mechanism or a separated global context attention mechanism; The global context attention mechanism includes the following step 2, and the separated global context attention mechanism includes the following steps 1 and 2; Step 1: feature map separation operation; Given a feature map Where N represents the batch number axis, C represents the number of channels of the feature map, and H and W represent the height axis and width axis of the feature map; First, standardize the feature map X: Where μ and σ are the mean and standard deviation of the input feature map X; ε is a very small positive constant added to ensure stable operation of division; Both α and β are trainable affine transformation parameters; α is regarded as the weight value of a single feature map; Normalize α to get the normalized weight value W α : Among them, C represents the number of channels of the feature map; α i represents the trainable affine transformation parameters of the i-th channel; α j represents the trainable affine transformation parameters of the jth channel; Next, the input feature X is input to the GN layer for normalization; then the output of the GN layer is combined with the weight value W α Multiply, and finally, W α The product result with the output of the GN layer is mapped to (0,1) through the Sigmoid activation function; then the threshold is set to 0.5 through the Gate gate containing the threshold, the weight above the threshold is set to 1, and the weight below the threshold is set to 0, and the information weight value ω is obtained. 1 ; Set the weights above the threshold to 0, and the weights below the threshold to 1, and get the non-information weight value ω 2 ;ω 1 ,ω 2 Obtained according to the following formula: oh i,i=1、2 =Gate(Sigmoid(W α (GN(X)))) (3) Then, the input feature X is respectively compared with the information weight value ω 1 and the non-information weight ω 2 Multiply to obtain a feature map with more information and feature maps with less information Step 2: Capture global contextual attention; Only feature maps with high information content Enter the global context attention mechanism, which consists of three parts: context modeling, bottleneck transformation, and broadcast element addition; the expression of the global context attention mechanism is: Among them, γ i Represents the output result of the i-th position in the feature map after the above formula; X i Represents the input feature value of the i-th position of the feature map; W k1 , W k2 Respectively represent the weight matrices used for linear transformation; ReLU() represents the activation function; LN() means layer normalization; W k represents a linear transformation matrix; is the weight of the global attention pool; It represents the context modeling part; W k2 *ReLU(LN(W k1 (.))) represents the bottleneck conversion operation; H and W represent the height and width of the input feature map; Finally, we will go through the GC attention mechanism Feature map and less information The feature maps are added.

2. According to claim 1, a small target detection method for a scene with foreground and background confusion, It is characterized in that The improvements of the SRN_YOLO model compared to the YOLOv8 model also include: adding a residual context aggregation module, namely, an RCA module, to the neck network; the input of the neck network is three output feature maps of different levels of the backbone network, namely, the first-level feature map, the second-level feature map, and the third-level feature map; at this time, the structure of the neck network is: The first-level feature map P3 is connected to the input end of the first Concat module of the neck network, the output end of the first Concat module is connected to the input end of the first C2f2 module, the output end of the first C2f2 module is connected to the input end of the first RCA module, the output end of the first RCA module outputs the first feature map, and is connected to the input end of the first Conv module, the output end of the first Conv module is connected to the input end of the second Concat module, the output end of the second Concat module is connected to the input end of the second C2f2 module, the output end of the second C2f2 module outputs the input end of the second RCA module, the output end of the second RCA module outputs the second feature map, and is connected to the input end of the second Conv module, the output end of the second Conv module and the third-level feature map P5 are connected to the input end of the third Concat module, the output end of the third Concat module is connected to the input end of the third C2f2 module, the output end of the third C2f2 module is connected to the input end of the third RCA module, and the output end of the third RCA module outputs the third feature map; The second-level feature map P4 is connected to the input end of the fourth Concat module, the output end of the fourth Concat module is connected to the input end of the fourth C2f2 module, the output end of the fourth C2f2 module is connected to the input end of the first Upsample module, and the output end of the Upsample module is connected to the input end of the first Concat module; The third level feature map P5 is connected to the input end of the second Upsample module and the input end of the third Concat module, and the output end of the second Upsample module is connected to the input end of the fourth Concat module; The first Conv module and the second Conv module are standard strided convolution modules; The residual context aggregation module enhances the input features by calculating attention scores, weighted feature aggregation, normalization and activation functions, and finally combines the enhanced feature map with the original feature map; The expression of the residual context aggregation module structure is: Among them, Q i Represents the input feature, which represents the feature of the i-th position; P i Represents the output features after the input features pass through the residual context aggregation module; δ i Represents the weighting coefficient, which is used to amplify or reduce the enhanced features; W v1 Represents a weight matrix used to perform linear transformation on the weighted feature map; W v2 Represents a weight matrix used to linearly transform the feature map after the activation operation; W v Represents a weight matrix used to linearly transform the input feature map; N i Indicates the number of features in the i-th layer, indicating how many features need to be processed in the current layer; Q m Indicates the value corresponding to a specific feature in the i-th layer; Q and P represent the input and output feature maps of the residual context aggregation module, respectively. Each feature map contains N i pixels; δ i The role of is to balance the degree of aggregation context of each pixel, δ i The formula is expressed as: Among them, W a Represents a weight matrix, which is mainly used to perform linear transformation on the input feature Qi; exp() is the exponential function of the natural base; m∈{1,2,…,N i }, m represents the index of each pixel in the feature map.

3. According to the small target detection method for a foreground-background confusion scene as described in claim 1, It is characterized in that The data processing process of the RCA module is as follows: The input feature map is first subjected to three convolution operations with a convolution kernel size of 1 to obtain the first convolution feature map, the second convolution feature map, and the third convolution feature map; the weight value of the first convolution is W a , the weight values ​​of the second and third convolutions are W v ; The first convolution feature map is normalized by sigmoid to obtain a normalized first convolution feature map; The third convolution feature map is normalized by softmax to obtain the weight value; Then the second convolution feature map is multiplied by the weight value to obtain the context feature; The context features are then sequentially subjected to convolution operations, layer normalization operations, and RELU activation functions; the activated feature map is then subjected to convolution operations to generate the final context feature representation; The normalized first convolutional feature map is point-multiplied with the final context feature representation, and then added to the original input feature map to form a residual connection, thereby outputting the final feature map.

4. According to claim 1, a small target detection method for a scene with foreground and background confusion, It is characterized in that The improvements of the SRN_YOLO model compared to the YOLOv8 model also include: replacing the standard strided convolution modules in the backbone network and the neck network with non-strided convolution modules; The non-stride convolution module includes: The input feature map first passes through the SPD module to halve the width and height of the feature map, and then changes the number of channels of the feature map through a 1*1 convolution operation; then the feature map is divided into two parts according to the number of channels, one part is operated, and the other part is not operated. The operation is: first perform an Addcoords operation, then pass a 3*3 convolution operation, and then perform group convolution and point-by-point convolution on the convolved feature map, respectively, and then combine the results of group convolution and point-by-point convolution; Then the feature maps of the operated part and the non-operated part are concat-connected, and finally the Channel-Shuffle operation is performed.

5. According to claim 4, a small target detection method for a scene with foreground and background confusion, It is characterized in that The Addcoords operation includes: Two coordinate channels are added to the feature map, and the two channels represent the X coordinate information and the Y coordinate information respectively.

6. A small target detection method for a foreground-background confusion scene according to claim 4, It is characterized in that The SPD module comprises: The SPD operation mainly splits the input feature map according to a certain scale value, and then connects the split feature maps; the output feature map obtained by inputting the feature map X into the SPD method is Where scale represents the specified ratio, scale 2 *C represents the number of channels of the output feature map, represents the height of the output feature map, Represents the width of the output feature map; C represents the number of channels of the feature map, and H and W represent the height axis of the feature map.

7. The small target detection method for a foreground-background confusion scene according to claim 4, It is characterized in that The number of parameters P of the non-strided convolution module NS The calculation formula is: Where Δ represents the split ratio; g represents the group size in grouped convolution; k represents the convolution kernel size; C 1 Indicates the number of channels of the input feature map; C 2 Indicates the number of channels of the output feature map.

Citation Information

Cited By

  • Method and device for detecting target visual element in image and medium

    CN122023780A

  • Methods, equipment, and media for detecting target visual elements in images.

    CN122023780B