A lightweight remote sensing image detection method based on YOLOv4

Abstract

A lightweight remote sensing image detection algorithm based on YOLOv4, GSC-YOLO, is constructed using the YOLOv4 network as the basic model to construct an improved network model, GSC-YOLO. Ghostnet is used as the feature extraction network of GSC-YOLO, and Ghost convolution replaces the ordinary convolution of the enhanced feature extraction network and prediction head to improve detection speed. The SA-A attention mechanism is introduced to enable the model to focus on important information and improve detection accuracy. The confidence propagation clustering algorithm CP-Cluster is used to process the prediction box to improve remote sensing target recognition. The processed DOTA dataset is input into the GSC-YOLO model for iterative training. The test set samples are input into the trained GSC-YOLO model to output the results. The present invention can ensure detection accuracy, improve detection speed, reduce model complexity, and achieve fast and accurate detection of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image detection. Specifically, a lightweight remote sensing image detection method based on YOLOv4 is designed for the case where remote sensing image detection speed is slow and model calculation complexity is high. The method is suitable for remote sensing image detection. Background Art

[0002] The primary task of object detection in optical remote sensing images is to locate and classify objects of interest within them. This approach has widespread applications in military investigations, disaster relief, and urban traffic management. Object detection in remote sensing images requires not only accuracy, but also speed, number of parameters, and model size. In practical applications, such as military investigations, disaster prediction, and port management, both for civilian and military purposes, rapid detection is crucial while maintaining high accuracy. Deep learning-based object detection algorithms utilize deeper networks to enhance feature extraction and improve detection accuracy. However, they often suffer from large model sizes, making them difficult to deploy on resource-constrained devices. Object detection in remote sensing images often occurs on devices with limited memory and computing power, such as drones and satellites. Therefore, model compression is necessary. However, model compression can lead to a certain degree of missed detections and false detections, resulting in reduced accuracy. Therefore, improving detection speed and reducing model size while maintaining accuracy is a key challenge in remote sensing image detection. Summary of the Invention

[0003] To address these issues, we designed a lightweight object detection algorithm, GSC-YOLO, based on YOLOv4, to achieve fast and accurate detection of remote sensing images. First, we adopted Ghostnet as the feature extraction network for GSC-YOLO, and replaced all standard convolutions in the enhanced feature extraction network and prediction head with Ghost convolutions, reducing the number of parameters and improving detection speed. Secondly, we introduced a permutation attention mechanism in the prediction head, focusing the model on important information and improving detection accuracy. Finally, we used the confidence propagation clustering algorithm, CP-Cluster, to post-process the predicted boxes and enhance target recognition.

[0004] The present invention is achieved through the following technical solution: a lightweight remote sensing image detection method based on YOLOv4, comprising the following steps:

[0005] Step 1: Obtain a dataset of remote sensing images; extract experimental data from the DOTA dataset, which contains 15 categories of aerial images. Crop the images to a size of 600*600, and then process the dataset into the VOC format for the detection framework to read.

[0006] Step 2: Using the YOLOv4 network as the basic model, construct the GSC-YOLO model.

[0007] 2.1: Use Ghostnet as the feature extraction network to extract the original features of the image, and use Ghost convolution to replace all ordinary convolutions except the feature extraction network;

[0008] First, resize the input image to 608×608×3 and feed it into Ghostnet;

[0009] Secondly, the feature maps after the third, fourth, and fifth residual blocks are taken out and sent to the enhanced feature extraction network. The feature map sizes are 76×76×40, 38×38×112, and 19×19×160 respectively.

[0010] Finally, Ghost convolution is used to replace all ordinary convolutions in the enhanced feature extraction network and prediction head;

[0011] Ghost convolution first uses ordinary convolution to perform channel compression to generate a feature map A without redundant information, and then generates the final feature map B by performing feature mapping and simple convolution operation on A.

[0012] The parameter ratio of traditional convolution and Ghost convolution is:

[0013]

[0014] In the formula, the numerator is traditional convolution, and the denominator is Ghost convolution; n and c are the number of input and output feature map channels, respectively; k is the convolution kernel size; d is the convolution kernel size for cheap operations; s is the number of transformations; where c>>s; the size of d is basically the same as the size of k. From the formula, it can be concluded that the Ghost module significantly reduces the number of parameters compared to traditional convolution.

[0015] 2.2: Use the SPP structure in the enhanced feature extraction network to perform pooling operations at different scales; use PANet to fuse feature maps of different scales from the backbone network.

[0016] 2.3: Introduce the SA-A attention mechanism in the detection head;

[0017] Input a feature map X, first divide it into G groups on the channel, X = [X1, ..., XG], where X K is a sub-feature; then the sub-feature is divided into two parts X in the channel dimension k1 、X K2 , X k1 Divide into X in the channel dimension k11 、X K12The two parts are respectively sent to the channel attention module using global average pooling and global maximum pooling to perform global average pooling and global maximum pooling to generate channel statistics; the obtained channel statistics are transformed, and after the transformation function and Sigmoid activation, two feature maps X′ are obtained. K11 , X′ K12 , the two feature maps X′ K11 , X′ K12 Stacking in the channel dimension, we get the sub-feature map X′ after attention. K1 , the channel attention calculation process is as follows:

[0018]

[0019] X′ K12 =σ(F c (F GMP (X K12 )))*X K12 =σ(W 12 *max(X K12 )+b2)*X K12

[0020] X′ K1 =CONAT(X′ K11 , X′ K12 )

[0021] X K2 Send it to the spatial attention module, first obtain spatial statistics through group normalization, activate the spatial statistics transformation function and Sigmoid, and multiply it with the input feature map to obtain the sub-feature map X′ after attention. K2 , the spatial attention calculation process is as follows:

[0022] X′ K2 =σ(W2*GN(X K2 )+b2)*X K2

[0023] For the two feature maps X′ K1 , X′ K2 , stacked on the channel dimension to form a new sub-feature map X K , all sub-feature maps X K Aggregation is performed, and finally the Channel Shuffle operator is used to realize the flow of cross-group information along the channel dimension. The final output feature map has the same size as the input feature map X.

[0024] 2.4: CP-Cluster is used instead of NMS-based method to post-process the prediction box;

[0025] 2.4.1) Generate Markov random field graph: For the target detection model, let B = {b1, b2, b3...} be the original bounding box set output by the model before post-processing. For each pair of boxes (b i , b j ∈B), if their IOU is greater than θ, draw an undirected edge between them and generate a set of Markov random field graphs G = {g1, g2...}; for each subgraph g i ∈G, Defined as an edge set, Defined as a node set, for a box Its adjacent node set Included in g n All with b i For connected nodes, the number of subgraphs in G is the same as the number of prediction boxes;

[0026] 2.4.2) Generate positive message: For the bounding box Neighbors with low confidence His neighbor The subset of its IOU(b i ,b j )>θ n And for each Its confidence level P(b j ) <P(b i );

[0027] The enhancement of a bounding box is mainly affected by two factors: ① the number of its weaker neighbors; ② the confidence values of the weaker neighbors; therefore, the bounding box b i Generated positive message M p (i) The calculation process is:

[0028]

[0029] Where Q is b i The number of weaker neighbors, (1-P(b i )) is a normalization term that ensures that P(b i ) is not greater than 1.0;

[0030] 2.4.3) Generate negative message: Given a bounding box Neighbor set with high confidence yes A subset of where IOU(b i , b j )>θ and P(b j )>P(b i ); for each If the bounding box's strong neighbor set is not empty, it will be replaced by neighbors with higher confidence Suppress, utilize to b i Negative impact factor To choose which bounding box to suppress b i ; The calculation process is:

[0031]

[0032] When α=1.0, select On the contrary, if α=0.0, then Select the one with the largest IOU (b i ,b j )’s nearest strong neighbor; finally generates negative message M n (i), M n The calculation process of (i) is as follows:

[0033]

[0034] Among them SUP j，i Used to limit b i B j The number of suppressions, 2 is the maximum number of suppressions, the frame with the largest negative impact factor will be selected to penalize b i ;

[0035] 2.4.4) Update confidence: Using positive message M p (i) and negative message M n (i) to update the confidence P(b i ), and update the IOU threshold for the next iteration. As the number of iterations increases, the low-confidence box will gradually become smaller than the IOU threshold, and the high-confidence box will be retained. P(b i ) is calculated as follows:

[0036] P(b i )←P(b i )+M p (i)-Mn(i).

[0037] Step 3: Input the processed DOTA dataset into the GSC-YOLO network model and perform iterative training until the network converges.

[0038] Step 4: Use the test set samples to input the trained GSC-YOLO network model, output the results, and view the detected remote sensing image effect diagram.

[0039] The present invention achieves the following beneficial effects: Using the YOLOv4 network as the basic model, the present invention constructs an improved network model, GSC-YOLO. To address the YOLOv4 model's large number of parameters, high computational cost, and poor real-time performance, Ghostnet is adopted as the feature extraction network for GSC-YOLO. Ghost convolution replaces the standard convolutions in the enhanced feature extraction network and prediction head with convolutions to reduce parameters and improve the detection speed of remote sensing images. While Ghostnet improves detection speed due to reduced network parameters and a shallower model, detection accuracy is sacrificed. The SA-A attention mechanism is introduced into the prediction head, allowing the model to focus on important information and improve remote sensing target detection accuracy. The confidence propagation clustering algorithm (CP-Cluster) is used to post-process the predicted boxes, enhancing remote sensing target recognition. The processed remote sensing image dataset (DOTA) is input into the GSC-YOLO model, and iterative training is performed until the network converges. The trained GSC-YOLO model is then fed with test set samples, outputting the results and displaying the detected renderings. The present invention improves detection speed and reduces model complexity while ensuring detection accuracy, thereby achieving rapid and accurate detection of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of an implementation of a lightweight remote sensing image detection method based on YOLOv4 in an example of the present invention;

[0041] Figure 2 Schematic diagram of the Ghost convolution structure of the present invention;

[0042] Figure 3 Schematic diagram of the SA-A attention mechanism of the present invention;

[0043] Figure 4 Schematic diagram of the CP-Cluster structure of an example of the present invention;

[0044] Figure 5 This is a detection framework diagram of a lightweight remote sensing image detection method based on YOLOv4 in an example of the present invention;

[0045] Figure 6 This is a comparison chart of the detection results of a lightweight remote sensing image detection method based on YOLOv4 in an example of the present invention. DETAILED DESCRIPTION

[0046] Example 1:

[0047] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0048] The present invention provides a lightweight remote sensing image detection method based on YOLOv4. The specific implementation flow chart of the detection is shown in Figure 1 , specifically including:

[0049] Step 1: Obtain a dataset of remote sensing images. The experimental dataset used in this paper is the DOTA dataset, published by Wuhan University. The DOTA dataset contains 2,806 aerial images from 15 categories. Because the image sizes range from 800*800 to 4,000*4,000 pixels, the aspect ratio is too large to be suitable for direct training. Therefore, the images are preprocessed. The images are cropped into 30,087 images of 600*600 pixels each. The dataset is then converted to the VOC format for easy access by the detection framework.

[0050] Step 2: Using the YOLOv4 network as the basic model, construct the GSC-YOLO model.

[0051] 2.1: Use Ghostnet as the feature extraction network to extract the original features of the image, and use Ghost convolution to replace all ordinary convolutions except the feature extraction network to improve the detection speed of remote sensing targets.

[0052] Ghostnet is used as the feature extraction network for GSC-YOLO, and Ghost convolution replaces standard convolutions in the enhanced feature extraction network and prediction head to reduce parameters and improve detection speed for remote sensing images. While Ghostnet improves detection speed due to fewer network parameters and a shallower model, it sacrifices accuracy. The SA-A attention mechanism is introduced in the prediction head to focus on important information, improving the accuracy of remote sensing target detection. CP-Cluster, a confidence propagation clustering algorithm, is also used to post-process the predicted boxes, enhancing remote sensing target recognition.

[0053] Furthermore, Ghost convolution is used to replace all ordinary convolutions in the enhanced feature extraction network and prediction head, see Figure 2 , the Ghost convolution specifically includes:

[0054] Ghost convolution first uses ordinary convolution to perform channel compression to generate a feature map A without redundant information, and then generates the final feature map B by feature mapping and simple convolution operation (3*3 or 5*5 depth-separable convolution) on A.

[0055] The parameter ratio of traditional convolution and Ghost convolution is:

[0056]

[0057] In the formula, the numerator is traditional convolution, and the denominator is Ghost convolution. n and c are the number of input and output feature map channels, respectively. k is the convolution kernel size, d is the size of the convolution kernel for low-cost operations, and s is the number of transformations. Here, c>>s. The size of d is essentially the same as the size of k. The formula shows that the Ghost module significantly reduces the number of parameters compared to traditional convolution.

[0058] 2.2: The SPP structure is used in the enhanced feature extraction network to perform pooling operations at different scales to increase the receptive field and better capture the contextual information of the target. Furthermore, PANet is used to fuse feature maps of different scales from the backbone network, thereby enhancing the representation ability and detection accuracy of the detection model.

[0059] 2.3: Introducing the SA-A attention mechanism in the detection head allows the model to focus on important information and improve detection accuracy.

[0060] See also Figure 3 , the SA-A attention mechanism specifically includes:

[0061] Input a feature map X, first divide it into G groups on the channel, X=[X1,......,XG]. K is a sub-feature. Then the sub-feature is divided into two parts in the channel dimension X k1 、X K2 . k1 Divide into X in the channel dimension k11 、X K12 The two parts are respectively sent to the channel attention module using global average pooling and global maximum pooling to generate channel statistics. The obtained channel statistics are then transformed, and after the transformation function and Sigmoid activation, two feature maps X′ are obtained. K11 , X′ K12 The two feature maps X′ are obtained K11 , X′ K12 Stack them in the channel dimension to obtain the sub-feature map X after attention. The channel attention calculation process is as follows:

[0062]

[0063] K' K12 =σ(F c (F GMP (X K12 )))*X K12 =σ(W 12 *max(X K12 )+b2)*X K12

[0064] X′ K1=CONAT(X′ K11 , K′ K12 )

[0065] X K2 Send it to the spatial attention module, first obtain spatial statistics through group normalization, activate the spatial statistics transformation function and Sigmoid, and multiply it with the input feature map to obtain the sub-feature map X′ after attention. K2 The spatial attention calculation process is as follows:

[0066] X′ K2 =σ(W2*GN(X K2 )+b2)*X K2

[0067] For the two feature maps X′ K1 , X′ K2 , stacked on the channel dimension to form a new sub-feature map X K . All sub-feature maps X K Aggregation is performed, and finally the Channel Shuffle operator is used to realize the flow of cross-group information along the channel dimension. The final output feature map has the same size as the input feature map X.

[0068] 2.4: CP-Cluster is used instead of NMS-based methods to post-process the prediction boxes to achieve higher accuracy, while achieving parallelization and improving detection speed.

[0069] See also Figure 4 , the CP-Cluster specifically includes:

[0070] (1) Generate a Markov random field graph. For the target detection model, assume that B = {b1, b2, b3...} is the original bounding box set output by the model before post-processing. For each pair of boxes (b i , b j ∈B), if their IOU is greater than θ, an undirected edge is drawn between them to generate a set of Markov random field graphs G = {g1, g2...}. For each subgraph g i ∈G, Defined as an edge set, Defined as a node set. For a box Its adjacent node set Included in g n All with b i The number of subgraphs in G is the same as the number of prediction boxes.

[0071] (2) Generate positive messages. For the bounding box Its weaker neighbor set His neighbor The subset of its IOU(b i , b j )>θ n And for each Its confidence level P(b j )<P(b i ). Specifically, the enhancement of the bounding box is mainly affected by two factors: ① the number of its weaker neighbors. ② the confidence value of the weaker neighbors. Therefore, the bounding box b i Generated positive message M P (i) The calculation process is:

[0072]

[0073] Where Q is b i The number of weaker neighbors, (1-P(b i )) is a normalization term to ensure that P(b i ) is not greater than 1.0.

[0074] (3) Generate negative messages. Given a bounding box Its strong neighbor set yes A subset of where IOU(b i , b j )>θ and P(b j )>P(b i ). For each If the bounding box's strong neighbor set is not empty, it will be replaced by its stronger neighbors Inhibit. Utilize to b i Negative impact factor To choose which bounding box to suppress b i .

[0075] The calculation process is:

[0076]

[0077] When α=1.0, select On the contrary, if α=0.0, then Select the one with the largest IOU (b i , b j )’s nearest strong neighbor. Finally, a negative message M is generated. n (i). M n The calculation process of (i) is as follows:

[0078]

[0079] Among them SUP j，i Used to limit b i B j The number of suppressions, 2 is the maximum number of suppressions, the frame with the largest negative impact factor will be selected to penalize b i .

[0080] (4) Update the confidence. Using the positive message M P (i) and negative message M n (i) to update the confidence P(b i ), and update the IOU threshold for the next iteration. As the number of iterations increases, the low-confidence box will gradually become smaller than the IOU threshold, and the high-confidence box will be retained. P(b i ) is calculated as follows:

[0081] P(b i )←P(b i )+M p (i)-Mn(i).

[0082] Step 3: Input the processed DOTA dataset into the GSC-YOLO network model and perform iterative training until the network converges.

[0083] Step 4: Use the test set samples to input the trained GSC-YOLO network model, output the results, and view the detected remote sensing image effect diagram.

[0084] Comparison of detection results of YOLOv4's lightweight remote sensing image detection method, see Figure 6 , where the first column shows the prediction results of YOLOv4, and the second column shows the prediction results of the algorithm proposed in this paper, GSC-YOLO. As shown in Figure a, when detecting dense small targets, the YOLOv4 algorithm may miss detections, while the GSC-YOLO algorithm detects all targets, and the model is more sensitive to small targets. As shown in Figure b, when detecting a target (harbor) covered by dense small targets (ship), YOLOv4 has the disadvantage of low target confidence. GSC-YOLO is more effective in this case, with target confidences of 0.95 and 0.97 for the harbor, respectively. As shown in Figure c, when identifying targets in complex backgrounds, the YOLOv4 algorithm fails to fully identify the bridge, which is similar to the background, while the algorithm proposed in this paper fully identifies the targets that are similar to the background. GSC-YOLO has better detection accuracy in complex backgrounds.

[0085] In this embodiment, experimental data and an analysis of a lightweight remote sensing image detection method based on YOLOv4 are provided:

[0086] The experimental dataset used in this paper is the DOTA dataset, published by Wuhan University. The DOTA dataset contains 2,806 aerial images from 15 categories. Because the images range in size from 800*800 to 4,000*4,000 pixels, their aspect ratio is too large to be suitable for direct training. Therefore, the images were preprocessed. The images were cropped into 30,087 images of 600*600 pixels each, and then converted to the VOC format for easy access by the detection framework.

[0087] The experiments in this paper use mean average precision (mAP) to evaluate model detection accuracy. mAP stands for mean average detection precision; a higher mAP value indicates better detection accuracy. Frames per second (FPS) is used to evaluate detection speed, and model size and number of parameters are used to evaluate model lightweightness. Model detection accuracy.

[0088] Table 1: Ablation experiment results

[0089]

[0090] Table 1 shows that using Ghostnet as the feature extraction network significantly improves detection speed, reaching 55 frames per second, approximately 1.77 times that of the YOLOv4 algorithm. Introducing the permutation attention mechanism in the prediction head improves detection accuracy by 1.12%, but reduces detection speed by 1 frame per second. Using CP-Cluster for post-processing the predicted boxes improves both detection accuracy and speed while maintaining the same model size. Applying all three methods to the improved YOLOv4 method improves all aspects of the model.

[0091] Table 2: Comparative experimental results

[0092]

[0093] Table 2 shows that the proposed lightweight remote sensing image detection method based on YOLOv4 performs best in detection accuracy, speed, model parameter count, and size compared to mainstream object detection algorithms: SSD, YOLOv3, and Faster RCNN. While the GSC-YOLO algorithm slightly improves on YOLOv5s in parameter count and model size, it achieves a 9.87% improvement in detection accuracy, which is crucial for real-time object detection. Furthermore, the GSC-YOLO algorithm achieves a detection speed of 58 frames per second, significantly improving detection accuracy while maintaining speed.

Claims

1. A lightweight remote sensing image detection method based on YOLOv4, characterized in that: The following steps are involved: Step 1: Obtain a dataset of remote sensing images; Step 2: Using the YOLOv4 network as the basic model, construct the GSC-YOLO model; 2.1: Use Ghostnet as the feature extraction network to extract the original features of the image, and use Ghost convolution to replace all ordinary convolutions except the feature extraction network; First, resize the input image to 608×608×3 and feed it into Ghostnet; Secondly, the feature maps after the third, fourth, and fifth residual blocks are taken out and sent to the enhanced feature extraction network. The feature map sizes are 76×76×40, 38×38×112, and 19×19×160 respectively. Finally, Ghost convolution is used to replace all ordinary convolutions in the enhanced feature extraction network and prediction head; Ghost convolution first uses ordinary convolution to perform channel compression to generate a feature map A without redundant information, and then generates the final feature map B by performing feature mapping and simple convolution operation on A. The parameter ratio of traditional convolution and Ghost convolution is: In the formula, the numerator is the traditional convolution, and the denominator is the Ghost convolution; n and c are the number of input and output feature map channels respectively; k is the convolution kernel size; d is the convolution kernel size for cheap operations; s is the number of transformations; where c>>s; the size of d is basically the same as the size of k. From the formula, it can be concluded that the Ghost module significantly reduces the number of parameters compared to traditional convolution; 2.2: Use the SPP structure in the enhanced feature extraction network to perform pooling operations at different scales; use PANet to fuse feature maps of different scales from the backbone network; 2.3: Introduce the SA-A attention mechanism in the detection head; 2.4: CP-Cluster is used instead of NMS-based method to post-process the prediction box; Step 3: Input the processed DOTA dataset into the GSC-YOLO network model and perform iterative training until the network converges; Step 4: Use the test set samples to input the trained GSC-YOLO network model, output the results, and view the detected remote sensing image effect diagram.

2. A lightweight remote sensing image detection method based on YOLOv4 according to claim 1, characterized in that: In step 1, the specific method is: extract experimental data from the DOTA dataset, which contains 15 categories of aerial images, crop the images to a size of 600*600, and then process the dataset into VOC format for reading by the detection framework.

3. The lightweight remote sensing image detection method based on YOLOv4 according to claim 1, characterized in that: In the step 2.3, the specific method is: Input a feature map X, first divide it into G groups on the channel, X = [X1, ..., XG], where X K is a sub-feature; Divide the sub-features into two parts X in the channel dimension k1 、X K2 , X k1 Divide into X in the channel dimension k11 、X K12 The two parts are respectively sent to the channel attention module using global average pooling and global maximum pooling to perform global average pooling and global maximum pooling to generate channel statistics; The obtained channel statistics are transformed, and after the transformation function and Sigmoid activation, two feature maps X′ are obtained. K11 , X′ K12 , the two feature maps X′ K11 , X′ K12 Stacking in the channel dimension, we get the sub-feature map X′ after attention. K1 , the channel attention calculation process is as follows: X′ K12 =σ(F c (F GMP (X K12 )))*X K12 =σ(W 12 *max(X K12 )+b2)*X K12 X′ K1 =CONAT(X′ K11 ,X′ K12 ) X K2 Send it to the spatial attention module, first obtain spatial statistics through group normalization, activate the spatial statistics transformation function and Sigmoid, and multiply it with the input feature map to obtain the sub-feature map X′ after attention. K2 , the spatial attention calculation process is as follows: X′ K2 =σ(W2*GN(X K2 )+b2)*X K2 For the two feature maps X′ K1 , X′ K2 , stacked on the channel dimension to form a new sub-feature map X K , all sub-feature maps X K Aggregation is performed, and finally the Channel Shuffle operator is used to realize the flow of cross-group information along the channel dimension. The final output feature map has the same size as the input feature map X.

4. The lightweight remote sensing image detection method based on YOLOv4 according to claim 1, characterized in that: In the step 2.4, the specific method is: 2.4.1) Generate Markov random field graph: For the target detection model, let B = {b1, b2, b3...} be the original bounding box set output by the model before post-processing. For each pair of boxes (b i , b j ∈B), if their IOU is greater than θ, draw an undirected edge between them and generate a set of Markov random field graphs G = {g1, g2...}; for each subgraph g i ∈G, Defined as an edge set, Defined as a node set, for a box Its adjacent node set Included in g n All with b i For connected nodes, the number of subgraphs in G is the same as the number of prediction boxes; 2.4.2) Generate positive message: For the bounding box Neighbors with low confidence His neighbor The subset of its IOU(b i , b j )>θ n And for each Its confidence level P(b j )<P(b i ); The enhancement of a bounding box is mainly affected by two factors: ① the number of its weaker neighbors; ② the confidence values of the weaker neighbors; therefore, the bounding box b i Generated positive message M p (i) The calculation process is: Where Q is b i The number of weaker neighbors, (1-P(b i )) is a normalization term that ensures that P(b i ) is not greater than 1.0; 2.4.3) Generate negative message: Given a bounding box Neighbor set with high confidence yes A subset of where IOU(b i , b j )>θ and P(b j )>P(b i ); for each If the bounding box's strong neighbor set is not empty, it will be replaced by neighbors with higher confidence Suppress, utilize to b i Negative impact factor To choose which bounding box to suppress b i ; The calculation process is: When α=1.0, select On the contrary, if α=0.0, then Select the one with the largest IOU (b i , b j )’s nearest strong neighbor; Finally, a negative message M is generated n (i), M n The calculation process of (i) is as follows: Among them SUP j，i Used to limit b i B j The number of suppressions, 2 is the maximum number of suppressions, the frame with the largest negative impact factor will be selected to penalize b i ; 2.4.4) Update confidence: Using positive message M P (i) and negative message M n (i) to update the confidence P(b i ), and update the IOU threshold for the next iteration. As the number of iterations increases, the low-confidence box will gradually become smaller than the IOU threshold, and the high-confidence box will be retained. P(b i ) is calculated as follows: P(b i )←P(b i )+M P (i)-Mn(i)。

Images