Optical remote sensing image small target identification method based on PSN-YOLO model

Abstract

The invention provides an optical remote sensing image small target identification method based on a PSN-YOLO model, and aims to solve the problems of complex background and low multi-scale detection precision of an optical remote sensing image caused by ground feature diversity. Firstly, a 160 * 160 small target detection head is added to a Head part of a YOLOv7 model, and the multi-scale detection capability of the model is improved. Then, a mixed attention module is embedded in a feature extraction module of the Backbone part to establish a remote channel dependency relationship improved YOLOv7 model, a convolution module for inhibiting feature redundancy is combined in a feature extraction module of the Neck part, on the basis of improving the detection precision, the parameter quantity and the calculation quantity of the model are not increased, and the detection accuracy is improved. And then a mixed loss function is used to reduce the situation of missing detection, and the robustness of the model is enhanced.

Description

Technical Field

[0001] The present invention relates to an optical remote sensing image detection and recognition method, in particular to an optical remote sensing image small target recognition method based on a PSN-YOLO (PSA-SCConv-NWD with YOLO) model, belonging to the fields of computer vision and pattern recognition. Background Art

[0002] Optical remote sensing images contain rich details and can intuitively reflect the shape, color, texture, and other characteristics of ground objects. Using high-resolution remote sensing images increases the computational load of the target detection network and reduces detection speed. Using low-resolution remote sensing images results in low target detection accuracy due to the blurring and loss of details and features.

[0003] In this context, there is an urgent need for a target detection network with high detection accuracy and good real-time detection performance. Although the learning ability of deep neural networks can be continuously improved with the increase of network layers, greatly improving the accuracy of detection and recognition, their high computational load, high parameter storage cost, and complex model make them unable to meet the requirements of real-time detection of remote sensing images. Arbitrarily reducing the number of network layers and pruning detection heads to lightweight the network structure will seriously affect the accuracy of detection and recognition. Summary of the Invention

[0004] This paper proposes a small target detection method in optical remote sensing images based on the PSN-YOLO model. The method mainly includes four parts: multi-scale detection capability optimization based on the detection head, feature extraction optimization based on the efficient convolution module, model optimization based on the attention mechanism, and a hybrid loss function based on the intersection-over-union ratio. It achieves accurate recognition of small targets in optical remote sensing images and improves detection accuracy without increasing network parameters and computational complexity.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions.

[0006] A small target detection method for optical remote sensing images based on the PAN-YOLO model is designed based on the YOLOv7 model and includes the following steps:

[0007] Step 1, data preparation: data conversion format and data division;

[0008] Step 2: Model Improvement: First, a 160×160 small object detection head is added. Then, the attention module that can establish long-range channel dependencies and the efficient convolution module that suppresses feature redundancy are integrated into the feature extraction modules of the Backbone and Neck parts of the YOLOv7 model, respectively. The improved network model is obtained, which includes the following sub-steps:

[0009] Step 2.1: First, add a 160×160 small object detection head to the head part of the YOLOv7 model, and then introduce the feature extraction module from the backbone part;

[0010] Step 2.2: First, use the attention module to integrate into the feature extraction module of the Backbone part of the YOLOv7 model;

[0011] Step 2.3: Based on step 2.2, replace the standard convolution in the feature extraction module with depth-separable convolution.

[0012] Step 2.4: Based on steps 2.2 and 2.3, convolution with suppressed feature redundancy is fused into the feature extraction module of the Neck part of the YOLOv7 model.

[0013] Step 3, model training: For the improved YOLOv7 model, iteratively train the model using training data and validation data until the recognition performance is optimal;

[0014] Step 4, model optimization: Use the hybrid loss function based on intersection-over-union to improve the YOLOv7 original position loss function and optimize the model described in step 3;

[0015] Step 5: Adjust the hybrid hyperparameters after model optimization: Verify the impact of different hybrid hyperparameters on network performance based on experimental results;

[0016] Step 6: Model testing: Use the test set to input the trained model for testing, and detect all the categories and objects of different scales marked in the dataset;

[0017] Step 7, model output: Output the model with optimal parameters after training to a *.pt file to facilitate the model to be ported to embedded devices;

[0018] The above-mentioned small target detection method in optical remote sensing images based on the PSN-YOLO model, in step 2.2, uses the Squeeze and Concat (SPC) module to obtain channel-level multi-scale feature maps, and uses the Squeeze and Excitation (SE) module to extract the attention of feature maps of different scales to obtain channel attention vectors, and then uses Softmax to recalibrate the channel-level attention vectors to obtain the weights of multi-scale channels, and finally, the element-by-element product operation recalibrates the weights and obtains the corresponding feature maps.

[0019] In the above-mentioned method for detecting small targets in optical remote sensing images based on the PSN-YOLO model, in step 2.4, the spatial reconstruction unit (SRU) and the channel reconstruction unit (CRU) are placed in sequence. Specifically, for the intermediate input feature X in the bottleneck residual block, the spatial refinement feature X is first obtained by the SRU operation. w , and then use the CRU operation to obtain the channel-refined feature Y. This utilizes the spatial and channel redundancy between features to improve the expression ability of CNN.

[0020] The above-mentioned optical remote sensing image small target detection method based on the PSN-YOLO model, in step 4, as shown in formula L NWD =1-NWD(N p ,N g ), the Gaussian distribution of the Normalized GaussianWasserstein Distance (NWD) is introduced to calculate the loss function L NWD ;

[0021] Among them, N p is the Gaussian distribution of the prediction box, N g is the Gaussian distribution of the real box GT, NWD(N p ,N g ) is the distance between the two Gaussian distributions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0023] Figure 1 It is a framework diagram of the present invention.

[0024] Figure 2 Modeling long-range channel dependencies for attention.

[0025] Figure 3 Flowchart for suppressing spatial redundancy. DETAILED DESCRIPTION

[0026] This paper, based primarily on the YOLOv7 automatic detection and recognition model, provides a method for detecting small targets in optical remote sensing images based on the PSN-YOLO model. To further improve recognition accuracy without increasing the number of parameters or computational complexity, this method improves detection accuracy by 3.2% compared to the YOLOv7 model, while maintaining comparable computational complexity and parameter requirements. It can process one optical remote sensing image every 8.3 milliseconds and achieve a recognition accuracy of 99%.

[0027] Figure 1 This is a flowchart of the present invention, and the specific implementation of the present invention will be described below.

[0028] Step 1, data preparation: dataset format conversion and dataset division;

[0029] Step 1.1: Use the annotated public optical remote sensing dataset to convert the annotated file into the Yolo file format.

[0030] In step 1.2, the remote sensing images and their annotation information are divided into training set, validation set and test set, accounting for 70%, 10% and 20% respectively.

[0031] Step 2: Establish an attention module for long-range channel dependencies and a convolution module for suppressing feature redundancy, and embed them into the YOLOv7 model to obtain an improved network model. This includes the following sub-steps:

[0032] Step 2.1: For each detection branch of the YOLOv7 model, the feature map X∈R is output C×H×W , first use the compression and merging (SPC) module to convolution kernel k i and the i-th group size G i Perform convolution operation to obtain channel-level multi-scale feature map F i , as shown in formula (1); then use the compression and extraction (SE) module to extract the attention of feature maps of different scales to obtain the channel attention vector Z i , as shown in formula (2); then use Softmax to recalibrate the channel-level attention vector to obtain the weight atti of the multi-scale channel, as shown in formula (3); finally, the element-by-element product operation recalibrates the weight and obtains the corresponding feature map Y i , as shown in formula (4). The specific process is as follows Figure 2 shown.

[0033] F i =Conv(k i ×k i ,G i )(X) i=0,1,2···S-1 (1)

[0034] Z i =SEWeight(F i ) i=0,1,2···S-1 (2)

[0035]

[0036] Step 2.2: Given an intermediate feature map X, perform group normalization (GN) on X using formula (5) to obtain the standardized input feature X out, where μ and σ are the mean and standard deviation of X, ξ is a small positive constant added for division stability, and γ and β are trainable affine transformations. Then pass W γ The re-weighted feature map is mapped to the (0, 1) range through the sigmoid function and gated by the threshold, so that the weight W is obtained by formula (6) to complete the operation of the spatial reconstruction unit. The channel reconstruction unit is divided into two steps. In the previous stage, the input feature map X up After M G and That is, the learnable weight matrices of GWC (group-wise convolution) and PWC (point-wise convolution) are adjusted and merged to obtain the output feature map Y1 of the previous stage, so that the previous stage conversion extracts rich representative features Y1 with less computational cost, which is expressed as formula (7). The input feature map X of the next stage low go through (PWC’s learnable weight matrix) connects the generated and reused features as the output Y2, expressed as formula (8).

[0037]

[0038] W=Gate(Sigmoid(W γ (GN(X)))) (6)

[0039]

[0040] Step 3: For the improved YOLOv7 model, use the training set to train the model and update the parameters. After each round of training, use the validation set to verify the recognition accuracy of the model. This process is iterated for 200 rounds. When the loss function remains stable and decreases discontinuously for multiple times, the iteration is terminated and the optimal parameter model is saved to *.pt;

[0041] Step 4: Based on the model saved in step 3, the hybrid loss function is used to improve the position loss function of the original YOLOv7 network.

[0042] Step 5: Fine-tune the hybrid hyperparameter model after modifying the loss function: adjust the improved model using different hybrid hyperparameters, observe the training output loss function, adjust the model performance to the best, and terminate the iteration;

[0043] Step 6, model output: Output the model with the optimal parameters after training to a *.pt file to facilitate the model to be ported to embedded devices;

[0044] The above disclosure is only a specific embodiment of the present invention. According to the technical concept provided by the present invention, any changes that can be thought of by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A method for small target recognition in optical remote sensing images based on the PSN-YOLO model, characterized in that The steps include: Step 1, data preparation: data conversion format and data division; Step 2: Model Improvement: First, a 160×160 small object detection head is added. Then, the attention module that can establish long-range channel dependencies and the efficient convolution module that suppresses feature redundancy are integrated into the feature extraction modules of the Backbone and Neck parts of the YOLOv7 model, respectively. The improved network model is obtained, which includes the following sub-steps: Step 2.1: First, add a 160×160 small object detection head to the Head part of the YOLOv7 model, and then introduce the feature extraction module from the Backbone part; Step 2.2: First, use the attention module to integrate into the feature extraction module of the Backbone part of the YOLOv7 model; Step 2.3: Based on step 2.2, replace the standard convolution in the feature extraction module with depth-separable convolution. Step 2.4: Based on steps 2.2 and 2.3, convolution with suppressed feature redundancy is fused into the feature extraction module of the Neck part of the YOLOv7 model. Step 3, model training: For the improved YOLOv7 model, iteratively train the model using training data and validation data until the recognition performance is optimal; Step 4, model optimization: Use the hybrid loss function based on intersection-over-union to improve the YOLOv7 original position loss function and optimize the model described in step 3; Step 5: Adjust the hybrid hyperparameters after model optimization: Verify the impact of different hybrid hyperparameters on network performance based on experimental results; Step 6: Model testing: Use the test set to input the trained model for testing, and detect all the categories and objects of different scales marked in the dataset; Step 7, model output: Output the model with optimal parameters after training to a *.pt file to facilitate the model's portability to embedded devices.

2. A method for detecting small targets in optical remote sensing images based on the PSN-YOLO model as claimed in claim 1, wherein in step 2.2, a compression and concatenation (Squeeze and Concat: SPC) module is used to obtain a channel-level multi-scale feature map, and a compression and extraction (Squeeze and Excitation: SE) module is used to extract the attention of the feature maps of different scales to obtain a channel attention vector, and then the channel-level attention vector is recalibrated using Softmax to obtain the weight of the multi-scale channel, and finally the element-by-element product operation is performed to recalibrate the weight and obtain the corresponding feature map.

3. According to the method for detecting small targets in optical remote sensing images based on the PSN-YOLO model as claimed in claim 1, in step 2.4, the spatial reconstruction unit (SRU) and the channel reconstruction unit (CRU) are placed in sequence. Specifically, for the intermediate input feature X in the bottleneck residual block, the spatial refinement feature X is first obtained by the SRU operation. w , and then use the CRU operation to obtain the channel-refined feature Y. This utilizes the spatial and channel redundancy between features to improve the expression ability of CNN.

4. A method for detecting small targets in optical remote sensing images based on the PSN-YOLO model as claimed in claim 1, wherein in step 4, as shown in formula L NWD =1-NWD(N p ,N g ), the Gaussian distribution of the Normalized Gaussian Wasserstein Distance (NWD) is introduced to calculate the loss function L NWD ; in, N p is the Gaussian distribution of the prediction box, N g is the Gaussian distribution of the real box GT, NWD(N p ,N g ) is the distance between the two Gaussian distributions.

Images