A small target detection method for low and medium resolution optical remote sensing images

Abstract

The present invention relates to a small target detection method for medium and low resolution optical remote sensing images, including: obtaining remote sensing image data training samples and test samples; using a DEM degradation model to randomly degrade the training samples; constructing a medium and low resolution remote sensing image target detection framework; constructing a loss function for joint iterative training; and detecting and evaluating medium and low resolution remote sensing image test samples. The present invention designs an additional degradation reconstruction branch through the constructed medium and low resolution remote sensing image target detection framework to effectively enhance the ability of the target detection network to cope with degraded images of different resolutions; in addition, a hybrid parallel attention feature fusion module is introduced in the target detection branch to achieve concentrated attention on target features and suppress redundant complex backgrounds; the present invention improves the accuracy of small target detection in medium and low resolution optical remote sensing images without generating additional computational complexity.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and in particular to a small target detection method for medium and low resolution optical remote sensing images. Background Art

[0002] With the continuous development of aerospace remote sensing technology, the spatial resolution of optical remote sensing images continues to improve, which has promoted the progress in the field of remote sensing target detection. High-resolution optical remote sensing images can present rich ground details and clear target texture features, which is conducive to the characteristic expression of the target. However, in actual application, due to the limitation of sensor dot size, it is still challenging to obtain large-scale high-resolution remote sensing images, which makes it difficult to carry out target monitoring in large areas.

[0003] Currently, many target detection methods focus on detection and positioning technology of high-resolution remote sensing images, while relatively few studies have been conducted on target detection methods of medium and low-resolution remote sensing images. Medium and low-resolution remote sensing images are of great significance in the rapid acquisition of intelligence in aerospace applications due to their wide coverage area and short observation period. However, since such images contain a small number of pixels, low clarity, and blurred imaging, the appearance information and texture features of the target are less, and the details are blurred, which brings great challenges to the small target detection task. Summary of the invention

[0004] The present invention aims to solve the technical problems in the prior art and provide a small target detection method for medium and low resolution optical remote sensing images. Unlike the previous method of using image super-resolution as preprocessing and then directly inputting it into the detector, the present invention designs an additional degraded reconstruction auxiliary branch to effectively improve the performance of the detector in medium and low resolution remote sensing image detection tasks; in addition, a hybrid parallel attention feature fusion module is introduced in the detector to achieve concentrated attention on target features and suppress redundant complex backgrounds, thereby improving the accuracy of the model in small target positioning.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] A small target detection method for medium and low resolution optical remote sensing images, comprising the following steps:

[0007] Step 1: Obtain remote sensing image data samples;

[0008] Step 1.1: Obtain high-resolution remote sensing images and corresponding target detection annotation data from the database as training samples;

[0009] Step 1.2: Obtain low- and medium-resolution remote sensing images from the database as test samples;

[0010] Step 2: Use the DEM degradation model to randomly degrade the training samples obtained in step 1.1 to obtain a degraded image that is closer to the real world;

[0011] Step 3: Construct a low- to medium-resolution remote sensing image target detection framework;

[0012] The construction of the medium and low resolution remote sensing image target detection framework includes two parts: the construction of the SISR degradation reconstruction branch and the construction of the target detection branch. The construction process is as follows:

[0013] Step 3.1: Construct SISR degradation reconstruction branch;

[0014] The SISR degradation reconstruction branch consists of a backbone network, an HPA-FPN feature fusion module and an SSR feature reconstruction module;

[0015] The degraded image obtained in step 2 is passed through the backbone network and the HPA-FPN feature fusion module in sequence to obtain a fused feature image, and then the fused feature image is passed through the SSR feature reconstruction module to complete the construction of the SISR degradation reconstruction branch;

[0016] Step 3.2: Build the target detection branch;

[0017] The target detection branch consists of a backbone network, an HPA-FPN feature fusion module and a detector;

[0018] The degraded image obtained in step 2 is passed through the backbone network and the HPA-FPN feature fusion module in sequence to obtain a fused feature image, and then the fused feature image is passed through the detector to complete the construction of the target detection branch;

[0019] Step 4: Construct a loss function, and perform joint iterative training on the SISR degradation reconstruction branch and target detection branch constructed in step 3.1 and step 3.2, respectively, to obtain the trained model;

[0020] Step 5: Detect and evaluate low- and medium-resolution remote sensing images;

[0021] Use the trained model obtained in step 4 to detect and evaluate the medium and low resolution remote sensing image test samples obtained in step 1.2; only the target detection branch is involved in the detection and evaluation process, and the SISR degradation reconstruction branch is only used in the iterative training process of step 4.

[0022] In the above technical solution, the DEM degradation model in step 2 is implemented by a hybrid random downsampling mechanism and a gated random selection mechanism;

[0023] The hybrid random downsampling mechanism obtains degraded images of different sizes by randomly placing them at different locations in the DEM degradation model;

[0024] The gated random selection mechanism creates more degradation type subsets through the randomness of gated switches, and can randomly perform Gaussian blur processing and / or Gaussian noise processing on images with different downsampling degrees.

[0025] In the above technical solution, the DEM degradation model in step 2 includes Gaussian blur, downsampling and Gaussian noise processing; the high-resolution remote sensing image training samples obtained in step 1.1 are randomly subjected to Gaussian blur, downsampling and Gaussian noise processing to obtain medium- and low-resolution remote sensing images that are closer to real degradation;

[0026] Among them, downsampling must be experienced, p1, p2, and p3 represent different probabilities, which determine the position of the image downsampling. p1, p2, and p3 are set to 10%, 10%, and 80%, respectively. Gaussian blur and Gaussian noise processing are randomly experienced, which is controlled by the gate switch; after the high-resolution remote sensing image enters the DEM degradation model, it is first randomly downsampled and then Gaussian blur and Gaussian noise processing, or Gaussian blur processing is first performed and then downsampling and Gaussian noise processing, or only downsampling plus Gaussian blur and Gaussian noise processing are performed; at the same time, the positions of Gaussian blur, downsampling, and Gaussian noise processing are randomly selected.

[0027] In the above technical solution, the backbone network and the HPA-FPN feature fusion module described in step 3.1 and step 3.2 have the same structure and share parameters during the model training process.

[0028] In the above technical solution, the backbone network described in step 3.1 and step 3.2 adopts ResNet50, and the HPA-FPN feature fusion module adopts a feature pyramid structure based on the HPA attention mechanism.

[0029] In the above technical solution, the HPA attention mechanism is a dual-branch parallel structure. For a given input feature Pass it to each branch separately, and finally output a refined feature of the same shape;

[0030] For the left branch, the input feature map is first globally pooled to obtain a feature map of shape C×1×1; then the channel is scaled r times by 1x1 2d convolution and nonlinear activation function ReLU, and its feature shape is converted to C / r×1×1; then the channel is restored to C by 1x1 2d convolution; finally, the attention weight W of each feature channel is obtained through the mapping operation of Sigmoid activation function. C ;

[0031] For the right branch, the input feature map is firstly flat-pooled in the X and Y dimensions to obtain feature maps of shape C×H×1 and C×1×W respectively; then the two feature maps are concatenated in the spatial dimension and a 1x1 2d convolution is performed to obtain a feature map of shape C / r×1×(W+H); then the spatial information in the X and Y directions is encoded by BatchNorm and Non-linear; next, the split operation is performed first, and then the 1x1 2d convolution is performed to restore the channel to C; finally, the attention weight W is obtained by the same Sigmoid activation function mapping. S ;

[0032] The attention weights W obtained from the left branch and the right branch respectively C and W S The refined feature image is obtained by multiplying the original input feature map element by element.

[0033] In the above technical scheme, the SSR feature reconstruction module described in step 3.1 is composed of a spatial shift residual block (SSCBlock), an upsampling block and a pixel shuffle operation; given a low-resolution deep feature, the feature is firstly upsampled continuously by alternately cascading the spatial shift residual block and the upsampling block; then, the upsampled features generated by the jump connection are aggregated; finally, the aggregated features are superimposed and a pixel shuffle operation is performed to obtain a reconstructed super-resolution image.

[0034] In the above technical solution, the spatial shift residual block includes 1x1 convolution, 3x3 convolution, spatial shift operation and ReLU activation function; the spatial shift residual block aligns the features of neighbors through spatial shift operation, and can aggregate local features; or further expands the extraction of long-distance feature relationships by controlling the selection of shifted feature points.

[0035] In the above technical solution, the loss function described in step 4 is composed of degradation reconstruction loss and target detection loss, and its construction process is as follows:

[0036] Step 4.1: Construct degradation reconstruction loss;

[0037] L DR Defined as the reconstructed image I after degradation SR and the original high-resolution image I HR The L1 distance between DR =||I SR -I HR ||1;

[0038] Step 4.2: Construct object detection loss;

[0039] The target detection loss L Det Including size prediction branch loss L s , center point displacement loss L O And the center point prediction loss L C , whose mathematical expression is L Det =λ1L S +λ2L O +L C , where λ1 and λ2 are parameters;

[0040] Step 4.3: Construct the total loss function;

[0041] The total loss function L Total Including the degradation reconstruction loss in step 4.1 and the target detection loss in step 4.2, its mathematical expression is L Total =αL DR +βL Det , where α and β are parameters.

[0042] In the above technical solution, the target detection loss L in step 4.2 is Det =λ1L S +λ2L O +L C , where the size prediction branch loss L S Using Gauss Focal loss, center point displacement loss L O And the center point prediction loss L C L1 loss is used in both cases; parameters λ1 and λ2 are set to 0.1 and 1 respectively;

[0043] The total loss function L described in step 4.3 Total =αL DR +βL Det , where the parameters α and β are set to 0.4 and 1, respectively.

[0044] The beneficial effects of the present invention are:

[0045] The small target detection method for medium and low resolution optical remote sensing images provided by the present invention is conducive to enhancing the feature information of the target and guiding the detection branch to have the ability to learn super-resolution reconstruction features by constructing a dual-branch deep learning framework consisting of a degenerate reconstruction branch and a target detection branch; the constructed feature fusion module based on hybrid parallel attention can realize the aggregation and reconstruction of feature maps, and introduces an attention enhancement mechanism to guide the network to pay more attention to the feature information of small targets, so that the network can effectively capture the detailed information of the target and suppress the interference of redundant background; in the reasoning detection process, only the detection branch is used to improve the performance of the algorithm for small target detection in medium and low resolution remote sensing images without increasing a large amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0047] Figure 1 It is a flowchart of the steps of the small target detection method for medium and low resolution optical remote sensing images of the present invention;

[0048] Figure 2 A network structure diagram of a small target detection method for medium and low resolution optical remote sensing images of the present invention;

[0049] Figure 3 It is a schematic diagram of a DEM degradation model of a small target detection method for medium and low resolution optical remote sensing images of the present invention;

[0050] Figure 4 It is a structural diagram of an SSR feature reconstruction module of a small target detection method for medium and low resolution optical remote sensing images of the present invention;

[0051] Figure 5 It is a structural diagram of the HPA-FPN feature fusion module of the small target detection method for medium and low resolution optical remote sensing images of the present invention;

[0052] Figure 6 This is a detection effect diagram of the small target detection method for medium and low resolution optical remote sensing images of the present invention on the VEDAI dataset;

[0053] Figure 7 This is a diagram showing the detection effect of the small target detection method for medium and low resolution optical remote sensing images of the present invention on the Airbus-Ships dataset. DETAILED DESCRIPTION

[0054] A small target detection method for medium and low resolution optical remote sensing images of the present invention comprises the following steps:

[0055] Step 1: Obtain remote sensing image data samples, which include training samples and test samples;

[0056] Step 2: Use the DEM degradation model to randomly degrade the training samples obtained in step 1;

[0057] Step 3: Construct a low- to medium-resolution remote sensing image target detection framework;

[0058] Step 4: Construct loss function for joint iterative training;

[0059] Step 5: Detect and evaluate the low- and medium-resolution remote sensing image test samples.

[0060] The above step 1 specifically includes the following steps:

[0061] Step 1.1: Obtain high-resolution remote sensing images and corresponding target detection annotation data from the database as training samples;

[0062] Step 1.2: Obtain low- and medium-resolution remote sensing images from the database as test samples.

[0063] The above step 2 specifically includes the following steps:

[0064] The high-resolution remote sensing image training samples obtained in step 1.1 are randomly degraded through the DEM degradation model to obtain a degraded image that is closer to the real world;

[0065] The DEM degradation model is implemented by hybrid random downsampling mechanism and gated random selection mechanism;

[0066] The hybrid random downsampling mechanism obtains degraded images of different sizes by randomly placing them at different positions in the DEM degradation model, unlike the traditional degradation model which can only obtain images of the same size by fixing the downsampling multiple, thus having a more effective expression of real image degradation;

[0067] The gated random selection mechanism creates more degradation type subsets through the randomness of the gate switch. It can randomly perform Gaussian blur or Gaussian noise processing, or a combination of the two, on images with different downsampling levels. Unlike traditional methods that are restricted by fixed degradation types and degradation orders, it forms a more powerful degradation representation model.

[0068] The DEM degradation model includes Gaussian blur, downsampling and Gaussian noise processing; the high-resolution remote sensing image training samples are randomly processed by the above Gaussian blur, downsampling and Gaussian noise processing to obtain low- and medium-resolution remote sensing images that are closer to the actual degradation;

[0069] The blur degradation is modeled as a convolution operation between an image and a Gaussian filter. In order to include various Gaussian blur types, the Gaussian filter uses two commonly used kernel forms, isotropic and anisotropic, in the degradation model. The kernel size is randomly obtained from {7×7, 9×9,..., 21×21}. The kernel width of the isotropic Gaussian kernel is uniformly sampled from [0.1, 2.4], the longer kernel width of the anisotropic Gaussian kernel is uniformly sampled from [0.5, 6], and the kernel angle is uniformly sampled from [0, π].

[0070] Downsampling is the most basic operation for transforming from high resolution to low resolution. Different downsampling methods will bring different effects. There are many common downsampling methods, including nearest neighbor interpolation, bilinear interpolation, bicubic interpolation and area adjustment. Since nearest neighbor interpolation downsampling will cause spatial offset errors, it is discarded. The downsampling rate is obtained from uniform sampling of [1.0, 4.0].

[0071] Gaussian noise is the most widely used type of noise, so the Gaussian noise model is used to simulate sensor noise and other external noise; the AWGN noise model is adopted, where the additive noise n~N(0,σ) and the variance σ are uniformly sampled from {1 / 255, 2 / 255,···, 25 / 255}.

[0072] Among them, downsampling must be experienced, p1, p2, and p3 represent different probabilities, which determine the position of the image downsampling, which is set to 10%, 10%, and 80% here, while Gaussian blur and Gaussian noise processing are randomly experienced, which is controlled by the gate switch. For example, after the image enters the DEM degradation model, it may be randomly downsampled first, and then Gaussian blur and Gaussian noise processing, or it may be Gaussian blur processing first, and then downsampling and Gaussian noise processing, or it may only be downsampling plus Gaussian blur and Gaussian noise processing; in short, downsampling must be experienced, Gaussian noise and Gaussian blur processing can be randomly selected through the gate switch, and the positions of the three are also randomly selected. This degradation method can obtain medium and low resolution images with different degradation degrees and different sizes, which are closer to the images obtained in the actual scene.

[0073] The above step 3 specifically includes the following steps:

[0074] Step 3.1: Construct SISR degradation reconstruction branch:

[0075] The SISR degradation reconstruction branch is composed of a backbone network, an HPA-FPN feature fusion module and an SSR feature reconstruction module; the degraded image obtained in step 2 is sequentially passed through the backbone network and the HPA-FPN feature fusion module to obtain fused features, and then passed through the SSR feature reconstruction module to complete the construction process of the SISR degradation reconstruction branch;

[0076] Step 3.2: Build the object detection branch:

[0077] The target detection branch is composed of a backbone network, an HPA-FPN feature fusion module and a detector; the degraded image obtained in step 2 is sequentially passed through the backbone network and the HPA-FPN feature fusion module to obtain fused features, and then passed through the detector to complete the construction process of the target detection branch;

[0078] The backbone network and HPA-FPN feature fusion module described in step 3.1 and step 3.2 have the same structure and share parameters during model training;

[0079] The backbone network shared in step 3.1 and step 3.2 adopts ResNet50, and the HPA-FPN feature fusion module adopts a feature pyramid structure based on the HPA attention mechanism;

[0080] The HPA attention mechanism is a dual-branch parallel structure. For a given input feature Pass it to each branch separately, and finally output a refined feature of the same shape;

[0081] For the left branch, the input feature map is first globally pooled to obtain a feature map of shape C×1×1. Then the channel is scaled r times by 1x1 2d convolution and nonlinear activation function ReLU, and its feature shape is converted to C / r×1×1; then the channel is restored to C by 1x1 2d convolution; finally, the attention weight W of each feature channel is obtained through the mapping operation of Sigmoid activation function. C ;

[0082] For the right branch, the input feature map is first flat-pooled in the X and Y dimensions to obtain feature maps of shape C×H×1 and C×1×W respectively; then the two feature maps are concatenated in the spatial dimension and a 1x1 2d convolution is performed to obtain a feature map of shape C / r×1×(W+H); BatchNorm and Non-linear are used to encode the spatial information in the X and Y directions; next, the split operation is performed first, and then the 1x1 2d convolution is performed to restore the channel to C; finally, the weighted attention W is obtained by the same Sigmoid activation function mapping.S ;

[0083] The attention weights W obtained from the two branches are C and W S Multiply the original input feature map element by element to obtain the refined features;

[0084] The SSR feature reconstruction module of step 3.1 is composed of a spatial shift residual block (SSCBlock), an upsampling block and a pixel shuffle operation; given a low-resolution deep feature, the feature is firstly upsampled continuously by alternately cascading the spatial shift residual block and the upsampling block; then, the upsampled features generated by the jump connection are aggregated; finally, the aggregated features are superimposed and a pixel shuffle operation is performed to obtain a reconstructed super-resolution image;

[0085] Among them, SSCBlock mainly includes 1x1 convolution, 3x3 convolution, spatial shift operation and ReLU activation function; SSCBlock can align the features of neighbors through spatial shift operation and aggregate local features; it can further expand the extraction of long-distance feature relationships by controlling the selection of shifted feature points.

[0086] The above step 4 specifically includes the following steps:

[0087] The loss function includes two parts: constructing degradation reconstruction loss and constructing target detection loss;

[0088] Step 4.1: Construct the degraded reconstruction loss: L DR Defined as the reconstructed image I after degradation SR and the original high-resolution image I HR The L1 distance between DR =||I SR -I HR ||1;

[0089] Step 4.2: Construct object detection loss: The object detection loss L Det Including size prediction branch loss L S , center point displacement loss L O And the center point prediction loss L C ; Its mathematical expression is L Det =λ1L s +λ2L O +L C , where λ1 and λ2 are parameters;

[0090] Among them, the size prediction branch loss L S Using Gauss Focal loss, center point displacement loss LO And the center point prediction loss L C L1 loss is used in both cases; parameters λ1 and λ2 are set to 0.1 and 1 respectively;

[0091] Step 4.3: Construct the total loss function: The total loss function L Total It includes the degradation reconstruction loss in step 4.1 and the target detection loss in step 4.2; its mathematical expression is L Total =αL DR +βL Det , where α and β are parameters;

[0092] The parameters α and β are set to 0.4 and 1 respectively.

[0093] The above step 5 specifically includes the following steps:

[0094] Use the trained model obtained in step 4 to detect and evaluate the low- and medium-resolution remote sensing image test samples obtained in step 1.2;

[0095] The detection and evaluation process only involves the target detection branch, while the degraded reconstruction branch is only used in the iterative training process of step 4.

[0096] The present invention will be described more clearly and completely below through examples.

[0097] See also Figure 1 , the step flow chart of the small target detection method for medium and low resolution optical remote sensing images of the present invention specifically includes the following steps:

[0098] 1) Obtain remote sensing image data samples, including the following steps:

[0099] 1.1) Obtain high-resolution remote sensing images and corresponding target detection annotation data from the database as training samples, with a ratio of 80%;

[0100] 1.2) Obtain medium and low resolution remote sensing images from the database as test samples, with a ratio of 20%;

[0101] 1.3) Dataset 1 uses the VEDAI dataset, in which the background and interference of the targets are relatively diverse, covering complex scenes such as cities, fields, forests, and mountains. After removing extremely rare targets and erroneous data in the dataset, there are 1,245 images in the data, covering 9 common modes of transportation, namely ships, campers, cars, pickups, airplanes, tractors, trucks, vans, and other categories. There are 3,750 target instances in the dataset, and the size of each image is 1024 pixels × 1024 pixels and 512 pixels × 512 pixels, respectively, with spatial resolutions of 12.5 cm / pixel × 12.5 cm / pixel and 25 cm / pixel × 25 cm / pixel, respectively. The training sample set and the test sample set consist of 996 images and 249 images, respectively. This dataset not only has small target volumes, but also has conditions such as illumination changes and occlusion, and has strong robustness.

[0102] 1.4) Dataset 2 uses the Airbus-Ships dataset, which contains remote sensing images of single or multiple ships located in open seas, ports or docks; there are 2730 images in the dataset, of which 2184 images are used for training and 546 images are used for testing; each image has two specifications of 768 pixels × 768 pixels and 384 pixels × 384 pixels;

[0103] 2) Using the DEM degradation model to randomly degrade the training samples, the specific steps include:

[0104] 2.1) The hybrid random downsampling mechanism obtains degraded images of different sizes by randomly placing them at different positions in the DEM degradation model, where p1, p2, and p3 represent different probabilities, which determine the position where the image is downsampled, which are set to 10%, 10%, and 80% here;

[0105] 2.2) The gated random selection mechanism creates more degradation type subsets through the randomness of the gate switch. It randomly performs Gaussian blur or Gaussian noise processing on images with different downsampling levels, or a combination of the two, to form a more powerful degradation representation model;

[0106] The DEM degradation model includes Gaussian blur, downsampling and Gaussian noise processing; the high-resolution remote sensing image is randomly subjected to the above Gaussian blur, downsampling and Gaussian noise processing to obtain a medium- and low-resolution remote sensing image that is closer to the actual degradation;

[0107] The blur degradation is modeled as a convolution operation between an image and a Gaussian filter. In order to include various Gaussian blur types, the Gaussian filter in the DEM degradation model uses two commonly used kernel forms, isotropic and anisotropic, and the kernel size is randomly obtained from {7×7, 9×9,..., 21×21}. The kernel width of the isotropic Gaussian kernel is uniformly sampled from [0.1, 2.4], the longer kernel width of the anisotropic Gaussian kernel is uniformly sampled from [0.5, 6], and the kernel angle is uniformly sampled from [0, π].

[0108] Downsampling is the most basic operation of transforming from high resolution to low resolution. Different downsampling methods will bring different effects. There are many common downsampling methods, including nearest neighbor interpolation, bilinear interpolation, bicubic interpolation and area adjustment. Since the nearest neighbor interpolation downsampling will cause spatial offset error, it is discarded. The downsampling rate is obtained from uniform sampling of [1.0, 4.0].

[0109] Gaussian noise is the most widely used type of noise, so the Gaussian noise model is used to simulate sensor noise and other external noise; the AWGN noise model is adopted, where the additive noise n~N(0,σ) and the variance σ are uniformly sampled from {1 / 255, 2 / 255,···, 25 / 255}.

[0110] The principle diagram of the DEM degradation model of a small target detection method for medium and low resolution optical remote sensing images involved in this embodiment is as follows: Figure 3 As shown;

[0111] 3) Construct a low- to medium-resolution remote sensing image target detection framework, which includes the following steps:

[0112] The construction of the medium and low resolution remote sensing image target detection framework includes two parts: the construction of the SISR degradation reconstruction branch and the construction of the target detection branch. The construction process is as follows:

[0113] 3.1) Constructing a SISR degradation reconstruction branch: The SISR degradation reconstruction branch consists of a backbone network, an HPA-FPN feature fusion module, and an SSR feature reconstruction module; the degraded image obtained in step 2 is sequentially passed through the backbone network and the HPA-FPN feature fusion module to obtain fused features, and then passed through the SSR feature reconstruction module to complete the construction process of the SISR degradation reconstruction branch;

[0114] 3.2) Constructing a target detection branch: The target detection branch is composed of a backbone network, an HPA-FPN feature fusion module and a detector; the degraded image obtained in step 2 is sequentially passed through the backbone network and the HPA-FPN feature fusion module to obtain fused features, and then passed through the detector to complete the construction process of the target detection branch;

[0115] The backbone network and the HPA-FPN feature fusion module described in step 3.1 and step 3.2 have the same structure, and parameters are shared during the model training process; wherein, the backbone network adopts ResNet50, and the HPA-FPN feature fusion module adopts a feature pyramid structure based on the HPA attention mechanism;

[0116] The HPA attention mechanism is a dual-branch parallel structure. For a given input feature It is passed to each branch respectively, and finally a refined feature of the same shape is output; for the left branch, the input feature map is first globally pooled to obtain a feature map of shape C×1×1. Then the channel is scaled r times by 1x1 2d convolution and nonlinear activation function ReLU, and its feature shape is converted to C / r×1×1; then the channel is restored to C by 1x1 2d convolution; finally, the attention weight W of each feature channel is obtained through the mapping operation of Sigmoid activation function. C ; For the right branch, firstly, the input feature map is flat-pooled in the X and Y dimensions respectively to obtain feature maps of shape C×H×1 and C×1×W respectively; then the two feature maps are concatenated in the spatial dimension and a 1x1 2d convolution is performed to obtain a feature map of shape C / r×1×(W+H); then BatchNorm and Non-linear are used to encode the spatial information in the X and Y directions; next, the split operation is performed first, and then the 1x1 2d convolution is performed to restore the channel to C; finally, the attention weight W is obtained by the same Sigmoid activation function mapping. S ; The attention weights W obtained from the two branches C and W S Multiply the original input feature map element by element to obtain the refined features;

[0117] The structure diagram of the HPA-FPN feature fusion module of a small target detection method for medium and low resolution optical remote sensing images involved in this embodiment is as follows: Figure 5 As shown;

[0118] The SSR feature reconstruction module of step 3.1 is composed of a spatial shift residual block (SSCBlock), an upsampling block and a pixel shuffle operation; given a low-resolution deep feature, the feature is firstly upsampled continuously by alternately cascading the spatial shift residual block and the upsampling block; then, the upsampled features generated by the jump connection are aggregated; finally, the aggregated features are superimposed and a pixel shuffle operation is performed to obtain a reconstructed super-resolution image; wherein, the SSCBlock mainly includes a 1x1 convolution, a 3x3 convolution, a spatial shift operation and a ReLU activation function;

[0119] The structure diagram of the SSR feature reconstruction module of a small target detection method for medium and low resolution optical remote sensing images involved in this embodiment is as follows: Figure 4 As shown;

[0120] The network structure diagram of a small target detection method for medium and low resolution optical remote sensing images involved in this embodiment is as follows: Figure 2 As shown;

[0121] 4) Construct a loss function for joint iterative training, which specifically includes the following steps:

[0122] The loss function is composed of degradation reconstruction loss and target detection loss, and its construction process is as follows:

[0123] 4.1) Constructing the degradation reconstruction loss: L DR Defined as the reconstructed image I after degradation SR and the original high-resolution image I HR The L1 distance between DR =||I SR -I HR ||1;

[0124] 4.2) Constructing target detection loss: The target detection loss L Det Including size prediction branch loss L S , center point displacement loss L O And the center point prediction loss L C ; Its mathematical expression is L Det =λ1L S +λ2L O +L C , where λ1 and λ2 are parameters; where the size prediction branch loss L S Using Gauss Focal loss, center point displacement loss L O And the center point prediction loss L C L1 loss is used in both cases; parameters λ1 and λ2 are set to 0.1 and 1 respectively;

[0125] 4.3) Constructing the total loss function: The total loss function L Total It includes the degradation reconstruction loss in step 4.1 and the target detection loss in step 4.2; its mathematical expression is L Total =αL DR +βL Det , where α and β are parameters; where parameters α and β are set to 0.4 and 1 respectively;

[0126] 4.4) Joint iterative training: The iterative training was built on the Ubuntu platform, and the model training and inference work was performed on the NVIDIA3090 graphics card based on the PyTorch deep learning framework; in the experiment, the stochastic gradient descent (SGD) method was used for training, the initial learning rate was 0.02, and the weight decay parameter and momentum were set to 0.9 and 0.0001 respectively;

[0127] 5) Detect and evaluate the low and medium resolution remote sensing image test samples, including the following steps:

[0128] 5.1) Input the test samples obtained in step 1.2 into the model obtained after training in step 4.4) to test the detection performance of the small target detection network for medium and low resolution optical remote sensing images;

[0129] 5.2) Evaluate the detection effect of small target detection network for low and medium resolution optical remote sensing images: For experimental evaluation, the commonly used mean average precision (mAP) is used as the main indicator for method performance comparison;

[0130] Some experimental results of this embodiment on the VEDAI dataset are shown in the figure below. Figure 6 As shown;

[0131] In order to demonstrate the detection effect of the embodiment of the present invention on the VEDAI data set, the embodiment of the present invention is compared with the existing mainstream detection network, and the experimental indicators are shown in the following table:

[0132] Table 1: Comparison of average precision of different methods on VEDAI dataset;

[0133]

[0134] It can be seen from Table 1 that the average accuracy evaluation index of the present invention on the VEDAI dataset reaches 82.7%, which is significantly better than other mainstream methods.

[0135] Some experimental results of this embodiment on the Airbus-Ships dataset are shown in the figure below. Figure 7 As shown;

[0136] In order to demonstrate the detection effect of the embodiment of the present invention on the Airbus-Ships dataset, the embodiment of the present invention is compared with the existing mainstream detection network. The experimental indicators are shown in the following table:

[0137] Table 2: Comparison of average precision of different methods on Airbus-Ships dataset;

[0138]

[0139] It can be seen from Table 2 that the average accuracy evaluation index of the present invention on the Airbus-Ships dataset reaches 77.5%, which is significantly better than other mainstream methods.

[0140] In summary, the present invention provides a small target detection method for medium and low resolution optical remote sensing images, which constructs a medium and low resolution remote sensing image target detection framework and designs an additional degradation reconstruction branch to effectively enhance the ability of the target detection network to cope with degraded images of different resolutions; in addition, a hybrid parallel attention feature fusion module is introduced in the target detection branch to achieve focused attention on target features and suppress redundant complex backgrounds; the present invention improves the accuracy of small target detection in medium and low resolution optical remote sensing images without generating additional computational complexity.

[0141] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A small target detection method for medium and low resolution optical remote sensing images, characterized in that: The following steps are involved: Step 1: Obtain remote sensing image data samples; Step 1.1: Obtain high-resolution remote sensing images and corresponding target detection annotation data from the database as training samples; Step 1.2: Obtain low- and medium-resolution remote sensing images from the database as test samples; Step 2: Use the DEM degradation model to randomly degrade the training samples obtained in step 1.1 to obtain a degraded image that is closer to the real world; Step 3: Construct a low- to medium-resolution remote sensing image target detection framework; The construction of the medium and low resolution remote sensing image target detection framework includes two parts: the construction of the SISR degradation reconstruction branch and the construction of the target detection branch. The construction process is as follows: Step 3.1: Construct SISR degradation reconstruction branch; The SISR degradation reconstruction branch consists of a backbone network, an HPA-FPN feature fusion module and an SSR feature reconstruction module; The degraded image obtained in step 2 is passed through the backbone network and the HPA-FPN feature fusion module in sequence to obtain a fused feature image, and then the fused feature image is passed through the SSR feature reconstruction module to complete the construction of the SISR degradation reconstruction branch; Step 3.2: Build the target detection branch; The target detection branch consists of a backbone network, an HPA-FPN feature fusion module and a detector; The degraded image obtained in step 2 is passed through the backbone network and the HPA-FPN feature fusion module in sequence to obtain a fused feature image, and then the fused feature image is passed through the detector to complete the construction of the target detection branch; Step 4: Construct a loss function, and perform joint iterative training on the SISR degradation reconstruction branch and target detection branch constructed in step 3.1 and step 3.2, respectively, to obtain the trained model; Step 5: Detect and evaluate low- and medium-resolution remote sensing images; Use the trained model obtained in step 4 to detect and evaluate the low- and medium-resolution remote sensing image test samples obtained in step 1.2; only the target detection branch is involved in the detection and evaluation process, and the SISR degradation reconstruction branch is only used in the iterative training process of step 4; The DEM degradation model described in step 2 is implemented by hybrid random downsampling mechanism and gated random selection mechanism; The hybrid random downsampling mechanism obtains degraded images of different sizes by randomly placing them at different locations in the DEM degradation model; The gated random selection mechanism creates more degradation type subsets through the randomness of gated switches, and can randomly perform Gaussian blur processing and / or Gaussian noise processing on images with different downsampling degrees; The DEM degradation model in step 2 includes Gaussian blur, downsampling and Gaussian noise processing; the high-resolution remote sensing image training samples obtained in step 1.1 are randomly subjected to Gaussian blur, downsampling and Gaussian noise processing to obtain medium- and low-resolution remote sensing images that are closer to real degradation; Among them, downsampling must be experienced, p1, p2, and p3 represent different probabilities, which determine the position of the image downsampling. p1, p2, and p3 are set to 10%, 10%, and 80% respectively, while Gaussian blur and Gaussian noise processing are randomly experienced, which is controlled by the gate switch; after the high-resolution remote sensing image enters the DEM degradation model, it is first randomly downsampled and then Gaussian blur and Gaussian noise processing, or Gaussian blur processing is first performed and then downsampling and Gaussian noise processing, or only downsampling plus Gaussian blur and Gaussian noise processing are performed; at the same time, the positions of Gaussian blur, downsampling, and Gaussian noise processing are randomly selected; The HPA-FPN feature fusion module adopts a feature pyramid structure based on the HPA attention mechanism; The HPA attention mechanism is a dual-branch parallel structure. For a given input feature Pass it to each branch separately, and finally output a refined feature of the same shape; For the left branch, the input feature map is first globally pooled to obtain a feature map of shape C×1×1; then the channel is scaled r times by 1x1 2d convolution and nonlinear activation function ReLU, and its feature shape is converted to C / r×1×1; then the channel is restored to C by 1x1 2d convolution; finally, the attention weight W of each feature channel is obtained through the mapping operation of Sigmoid activation function. C ; For the right branch, the input feature map is first average pooled in the X and Y dimensions to obtain feature maps of shape C×H×1 and C×1×W respectively; then the two feature maps are concatenated in the spatial dimension and a 1x1 2d convolution is performed to obtain a feature map of shape C / r×1×(W+H); BatchNorm and Non-linear are used to encode the spatial information in the X and Y directions; next, the split operation is performed first, and then the 1x1 2d convolution is performed to restore the channel to C; finally, the attention weight W is obtained by the same Sigmoid activation function mapping. S ; The attention weights W obtained from the left branch and the right branch respectively C and W S The refined feature image is obtained by multiplying the original input feature map element by element.

2. The small target detection method for medium and low resolution optical remote sensing images according to claim 1 is characterized in that: The backbone network and HPA-FPN feature fusion module described in step 3.1 and step 3.2 have the same structure and share parameters during model training.

3. The small target detection method for medium and low resolution optical remote sensing images according to claim 2 is characterized in that: The backbone network described in step 3.1 and step 3.2 adopts ResNet50.

4. The small target detection method for medium and low resolution optical remote sensing images according to claim 1, characterized in that: The SSR feature reconstruction module described in step 3.1 consists of a spatial shift residual block, an upsampling block and a pixel shuffle operation; given a low-resolution deep feature, the feature is first upsampled continuously by alternately cascading the spatial shift residual block and the upsampling block; then, the upsampled features generated by the jump connection are aggregated; finally, the aggregated features are superimposed and a pixel shuffle operation is performed to obtain a reconstructed super-resolution image.

5. The small target detection method for medium and low resolution optical remote sensing images according to claim 4 is characterized in that: The spatial shift residual block includes 1x1 convolution, 3x3 convolution, spatial shift operation and ReLU activation function; the spatial shift residual block aligns the features of neighbors through spatial shift operation, and can achieve aggregation of local features; or further expands the extraction of long-distance feature relationships by controlling the selection of shifted feature points.

6. The small target detection method for medium and low resolution optical remote sensing images according to any one of claims 1 to 5, characterized in that: The loss function described in step 4 consists of the degraded reconstruction loss and the target detection loss, and its construction process is as follows: Step 4.1: Construct degradation reconstruction loss; L DR Defined as the reconstructed image I after degradation SR and the original high-resolution image I HR The L1 distance between DR =||I SR -I HR ||1; Step 4.2: Construct object detection loss; The target detection loss L Det Including size prediction branch loss L S , center point displacement loss L O And the center point prediction loss L C , whose mathematical expression is L Det =λ1L S +λ2L O +L C , where λ1 and λ2 are parameters; Step 4.3: Construct the total loss function; The total loss function L Total Including the degradation reconstruction loss in step 4.1 and the target detection loss in step 4.2, its mathematical expression is L Total =αL DR +βL Det , where α and β are parameters.

7. The small target detection method for medium and low resolution optical remote sensing images according to claim 6, characterized in that: The target detection loss L described in step 4.2 Det =λ1L S +λ2L O +L C , where the size prediction branch loss L S Using Gauss Focal loss, center point displacement loss L O And the center point prediction loss L C L1 loss is used in both cases; parameters λ1 and λ2 are set to 0.1 and 1 respectively; The total loss function L described in step 4.3 Total =αL DR +βL Det , where the parameters α and β are set to 0.4 and 1, respectively.

Images