A remote sensing image object detection method based on adaptive downsampling and scale enhancement detection head
Through adaptive downsampling and scale-enhanced detection head methods, the problem of low detection accuracy of small objects in remote sensing images is solved, and efficient and accurate micro object detection is achieved, reducing model complexity and inference time.
Patent Information
- Application Number
- CN202510099535.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2025-08-08
- Estimated Expiration
- 2045-01-22
AI Technical Summary
In the prior art, when detecting small objects in remote sensing images, key feature information is easily lost during downsampling, resulting in low detection accuracy. At the same time, the number of parameters and calculations of the detection head is large, which increases the inference time, and it is difficult to identify small objects based on appearance.
Adaptive downsampling and scale-enhanced detection heads are adopted to adjust the receptive field through the dynamic feature extraction module, and key information is retained by the adaptive downsampling module, decoupled detection heads are designed to improve the detection accuracy of small targets, and the amount of model parameters is reduced through shared convolution.
The detection accuracy of tiny targets in remote sensing images is improved, the parameter quantity and calculation cost of the model are reduced, the inference time is shortened, and the sensitivity to tiny targets is enhanced.
Smart Images
Figure CN120032249B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head, and belongs to the field of computer vision target detection. Background Art
[0002] In the field of computer vision, downsampling plays a key role. Max pooling is one of the commonly used downsampling methods. It slides a predefined window of a fixed size in the feature map and selects the maximum value in the window as the output. Strided convolution is also a common downsampling method. During the convolution process, a step size greater than 1 is used to change the sliding mode of the convolution kernel. However, for small targets in remote sensing images, since these targets are usually small in size and have fewer features, key feature information is easily lost during the downsampling process, which affects the detection performance. To solve this problem, many researchers are committed to optimizing the downsampling operation to retain more texture and detail information in the feature map, thereby improving the detection accuracy of small targets. Lu et al. proposed a plug-and-play robust feature downsampling method. This method combines three different downsampling operations: convolution, slicing, and max pooling, to extract multiple feature maps to create a feature map containing a complementary feature set.
[0003] In the field of object detection, the design of the detection head is key. Many researchers have improved the detection head to improve detection accuracy. For example, Wu et al. proposed a dual-head method, which includes a fully connected head focused on classification and a convolutional head for bounding box regression. Li et al. proposed an attention-based decoupling that allows classification and regression tasks to focus on the required features. The hybrid attention module assigns different weights to classification and regression in different branches, thereby effectively improving detection performance. Zhang et al. combined an asymmetric decoupled lightweight detector head designed with depthwise separable convolutions, achieving a significant reduction in computational complexity and a significant improvement in inference speed.
[0004] As convolutional neural networks deepen, their receptive field can theoretically increase linearly. However, the effective receptive field is only a fraction of the theoretical receptive field. To this end, many researchers have designed convolutional neural networks with larger receptive fields. For example, Azad et al. introduced the concept of deformable large kernel attention, employing large kernel convolution to fully understand context. Deformable convolutions can flexibly distort the sampling network, allowing the model to adapt to diverse data patterns.
[0005] These methods optimize the object detection task from different perspectives, but the importance of different features is overlooked during the downsampling process, and the detection head fails to fully consider the feature representation and detection requirements of densely packed small objects, resulting in low small object detection accuracy. Furthermore, the large number of parameters and computational complexity of the detection head leads to long training times and significantly increases inference time. Furthermore, small objects are difficult to identify based solely on appearance and require contextual identification. Therefore, a method specifically designed for small object detection is urgently needed to address these issues. Summary of the Invention
[0006] In view of the shortcomings of the above-mentioned existing methods, the present invention provides a remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head.
[0007] The present invention is implemented by the following scheme:
[0008] Step 1: Obtain remote sensing image target detection dataset;
[0009] Step 2: Preprocess the remote sensing image dataset;
[0010] Step 3: Establish a remote sensing image target detection model;
[0011] Step 4: Use the training set of the data set to train the model, construct a loss function to update the model parameters, and obtain the optimal model;
[0012] Step 5: Use the optimal model to detect the test set images of the dataset to obtain the final test results.
[0013] Furthermore, in step 2, the AI-TOD remote sensing dataset is cropped into 800×800 pixel blocks with an overlap of 200 pixels; if the original image is smaller than 800×800, it is padded with zeros; in addition, the label file is converted into txt format.
[0014] Furthermore, the step 3 establishes a remote sensing image target detection model: in the feature extraction stage, a dynamic feature extraction module is used to flexibly adjust the receptive field of the feature extraction network to obtain contextual information of small targets; in the feature fusion stage, an adaptive downsampling module is used to emphasize the importance of different regions in the feature map and retain key information features to achieve efficient and adaptive downsampling; in the prediction stage, a scale-enhanced detection head with a decoupled design is used to provide the best feature representation for classification and regression tasks through shared convolution and reduce the number of model parameters; a high-resolution feature map is input into the detection head to obtain a feature map specifically for predicting small targets, thereby improving sensitivity to small targets.
[0015] Furthermore, the dynamic feature extraction module divides the channel dimension into two branches through 1×1 convolution, one of which is directly passed to the output, and the other branch dynamically adjusts the effective receptive field through processing by the dynamic perception unit; then, the features of the two branches are spliced; finally, the number of channels is adjusted through 1×1 convolution to obtain the output.
[0016] Furthermore, the dynamic perception unit is composed of an expansion re-parameter module, a channel attention and a feed-forward network.
[0017] Furthermore, the expansion re-parameterization module uses multiple expansion convolutions to enhance the effect of a large-kernel standard convolution. The expansion re-parameterization module structure is divided into two types, a and b. During the training phase, a is composed of three parallel branches, and the kernel sizes in the convolution layer are {5, 3, 3}, and the void rates are {1, 2, 3}. b is composed of five parallel branches, and the convolution kernels are {5, 7, 3, 3, 3}, and the void rates are {1, 2, 3, 4, 5}. After the convolution layer, their outputs are dimensionally spliced after the normalization layer. In the inference phase, the structural re-parameterization method is used to fuse the batch normalization layer into the convolution layer, thereby improving the inference speed. The size of the fused convolution kernel is related to the size of the expanded convolution kernel in each branch, and the largest convolution kernel is taken as the size of the fused convolution kernel.
[0018] Furthermore, the channel attention makes the model emphasize useful features by weighting the channels, thereby improving the quality of features generated by the model. First, input feature map X∈R C×H×W Through global average pooling, space compression is performed to obtain the new feature Z∈R C×1×1 Then, a gating mechanism consisting of two fully connected layers and an activation function is used to generate weights for each feature channel. Finally, the weights are applied to the original feature map through channel-by-channel weighting.
[0019] Furthermore, the feedforward network improves the nonlinearity of feature representation through two layers of linear transformation, activation function, and normalization operations, while extracting more advanced and comprehensive features. First, the input features are mapped to a higher-dimensional space through a linear transformation. Next, the GELU activation function introduces nonlinear elements, allowing the model to capture more complex feature representations. In addition, global response normalization is used for channel normalization to adjust the contribution of each channel feature. Finally, a linear transformation is used to map the high-dimensional space features to the original dimension. At the same time, batch normalization is used to normalize the data, which helps to reduce covariate shift during training.
[0020] Furthermore, the adaptive downsampling module consists of a weight branch and a downsampling branch. First, the spatial weight of each pixel position in the input feature map is calculated through average pooling and convolution operations, enabling the model to adaptively assign weights to the input features. The obtained feature weights need to be rearranged for element-by-element multiplication with the downsampled feature map. The specific feature rearrangement operation refers to extracting each 2×2 region of the feature map into a separate set of information. Each 2×2 region corresponds to 4 elements, one from each of the 4 pixels within the region (top left, top right, bottom left, and bottom right). The four values of each small region are probabilistically distributed to obtain the attention level of each position in the small region, helping the model focus on important small regions in the feature map and more accurately extract small target features. Secondly, the downsampling branch uses grouped convolution to divide the channels of the input feature map into multiple groups. Each group is convolved separately, reducing the spatial dimension and maintaining the correlation between channels, while reducing the number of model parameters and computational cost. Finally, the adaptive weights are element-by-element multiplied with the rearranged downsampled feature map, and the results within each group are weighted summed. This selective aggregation of information ensures that key features can receive higher weights, thereby achieving effective and efficient downsampling operations.
[0021] Furthermore, the scale enhancement detection head adopts the design concept of the decoupling head and consists of four decoupling heads. First, each decoupling head is responsible for learning to extract specific scale and semantic information. By processing the feature map processed by the neck network, it can predict targets of small, medium and large scales. At the same time, the high-resolution feature map is input into the detection head to obtain a feature map specifically for predicting tiny targets. Secondly, shared convolution is used to extract features and reduce the number of model parameters. The features after shared convolution are used to predict the position of the bounding box and the probability of the target category respectively. These predicted values will be used to generate the final detection results. Finally, a scaling operation is applied to the feature map in the regression branch to match the scale of the detection result.
[0022] Furthermore, the loss function consists of two parts: classification loss and regression loss; the classification loss adopts the binary cross entropy loss function; the regression loss adopts the Distribution Focal loss and the NWD loss function.
[0023] Beneficial effects of the present invention:
[0024] The present invention proposes a remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head. The model includes a feature extraction network, a feature fusion network and a prediction network. The present invention constructs a dynamic feature extraction module to flexibly adjust the receptive field of the feature extraction network to obtain the surrounding environment information of small targets. The present invention designs an adaptive downsampling module to emphasize the importance of different areas in the feature map and retain key information to achieve efficient and adaptive downsampling. The present invention proposes a scale enhancement detection head, which inputs high-resolution feature maps into the detection head to improve sensitivity to small targets and reduces the number of model parameters by extracting features through shared convolution. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.
[0026] Figure 1 is an overall flow chart of an embodiment of the present invention;
[0027] Figure 2 This is a structural diagram of a dynamic feature extraction module in an embodiment of the present invention;
[0028] Figure 3 This is a schematic diagram of the expansion parameter module structure in an embodiment of the present invention;
[0029] Figure 4 Schematic diagram of the structure of channel attention in an embodiment of the present invention;
[0030] Figure 5 Schematic diagram of the structure of a feedforward network in an embodiment of the present invention;
[0031] Figure 6 Schematic diagram of the structure of the adaptive downsampling module in an embodiment of the present invention;
[0032] Figure 7 Schematic diagram of the structure of the scale enhancement detection head in an embodiment of the present invention. DETAILED DESCRIPTION
[0033] The technical solutions in the embodiments of the present invention will be further described below with reference to the accompanying drawings in the embodiments of the present invention, but are not intended to limit the present invention.
[0034] like Figure 1 As shown, the present invention provides a remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head, the steps are as follows:
[0035] Step 1: Obtain remote sensing image target detection dataset;
[0036] Step 2: Preprocess the remote sensing image dataset;
[0037] Step 3: Establish a remote sensing image target detection model;
[0038] Step 4: Use the training set of the data set to train the model, construct a loss function to update the model parameters, and obtain the optimal model;
[0039] Step 5: Use the optimal model to detect the test set images of the dataset to obtain the final test results.
[0040] Furthermore, in step 2, the AI-TOD remote sensing dataset was cropped into 800×800 pixel blocks with an overlap of 200 pixels. If the original image was smaller than 800×800, it was padded with zeros. Furthermore, the label file was converted to txt format.
[0041] Furthermore, the step 3 establishes a remote sensing image target detection model: in the feature extraction stage, a dynamic feature extraction module is used to flexibly adjust the receptive field of the feature extraction network to obtain contextual information of small targets and generate high-quality feature representations; channel attention is used to weight the channels so that the model emphasizes useful features, thereby improving the quality of features generated by the model; a feedforward network is used to enhance the nonlinearity of feature representation while extracting more advanced and complex features; in the feature fusion stage, an adaptive downsampling module is used to emphasize the importance of different regions of features and retain key information to achieve efficient and adaptive downsampling; in the prediction stage, a scale-enhanced detection head with a decoupled design is used to extract features through shared convolution to provide the best feature representation for classification and regression tasks and reduce the number of parameters and computational complexity of the model; a high-resolution feature map is input into the detection head to obtain a feature map specifically for predicting small targets; and sensitivity to small targets is improved.
[0042] like Figure 2 As shown, the dynamic feature extraction module uses 1×1 convolution to divide the channel dimension into two branches. One branch is directly passed to the output, while the other branch is processed by the dynamic perception unit. The dynamic perception unit consists of a dilation re-parameterization module, channel attention, and a feedforward network. The features of the two branches are then concatenated. Finally, the number of channels is adjusted through 1×1 convolution to obtain the output. The implementation definition process of this module is as follows:
[0043] Branch1=Res(Conv(x))
[0044] Branch2 = Dsu(Conv(x))
[0045] Output=Conv(concat(Branch1,Branch2))
[0046] Where Branch1 and Branch2 represent the results of the two branches respectively, and Dsu represents the dynamic perception unit.
[0047] like Figure 3 As shown, the expansion re-parameter module uses multiple expansion convolutions to enhance the effect of a large kernel standard convolution. The expansion re-parameter module structure is divided into two types, a and b. During the training phase, a is composed of three parallel branches, and the kernel sizes in the convolution layer are {5, 3, 3}, and the void rates are {1, 2, 3}. b is composed of five parallel branches, and the convolution kernels are {5, 7, 3, 3, 3}, and the void rates are {1, 2, 3, 4, 5}. After the convolution layer, their outputs are dimensionally spliced after the normalization layer. When splicing dimensions, the feature map size is made consistent by padding size. The calculation formula for padding is as follows:
[0048]
[0049] Where k represents the size of the convolution kernel, and r represents the dilation ratio. During the inference phase, the normalization layer is fused into the convolution layer using a structural reparameterization method to improve inference speed. The size of the fused convolution kernel is related to the size of the expanded convolution kernel in each branch, and the largest convolution kernel is taken as the size of the fused convolution kernel. The formula for calculating the expanded convolution kernel size is:
[0050] K=r*(k-1)+1
[0051] like Figure 4 As shown in the figure, the channel attention makes the model emphasize useful features by weighting the channels, thereby improving the quality of the features generated by the model. First, input the feature map X∈R C×H×W Through global average pooling, space compression is performed to obtain the new feature Z∈R C×1×1 Then, through a gating mechanism consisting of two fully connected layers and an activation function, weights are generated for each feature channel. Finally, the weights are applied to the original features through channel-by-channel weighting. The process is defined as:
[0052]
[0053] Where GAP and FC represent global average pooling and fully connected layer operations respectively. Represents a weighted operation.
[0054] like Figure 5As shown in Figure 1, the feedforward network operates through two layers of linear transformation, activation function, and normalization. First, the input features are mapped to a higher-dimensional space through a linear transformation. Next, the channel normalization is performed through the GELU activation function and global response normalization to adjust the contribution of each channel feature. Finally, the high-dimensional space features are mapped back to the original dimension through a linear transformation. This process is defined as:
[0055] Output FF =Linear(GRN(GELU(Linear(x))))
[0056] Where Linear represents linear change, and GRN represents the global response normalization operation.
[0057] like Figure 6 As shown, the adaptive downsampling module consists of a weight branch and a downsampling branch. First, the spatial weight of each pixel position in the input feature map is calculated through average pooling and convolution operations, and rearranged so that it can be multiplied element-by-element with the downsampled feature map. The specific feature rearrangement operation refers to extracting each 2×2 area of the feature map into a separate set of information. Each 2×2 area corresponds to 4 elements, which come from the 4 pixels in the area (upper left, upper right, lower left, and lower right). The four values of each small area are probabilistically distributed to obtain the attention of each position in the small area, helping the model focus on important small areas in the feature map and extract small target features more accurately. Secondly, the downsampling branch uses grouped convolution to divide the channels of the input feature map into multiple groups. Each group is convolved separately to reduce the spatial dimension and maintain the correlation between channels, while reducing the number of parameters and computational cost of the model. Finally, the adaptive weight is multiplied element-by-element with the rearranged downsampled feature map, and the results in each group are weighted summed.
[0058] like Figure 7 As shown in the figure, the scale enhancement detection head adopts the design concept of the decoupling head and consists of four decoupling heads. First, a 1×1 convolution is used to adjust the number of channels. Next, the feature map is passed through two shared 3×3 convolutions to extract the relevant features of the bounding box and category prediction, thereby obtaining richer features and reducing the number of model parameters. Subsequently, the feature map is convolved to convert it into the prediction of the bounding box and category probability required by the model, where the regression branch performs a scaling operation on the feature map to improve the robustness of the model. Finally, the rescaled feature map is spliced with the feature map processed by the classification convolution layer in the channel dimension to obtain the final result.
[0059] Furthermore, the loss function consists of two parts: classification loss and regression loss. Classification loss represents the difference between the predicted category and the true label, helping the model correctly classify the object; while regression loss mainly measures the error between the predicted box and the true box, helping the model accurately locate the object.
[0060] The classification loss uses binary cross entropy loss, and multiple binary classifications are superimposed to achieve multi-label classification. The classification loss is defined as follows:
[0061] L cls = -LlogP-(1-L)log(1-P)
[0062] Where L represents the label confidence and P represents the prediction confidence.
[0063] In the regression loss, Distribution Focal Loss is first used to quickly focus the model's predicted position on the value near the label. The calculation formula is as follows:
[0064] DFL(p(i),p(i+1))=-[(y i+1 -y)log(p(i))+(yy i )·log(p(i+1))]
[0065] where p(i) and p(i+1) are given by y i 、y i+1 , y is determined, where y is the label value. Then, to reduce the sensitivity of IoU to small deviations in target position, the NWD loss is used to further refine the target position. This loss function first models the bounding box as a 2D Gaussian distribution and then calculates the similarity using the Gaussian distribution corresponding to the NWD metric. The calculation formula is as follows:
[0066] L NWD =1-NWD(N p ,N g )
[0067] Among them, N p Represents the Gaussian distribution model of the prediction box, N g A Gaussian distribution model representing the ground-truth box.
[0068] The above are specific embodiments of the present invention. It should be noted that the present invention is not limited to the above specific embodiments. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head, characterized in that: The method comprises: Step 1: Obtain remote sensing image target detection dataset; Step 2: Preprocess the remote sensing image dataset; Step 3: Establish a remote sensing image target detection model; In the feature extraction stage, a dynamic feature extraction module is used to flexibly adjust the receptive field of the feature extraction network to obtain contextual information of small targets. The dynamic feature extraction module divides the channel dimension into two branches through 1×1 convolution, one of which is directly passed to the output, and the other is processed by the dynamic perception unit. By setting different void rates, features at different scales are captured, thereby dynamically adjusting the effective receptive field. The dynamic perception unit consists of an expansion re-parameter module, a channel attention, and a feedforward network. The expansion re-parameter module consists of multiple expansion convolutions. The channel attention consists of global average pooling, a fully connected layer, and an activation function. The feedforward network consists of a linear change operation, an activation function, and a normalization operation. Then, the features of the two branches are spliced. Finally, the number of channels is adjusted through 1×1 convolution to obtain the output. During the feature fusion stage, an adaptive downsampling module is used to emphasize the importance of different regions in the feature map and retain the key information features of small targets to achieve efficient and adaptive downsampling. The adaptive downsampling module consists of a weight branch and a downsampling branch. The weight branch requires feature rearrangement to extract each 2×2 region of the feature map into a separate set of information. Each 2×2 region corresponds to four elements, which come from the top left, top right, bottom left, and bottom right pixels in the region. The four values of each small region are probabilistically distributed to obtain the attention level of each position in the small region, helping the model focus on important regions in the feature map and more accurately extract small target features. During the prediction phase, the scale-enhanced detection head adopts a decoupled head design concept. It consists of four decoupled heads capable of predicting tiny, small, medium, and large objects. At the same time, shared convolution is used to extract features from classification and regression tasks to predict bounding box locations and object category probabilities, reducing the number of model parameters. Step 4: Construct the target detection loss function and perform iterative training; Step 5: Use the trained weights for the test set and output the detection results.
2. The method for remote sensing image target detection based on adaptive downsampling and scale enhancement detection head according to claim 1, characterized in that: The AI-TOD remote sensing dataset was cropped into 800 × 800 pixel blocks with an overlap of 200 pixels; If the original image is smaller than 800×800, fill it with zeros; In addition, convert the label file to txt format.
3. The method for remote sensing image target detection based on adaptive downsampling and scale enhancement detection head according to claim 1, characterized in that: The dilation re-parameterization module uses multiple dilated convolutions to enhance the effect of a large kernel standard convolution. During the training phase, the module uses different dilation rates to capture features of different scales, increase the receptive field, and reduce the overlap of feature maps. During the inference phase, the normalization layer is integrated into the convolution layer using a structural reparameterization method, thereby improving the inference speed. The channel attention method enables the model to emphasize useful features by weighting the channels, thereby improving the quality of features generated by the model. The feedforward network retains the spatial structural information of the features through two layers of linear changes, activation functions, and normalization operations.
4. The method for remote sensing image target detection based on adaptive downsampling and scale enhancement detection head according to claim 1, characterized in that: The adaptive downsampling module allows the model to learn the optimal weight distribution through weight branches. Specifically, each 2×2 area corresponds to 4 elements in the area, which come from the upper left corner, upper right corner, lower left corner and lower right corner pixels in the area respectively; these four values are probabilistically distributed to obtain the attention in each area, which helps the model focus on important detail information in the feature map and more accurately extract small target features; secondly, the downsampling branch adopts grouped convolution to reduce the spatial dimension and maintain the correlation between channels, while reducing the number of parameters and computational cost of the model; finally, the adaptive weight is weighted with the downsampled feature map; this selectively aggregates information to ensure that key features receive higher weights, thereby achieving effective and efficient downsampling operations.
5. The method for remote sensing image target detection based on adaptive downsampling and scale enhancement detection head according to claim 1, characterized in that: The scale-enhanced detection head inputs the high-resolution feature map into the detection head to obtain a feature map specifically used to predict small targets. Secondly, to reduce the number of model parameters, features are extracted through shared convolution. The features after shared convolution are used to predict the position of the bounding box and the probability of the target category respectively. These predicted values will be used to generate the final detection results.
6. The method for remote sensing image target detection based on adaptive downsampling and scale enhancement detection head according to claim 1, characterized in that: The loss function consists of two parts Composition: classification loss and regression loss; The classification loss uses the binary cross entropy loss function; the regression loss uses the Distribution Focal loss and NWD loss functions.