Remote sensing target detection method, device, equipment and storage medium

Abstract

The present application provides a remote sensing target detection method, device, equipment and storage medium. It relates to the field of remote sensing image processing technology. The method includes: the remote sensing image data set is pre-processed and input into the backbone network to perform feature extraction through i stages to obtain features of different resolutions and dimensions; each stage of the i stages first passes through a large kernel convolution module to capture long-range dependencies and large-scale features in the remote sensing image; the features extracted by the large kernel convolution pass through a multi-order gated aggregation module to calculate the gated signal and value signal, dynamically weight and aggregate features of different scales; for the features extracted by the large kernel convolution and the dilated convolution, the channel weights are calculated using average pooling and maximum pooling operations to obtain the output feature map of the backbone network. The present application realizes the adaptive aggregation of feature information of different orders through the multi-order gated aggregation module, effectively reduces redundant information, and enhances the expression ability of multi-order features.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image processing technology, and in particular to a remote sensing target detection method, device, equipment and storage medium. Background Art

[0002] Target detection in remote sensing images is a key application in remote sensing, widely used in fields such as military defense, agricultural management, and intelligent transportation. As the resolution of remote sensing images continues to increase, the complexity of target detection tasks has also greatly increased. Existing convolutional neural networks (CNNs) face numerous challenges when processing remote sensing images, particularly in areas such as accurately identifying targets in complex backgrounds, capturing local details, and integrating global contextual information.

[0003] Convolutional neural networks (CNNs) have long dominated image classification and object detection tasks, extracting local features through convolution operations and aggregating contextual information layer by layer. However, traditional CNNs often suffer from two major limitations when processing remote sensing imagery, especially in high-resolution and complex scenes: first, their ability to fuse local and global information is limited; second, their computational overhead is high when processing high-resolution images. Due to the multi-scale, ever-changing objects, and complex backgrounds in remote sensing images, traditional CNNs struggle to effectively fuse global and local information, thus compromising detection accuracy and efficiency.

[0004] To overcome these problems, the Visual Transformer (ViT) was proposed in recent years and has rapidly achieved promising results in tasks such as image classification and object detection. ViT utilizes a self-attention mechanism to enable direct interaction of global information, overcoming the limitations of traditional CNNs in global information aggregation. Although ViT demonstrates excellent performance in certain tasks, its high computational complexity and weak modeling of local features in images make it challenging to process remote sensing images, especially those with large amounts of local details and complex backgrounds.

[0005] Defects and shortcomings of existing technology:

[0006] 1. Limited ability to fuse local and global information: Although traditional convolutional neural networks have strong local feature extraction capabilities, they are relatively weak in capturing global contextual information. This is especially true in remote sensing images, where targets may be widely distributed and of varying scales. This causes convolutional neural networks to easily ignore global information when detecting dense or small targets, resulting in low detection accuracy.

[0007] 2. High computational overhead and parameter redundancy: Traditional convolutional neural networks typically require large model parameters to capture rich feature information when processing high-resolution remote sensing images. This results in high computational overhead, long training and inference times, and a high tendency to overfitting. These issues limit the application of convolutional neural networks in remote sensing image object detection. Summary of the Invention

[0008] The present application provides a remote sensing target detection method, apparatus, equipment and storage medium, which aims to reduce the number of parameters and computational complexity of remote sensing image target detection models in complex scenes. Through a multi-order gated aggregation module, adaptive aggregation of feature information of different orders is achieved, effectively reducing redundant information and enhancing the expressiveness of multi-order features.

[0009] In a first aspect, the present application provides a remote sensing target detection method, comprising:

[0010] Construct a large-core convolution multi-stage gated aggregation model; wherein, the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein, the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0011] After preprocessing, the remote sensing image dataset is input into the backbone network to perform feature extraction through i stages to obtain features of different resolutions and dimensions; wherein:

[0012] For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image;

[0013] Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-wise multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0014] The average pooling and maximum pooling layers respond to the features of different scales of the input to obtain the output feature map of the backbone network;

[0015] Use feature pyramid operations on the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters;

[0016] The large kernel convolution multi-order gated aggregation model is configured with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0017] In one possible design, the remote sensing image dataset is preprocessed and then input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions, including:

[0018] Preprocess the remote sensing images of the remote sensing image dataset. Segment the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where H and W represent the height and width of the segmented remote sensing images, respectively. Use scale factors rates = 0.5, 1.0, and 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset.

[0019] The labels of the remote sensing image are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0020] Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed at stage i in the backbone network. When i = 4, the relationship between resolution and dimension for the i-th stage is as follows:

[0021] First stage:

[0022] Second stage:

[0023] The third stage:

[0024] The fourth stage:

[0025] Where C1, C2, C3, and C4 represent the dimensions of the first, second, third, and fourth stages, respectively.

[0026] In one possible design, the large kernel convolution module extracts long-range dependency information of the remote sensing image in response to the input remote sensing image, connects each pixel with the surrounding receptive field, and obtains long-range dependency and large-scale features in the remote sensing image, including:

[0027] The input remote sensing image X is convolved with the first large kernel Conv1 and the second large kernel Conv2 of different sizes for preliminary feature extraction, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied:

[0028] k i-1 ≤k i ;d1=1,d i-1 <d i (5)

[0029] Where k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, d1 represents the kernel size of the 1st stage;

[0030] The receptive field is calculated using the following formula:

[0031] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0032] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage;

[0033] Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as:

[0034]

[0035] Where, Represents the feature map after the first large kernel convolution Conv1 processing, Indicates the size of the convolution kernel of the first large kernel convolution stage i, represents the expansion rate of the first large kernel convolution stage i, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Indicates the size of the convolution kernel of the second largest kernel convolution stage i, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0036] In a possible design, based on long-range dependency and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation, including:

[0037] The gating signal g is calculated using the following formula:

[0038]

[0039] In the formula, SiLU represents the SiLU activation function, Conv g Represents 1×1 convolution;

[0040] For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as:

[0041] Where C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 Indicates the second channel, split1 indicates the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel;

[0042] Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as:

[0043]

[0044] x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0045] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, and Conv represents convolution. Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels;

[0046] Aggregation results x for different channels multi-orderThe SiLU activation function is used to generate a calculated value signal v, which is used to determine the importance of each position in the feature map. The calculation process is expressed as:

[0047] v=SiLU(x multi-order ) (15)

[0048] The gate operation gate signal g and the value signal v are processed by the activation function and then multiplied element by element. The gate signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed, and features of different scales are obtained. The calculation process is expressed as:

[0049] out=g×v (16)

[0050] Wherein, out represents the second feature map, which is used to represent features of different scales.

[0051] In one possible design, the average pooling and maximum pooling layers respond to input features of different scales to obtain an output feature map of the backbone network, including:

[0052] Calculate the average pooling avg_attn and maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, thereby capturing the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as:

[0053] avg_out=mean(attn,dim=1) (17)

[0054] max_out=max(attn,dim=1) (18)

[0055] Where avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result;

[0056] The average pooling result and the maximum pooling result are spliced ​​in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as:

[0057] attn=avg_out⊕max_out (19)

[0058] In the formula, ⊕ represents splicing.

[0059] In one possible design, the feature pyramid operation is applied to the output feature map attn to obtain multi-scale features through the following formula:

[0060] F=FPN(attn) (20)

[0061] Where F represents multi-scale features and FPN represents feature pyramid operation.

[0062] In a second aspect, the present application provides a remote sensing target detection device, the device comprising a model construction module, a model training module and a model configuration module;

[0063] The model building module is used to build a large-core convolution multi-stage gated aggregation model; wherein, the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein, the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0064] The model training module is used to input the remote sensing image data set into the backbone network after preprocessing and perform feature extraction through i stages to obtain features of different resolutions and dimensions; wherein, the model training module includes:

[0065] The first feature extraction unit is configured to extract long-range dependency information of the remote sensing image in response to the input remote sensing image in each of the i stages, and associate each pixel with the surrounding receptive field to obtain long-range dependency and large-scale features in the remote sensing image;

[0066] The second feature extraction unit is used to extract N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-wise multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0067] A third feature extraction unit is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network;

[0068] Feature pyramid operation unit, used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters;

[0069] The model configuration module is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0070] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the remote sensing target detection method described in the first aspect and various possible designs of the first aspect.

[0071] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the remote sensing target detection method described in the first aspect and various possible designs of the first aspect is implemented.

[0072] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the remote sensing target detection method described in the first aspect and various possible designs of the first aspect.

[0073] The remote sensing target detection method, apparatus, device, and storage medium provided in this application effectively fuse local and global information and significantly optimize computational efficiency. They are applicable to complex remote sensing image scenes. The main advantages are summarized as follows:

[0074] 1. Adaptive context aggregation to improve feature utilization efficiency:

[0075] This application designs a compact spatial aggregation module and channel aggregation module that significantly improves feature integration efficiency through local perception and gated context aggregation mechanisms. It optimizes information redundancy in both spatial and channel dimensions to further explore overlooked potential interactions. Compared to traditional methods, this application achieves higher expressiveness and parameter efficiency within the same computational budget.

[0076] 2. Optimize computing efficiency and reduce model redundancy and overhead:

[0077] This application significantly reduces the computational complexity and parameter count of the model through lightweight design and gated dynamic aggregation mechanisms. Compared to the high computational cost of traditional methods for high-resolution remote sensing image processing, this application reduces resource consumption through multiple large-kernel depthwise separable convolutions and an optimized feature aggregation strategy, resulting in excellent performance in terms of inference speed and video memory usage. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0079] Figure 1 A flowchart of a remote sensing target detection method provided in an embodiment of the present application;

[0080] Figure 2 This is an architectural diagram of the large-kernel convolution multi-stage gated aggregation model provided in an embodiment of the present application;

[0081] Figure 3 This is a schematic diagram of the structure of the remote sensing target detection device provided in an embodiment of the present application.

[0082] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0084] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0086] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0087] An embodiment of the present application provides a remote sensing target detection method. Figure 1 This is a flow chart of a remote sensing target detection method provided in an embodiment of the present application. Figure 1 As shown, the remote sensing target detection method starts with step S10, building a large kernel convolution multi-order gated aggregation model.

[0088] In this embodiment, the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid, and a target detection head connected in sequence; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers.

[0089] Figure 2 This is an architectural diagram of the large-kernel convolution multi-order gated aggregation model provided in an embodiment of the present application. The large-kernel convolution multi-order gated aggregation model is based on the input remote sensing image and uses a backbone network to perform feature extraction to obtain an output feature map. The output feature map is further input into the feature pyramid and target detection head to ultimately identify the target in the remote sensing image.

[0090] Step S20: After preprocessing, the remote sensing image dataset is input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions.

[0091] In some embodiments, the remote sensing image dataset is preprocessed and then input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions. The detailed steps are as follows:

[0092] Step S201: Preprocess the samples of the training data set. First, set the overlapping pixel distance gap between the two segmented images of the training sample image to 200, the resolution of the segmented image subsize = H × W = 1024 * 1024, and use the scaling factors rates = 0.5, 1.0, 1.5 to obtain multi-scale training images as the data set;

[0093] Step S202: Arrange the training sample labels into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target box, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0094] Step S203: Input the sorted data set into the backbone network to extract features and gradually reduce the resolution H×W and channel dimension C. i The relationship between the resolution and dimension of the i-th stage is as follows:

[0095] First stage:

[0096] Second stage:

[0097] The third stage:

[0098] The fourth stage:

[0099] Where C1, C2, C3, and C4 represent the dimensions of the first, second, third, and fourth stages, respectively.

[0100] like Figure 1 As shown, step S20 is the process of feature extraction by the backbone network, and step S20 includes:

[0101] Step S21: For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image.

[0102] In some embodiments, the detailed steps of step S21 are as follows:

[0103] Step S211: Perform preliminary feature extraction on the input image X using two large kernel convolutions Conv1 and Conv2 of different sizes, where the expansion rate and kernel size of the i-th stage are k and k respectively. i and d i In order to capture larger scale features layer by layer, k i and d i The relationship in formula (5) needs to be satisfied:

[0104] k i-1 ≤k i ;d1=1,d i-1 <d i (5)

[0105] Where k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, and d1 represents the kernel size of the first stage.

[0106] Step S212: By combining two smaller large kernel convolutions, it is equivalent to obtaining a receptive field RF that is similar to the original large convolution kernel without significantly increasing the number of parameters and computational complexity. The calculation formula is as follows:

[0107] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0108] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage.

[0109] Step S213: Through the two convolution kernels Conv1 and Conv2, a wider range of context information is captured for the input remote sensing image X, and the processed feature map is output. The process can be represented by equations (7) and (8):

[0110]

[0111] Where, Represents the feature map after the first large kernel convolution Conv1 processing, Indicates the size of the convolution kernel of the first large kernel convolution stage i, represents the expansion rate of the first large kernel convolution stage i, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Indicates the size of the convolution kernel of the second largest kernel convolution stage i, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0112] Step S22: Based on long-range dependency and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation.

[0113] In some embodiments, the detailed steps of step S22 are as follows:

[0114] Step S221: Calculate the gate signal g, and convolve the features output by the large kernel. Through 1×1 convolution, a gate is generated to control the activation of certain feature channels, thereby realizing selective processing of input features. Further weighted aggregation is performed through the gating mechanism to weight features of different scales, and nonlinear transformation is performed on the projected features to better extract features. The process can be represented as formula (9):

[0115]

[0116] In the formula, SiLU represents the SiLU activation function, Conv g Represents a 1×1 convolution.

[0117] Step S222: The features output by the large kernel convolution Through k×k depth-wise separable convolution Conv v Convolution, by using different dilation rates d i and channel split iImprove the ability to perceive target diversity. Split through different channel split ratios i The features Divided into three channels C split1 、C split2 、C split3 , then the process can be represented as formula (10):

[0118]

[0119] Where C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 Indicates the second channel, split1 indicates the number of copies divided into the second channel, C split3 Indicates the third channel, and C indicates the total channel.

[0120] Step S223: For different channels C split1 、C split2 、C split3 Use different depth separable convolution Conv DW , and through 1×1 convolution fusion, the process can be characterized as follows:

[0121]

[0122] x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0123] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, and Conv represents convolution. Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels.

[0124] Step S224: Aggregation results x of different channels multi-order The calculated value signal v is generated by the SiLU activation function to determine the importance of each position in the feature map and represent the feature content to be aggregated. The process can be represented as formula (15):

[0125] v=SiLU(x multi-order ) (15)

[0126] Step S225: Gating operation: The gate signal g and the value signal v are processed by the activation function and then multiplied element by element. Here, the gate signal g controls the degree of passing of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed and weighted features are obtained. The process can be represented by formula (16):

[0127] out=g×v (16)

[0128] Wherein, out represents the second feature map, which is used to represent features of different scales.

[0129] Step S23: The average pooling and maximum pooling layers respond to the input features of different scales to obtain the output feature map of the backbone network.

[0130] In some embodiments, the detailed steps of step S23 are as follows:

[0131] Step S231, calculate the average pooling avg_attn and maximum pooling max_attn of the channel dimension, extract the information of the feature map in the channel dimension, and capture the significant pattern in the feature map. Based on the output of the model, the process can be represented by equations (17) and (18):

[0132] avg_out=mean(attn,dim=1) (17)

[0133] max_out=max(attn,dim=1) (18)

[0134] Where avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result.

[0135] Step S232: concatenate the average pooling result and the maximum pooling result in the channel dimension, aggregate the information from different pooling methods, and enhance the diversity of feature maps. The process can be represented by formula (19):

[0136] attn=avg_out⊕max_out (19)

[0137] In the formula, ⊕ represents splicing.

[0138] Step S30: Apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters.

[0139] In some implementations, the detailed steps of step S30 are as follows:

[0140] Step S31: Apply feature pyramid operation to the feature map attn to obtain multi-scale features. The process can be represented by formula (20):

[0141] F=FPN(attn) (20)

[0142] Where F represents multi-scale features and FPN represents feature pyramid operation.

[0143] Step S32: Input the multi-scale features output by the feature pyramid into the model detection head, and use the classification loss Regression loss The model parameters obtained by training the model based on large kernel convolution and multi-order gated aggregation are denoted as W.

[0144] Finally, in step S40, a large kernel convolution multi-order gated aggregation model is configured with the model parameters to obtain a remote sensing target detection model, which is used to realize target detection in remote sensing images.

[0145] The present application also provides a remote sensing target detection device, such as Figure 3 As shown, the remote sensing target detection device includes a model building module 310, a model training module 320 and a model configuration module 330;

[0146] The model building module 310 is used to build a large-core convolution multi-stage gated aggregation model; wherein the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0147] The model training module 320 is used to input the remote sensing image data set into the backbone network after preprocessing, and perform feature extraction through i stages to obtain features of different resolutions and dimensions; wherein, the model training module 320 includes:

[0148] The first feature extraction unit 321 is configured to extract long-range dependency information of the remote sensing image in response to the input remote sensing image in each of the i stages, and associate each pixel with the surrounding receptive field to obtain long-range dependency and large-scale features in the remote sensing image;

[0149] The second feature extraction unit 322 is used to extract N iA multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-wise multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0150] A third feature extraction unit 323 is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network;

[0151] Feature pyramid operation unit 324 is used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters;

[0152] The model configuration module 330 is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0153] In some embodiments, the model training module is further configured to:

[0154] Preprocess the remote sensing images of the remote sensing image dataset. Segment the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where h and W represent the height and width of the segmented remote sensing images, respectively. Use scale factors rates = 0.5, 1.0, and 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset.

[0155] The labels of the remote sensing image are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0156] Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed at stage i in the backbone network. When i = 4, the relationship between resolution and dimension for the i-th stage is as follows:

[0157] First stage:

[0158] Second stage:

[0159] The third stage:

[0160] The fourth stage:

[0161] Where C1, C2, C3, and C4 represent the dimensions of the first, second, third, and fourth stages, respectively.

[0162] In some embodiments, the first feature extraction unit is configured to:

[0163] The input remote sensing image X is convolved with the first large kernel Conv1 and the second large kernel Conv2 of different sizes for preliminary feature extraction, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied:

[0164] k i-1 ≤k i ;d1=1,d i-1 <d i (5)

[0165] Where k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, d1 represents the kernel size of the 1st stage;

[0166] The receptive field is calculated using the following formula:

[0167] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0168] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage;

[0169] Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as:

[0170]

[0171] Where, Represents the feature map after the first large kernel convolution Conv1 processing, Indicates the size of the convolution kernel of the first large kernel convolution stage i, represents the expansion rate of the first large kernel convolution stage i, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Indicates the size of the convolution kernel of the second largest kernel convolution stage i, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0172] In some embodiments, the second feature extraction unit:

[0173] The gating signal g is calculated using the following formula:

[0174]

[0175] In the formula, SiLU represents the SiLU activation function, Conv g Represents 1×1 convolution;

[0176] For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as:

[0177] Where C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 Indicates the second channel, split1 indicates the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel;

[0178] Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as:

[0179]

[0180] x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0181] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, and Conv represents convolution. Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels;

[0182] Aggregation results x for different channels multi-order The SiLU activation function is used to generate a calculated value signal v, which is used to determine the importance of each position in the feature map. The calculation process is expressed as:

[0183] v=SiLU(x multi-order ) (15)

[0184] The gate operation gate signal g and the value signal v are processed by the activation function and then multiplied element by element. The gate signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed, and features of different scales are obtained. The calculation process is expressed as:

[0185] out=g×v (16)

[0186] Wherein, out represents the second feature map, which is used to represent features of different scales.

[0187] The third feature extraction unit is used for:

[0188] Calculate the average pooling avg_attn and maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, thereby capturing the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as:

[0189] avg_out=mean(attn,dim=1) (17)

[0190] max_out=max(attn,dim=1) (18)

[0191] Where avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result;

[0192] The average pooling result and the maximum pooling result are spliced ​​in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as:

[0193] attn=avg_out⊕max_out (19)

[0194] In the formula, ⊕ represents splicing.

[0195] In some embodiments, the feature pyramid operation unit is configured to apply a feature pyramid operation to the output feature map attn to obtain multi-scale features using the following formula:

[0196] F=FPN(attn) (20)

[0197] Where F represents multi-scale features and FPN represents feature pyramid operation.

[0198] An embodiment of the present application provides an electronic device, which may include a processor and a memory, wherein the processor and the memory can communicate with each other; illustratively, the processor and the memory communicate with each other via a communication bus.

[0199] The processor executes the computer-executable instructions stored in the memory, so that the processor implements the solutions in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0200] The communication bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, among others. System buses can be divided into address buses, data buses, and control buses. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or type of bus. Transceivers are used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0201] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0202] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the remote sensing target detection method of the above embodiment.

[0203] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the remote sensing target detection method in the above embodiment.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0205] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment based on actual needs.

[0206] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each module may exist physically separately, or two or more modules may be integrated into a single unit. The above-mentioned modules may be implemented in the form of hardware or hardware plus software functional units.

[0207] The above-mentioned integrated module implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0208] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0209] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0210] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0211] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0212] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0213] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote sensing target detection method, characterized in that: The method comprises: Construct a large-core convolution multi-stage gated aggregation model; wherein, the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein, the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers; After preprocessing, the remote sensing image dataset is input into the backbone network to perform feature extraction through i stages to obtain features of different resolutions and dimensions; wherein: For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image; Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-wise multiplication operation, and obtains features of different scales through dynamic weighting and aggregation; The average pooling and maximum pooling layers respond to the features of different scales of the input to obtain the output feature map of the backbone network; Use feature pyramid operations on the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters; Configuring the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, wherein the remote sensing target detection model is used to realize target detection in remote sensing images; The large kernel convolution module extracts the long-range dependency information of the remote sensing image in response to the input remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image, including: The input remote sensing image X is convolved with the first large kernel Conv1 and the second large kernel Conv2 of different sizes for preliminary feature extraction, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied: k i-1 ≤k i ;d1=1,d i-1 <d i (5) Where k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, d1 represents the kernel size of the 1st stage; The receptive field is calculated using the following formula: RF1=k1,RF i =d i (k i -1)+RF i-1 (6) In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage; Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as: Where, Represents the feature map after the first large kernel convolution Conv1 processing, Indicates the size of the convolution kernel of the first large kernel convolution stage i, represents the expansion rate of the first large kernel convolution stage i, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Indicates the size of the convolution kernel of the second largest kernel convolution stage i, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

2. The remote sensing target detection method according to claim 1, characterized in that: After preprocessing, the remote sensing image dataset is input into the backbone network and subjected to feature extraction in i stages to obtain features of different resolutions and dimensions, including: Preprocess the remote sensing images of the remote sensing image dataset. Segment the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where H and W represent the height and width of the segmented remote sensing images, respectively. Use scale factors rates = 0.5, 1.0, and 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset. The labels of the remote sensing image are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target; Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed at stage i in the backbone network. When i = 4, the relationship between resolution and dimension for the i-th stage is as follows: First stage: Second stage: The third stage: The fourth stage: Where C1, C2, C3, and C4 represent the dimensions of the first, second, third, and fourth stages, respectively.

3. The remote sensing target detection method according to claim 2, characterized in that: Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation, including: The gating signal g is calculated using the following formula: In the formula, SiLU represents the SiLU activation function, Conv g Represents 1×1 convolution; For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as: Where C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 Indicates the second channel, split1 indicates the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel; Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as: x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14) In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, and Conv represents convolution. Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels; Aggregation results x for different channels multi-order The SiLU activation function is used to generate a calculated value signal v, which is used to determine the importance of each position in the feature map. The calculation process is expressed as: v=SiLU(x multi-order ) (15) The gate operation gate signal g and the value signal v are processed by the activation function and then multiplied element by element. The gate signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed, and features of different scales are obtained. The calculation process is expressed as: out=g×v (16) Wherein, out represents the second feature map, which is used to represent features of different scales.

4. The remote sensing target detection method according to claim 3, wherein: The average pooling and maximum pooling layers respond to the input features of different scales to obtain the output feature map of the backbone network, including: Calculate the average pooling avg_attn and maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, thereby capturing the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as: avg_out=mean(attn,dim=1) (17) max_out=max(attn,dim=1) (18) Where avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result; The average pooling result and the maximum pooling result are spliced ​​in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as: attn=avg_out⊕max_out (19) In the formula, ⊕ represents splicing.

5. The remote sensing target detection method according to claim 4, characterized in that: The following formula is used to apply feature pyramid operation to the output feature map attn to obtain multi-scale features: F=FPN(attn) (20) Where F represents multi-scale features and FPN represents feature pyramid operation.

6. A remote sensing target detection device, characterized in that: The device includes a model construction module, a model training module and a model configuration module; The model building module is used to build a large-core convolution multi-stage gated aggregation model; wherein, the large-core convolution multi-stage gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein, the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers; The model training module is used to input the remote sensing image data set into the backbone network after preprocessing and perform feature extraction through i stages to obtain features of different resolutions and dimensions; wherein, the model training module includes: The first feature extraction unit is configured to extract long-range dependency information of the remote sensing image in response to the input remote sensing image in each of the i stages, and associate each pixel with the surrounding receptive field to obtain long-range dependency and large-scale features in the remote sensing image; The second feature extraction unit is used to extract N i A multi-stage gated aggregation module calculates the gate signal g and the value signal v, combines the gate signal g and the value signal v using element-wise multiplication operation, and obtains features of different scales through dynamic weighting and aggregation; A third feature extraction unit is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network; Feature pyramid operation unit, used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression loss Perform model training to obtain model parameters; The model configuration module is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images; The first feature extraction unit is used to: The input remote sensing image X is convolved with the first large kernel Conv1 and the second large kernel Conv2 of different sizes for preliminary feature extraction, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied: k i-1 ≤k i ;d1=1,d i-1 <d i (5) Where k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, d1 represents the kernel size of the 1st stage; The receptive field is calculated using the following formula: RF1=k1,RF i =d i (k i -1)+RF i-1 (6) In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage; Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as: Where, Represents the feature map after the first large kernel convolution Conv1 processing, Indicates the size of the convolution kernel of the first large kernel convolution stage i, represents the expansion rate of the first large kernel convolution stage i, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Indicates the size of the convolution kernel of the second largest kernel convolution stage i, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

7. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the remote sensing target detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the remote sensing target detection method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the remote sensing target detection method according to any one of claims 1 to 5.

Images