Remote sensing target detection method and device, equipment and storage medium

Abstract

The invention provides a remote sensing target detection method and device, equipment and a storage medium. Relates to the technical field of remote sensing image processing. The method comprises the following steps: preprocessing a remote sensing image data set, inputting the preprocessed remote sensing image data set into a backbone network, and carrying out feature extraction in i stages to obtain features with different resolutions and dimensions; for each stage of the i stages, long-range dependence and large-scale features in the remote sensing image are captured through a large kernel convolution module; for the features extracted by the large kernel convolution, gate control signals and value signals are calculated through a multi-order gate control aggregation module, and the features of different scales are dynamically weighted and aggregated; and for features extracted through large kernel convolution and extended convolution, channel weights are calculated by using average pooling and maximum pooling operations, and an output feature map of the backbone network is obtained. Through the multi-order gating aggregation module, adaptive aggregation of different-order feature information is realized, redundant information is effectively reduced, and the expression ability of multi-order features is enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a remote sensing target detection method, device, equipment and storage medium. Background Art

[0002] Remote sensing image target detection technology is one of the important applications in the field of remote sensing, and is widely used in military defense, agricultural management, intelligent transportation and other fields. With the continuous improvement of remote sensing image resolution, the complexity of target detection tasks has also greatly increased. The existing convolutional neural network (CNN) faces many challenges when processing remote sensing images, especially in the accurate recognition of targets in complex backgrounds, the capture of local details and the integration of global context information.

[0003] Convolutional neural networks (CNNs) have long dominated image classification and target detection tasks, extracting local features through convolution operations and aggregating contextual information layer by layer. However, when processing remote sensing images, traditional convolutional neural networks often have the following two major shortcomings, especially in high-resolution and complex scenes: first, the ability to fuse local and global information is limited; second, the computational overhead is high when processing high-resolution images. Due to the multi-scale, ever-changing targets and complex backgrounds of remote sensing images, traditional CNNs are difficult to effectively fuse global and local information in images, which affects detection accuracy and efficiency.

[0004] In order to overcome these problems, in recent years, the Visual Transformer (ViT) has been proposed and has quickly achieved good results in tasks such as image classification and object detection. ViT achieves direct interaction of global information through the self-attention mechanism, breaking through the limitations of traditional CNN in global information aggregation. Although ViT has shown excellent performance in some tasks, its computational complexity is high and the modeling of local features in the image is relatively weak, which makes it still face certain challenges in processing remote sensing images, especially remote sensing images with a large number of local details and complex backgrounds.

[0005] Defects and shortcomings of the existing technology:

[0006] 1. Limited ability to fuse local information with global information: Although traditional convolutional neural networks have strong local feature extraction capabilities, their ability to capture global contextual information is relatively weak, especially in remote sensing images, where targets may be widely distributed and of different scales. This causes convolutional neural networks to easily ignore global information when detecting dense or small targets, resulting in low detection accuracy.

[0007] 2. High computational overhead and redundant parameters: When processing high-resolution remote sensing images, traditional convolutional neural networks usually require large model parameters to capture rich feature information, resulting in huge computational overhead, long training and inference time, and prone to overfitting problems. These problems limit the application of convolutional neural networks in remote sensing image target detection. Summary of the invention

[0008] The present application provides a remote sensing target detection method, apparatus, equipment and storage medium, aiming to reduce the number of parameters and the amount of calculation of the remote sensing image target detection model in complex scenes. Through a multi-order gated aggregation module, adaptive aggregation of feature information of different orders is achieved, which effectively reduces redundant information and enhances the expression capability of multi-order features.

[0009] In a first aspect, the present application provides a remote sensing target detection method, comprising:

[0010] Construct a large-core convolution multi-order gated aggregation model; wherein the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid and an object detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0011] After preprocessing, the remote sensing image data set is input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions; wherein:

[0012] For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image;

[0013] Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0014] The average pooling and maximum pooling layers respond to the features of different scales of the input to obtain an output feature map of the backbone network;

[0015] Use feature pyramid operations on the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Conduct model training and obtain model parameters;

[0016] The large kernel convolution multi-order gated aggregation model is configured with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0017] In a possible design, the remote sensing image data set is preprocessed and then input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions, including:

[0018] Preprocess the remote sensing images of the remote sensing image dataset. Separate the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where H and W represent the height and width of the segmented remote sensing images, respectively. Use the scale factors rates = 0.5, 1.0, 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset.

[0019] The labels of the remote sensing images are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0020] Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed in the backbone network for i stages. When i=4, the relationship between the resolution and dimension for the i-th stage is as follows:

[0021] First stage:

[0022] Second stage:

[0023] The third stage:

[0024] Fourth stage:

[0025] Where C1, C2, C3, and C4 represent the dimensions of the first stage, the second stage, the third stage, and the fourth stage, respectively.

[0026] In a possible design, the large kernel convolution module extracts long-range dependency information of the remote sensing image in response to the input remote sensing image, connects each pixel with the surrounding receptive field, and obtains long-range dependency and large-scale features in the remote sensing image, including:

[0027] The input remote sensing image X is subjected to preliminary feature extraction using the first large kernel convolution Conv1 and the second large kernel convolution Conv2 of different sizes, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied:

[0028] k i-1 ≤k i ; d1=1,d i-1 <d i (5)

[0029] In the formula, k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, and d1 represents the kernel size of the 1st stage;

[0030] The receptive field is calculated using the following formula:

[0031] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0032] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage;

[0033] Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as:

[0034]

[0035]

[0036] In the formula, Represents the feature map after processing by the first large kernel convolution Conv1, Indicates the size of the convolution kernel in the i-th stage of the first large kernel convolution, represents the expansion rate of the first large kernel convolution in the i-th stage, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Represents the size of the convolution kernel of the second largest kernel convolution in the i-th stage, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0037] In a possible design, based on long-range dependencies and large-scale features, N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation, including:

[0038] The gating signal g is calculated by the following formula:

[0039]

[0040] In the formula, SiLU represents the SiLU activation function, Conv g Represents a 1×1 convolution;

[0041] For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as:

[0042] In the formula, C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 represents the second channel, split1 represents the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel;

[0043] Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as:

[0044]

[0045]

[0046]

[0047] x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0048] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, Conv represents convolution, Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels;

[0049] Aggregation results x for different channels multi-order The calculated value signal v is generated by the SiLU activation function to determine the importance of each position in the feature map. The calculation process is expressed as:

[0050] v=SiLU(x multi-order ) (15)

[0051] Gating operation The gating signal g and the value signal v are multiplied element by element after being processed by the activation function. The gating signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed to obtain features of different scales. The calculation process is expressed as:

[0052] out=g×v (16)

[0053] Wherein, out represents the second feature map, and the second feature map is used to characterize features of different scales.

[0054] In a possible design, the average pooling and maximum pooling layers respond to input features of different scales to obtain an output feature map of the backbone network, including:

[0055] Calculate the average pooling avg_attn and the maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, so as to capture the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as:

[0056] avg_out=mean(attn,dim=1) (17)

[0057] max_out=max(attn,dim=1) (18)

[0058] In the formula, avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result;

[0059] The average pooling result and the maximum pooling result are concatenated in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as:

[0060] attn=avg_out⊕max_out (19)

[0061] In the formula, ⊕ represents splicing.

[0062] In one possible design, the output feature map attn is operated using a feature pyramid to obtain multi-scale features through the following formula:

[0063] F=FPN(attn) (20)

[0064] In the formula, F represents multi-scale features and FPN represents feature pyramid operation.

[0065] In a second aspect, the present application provides a remote sensing target detection device, the device comprising a model building module, a model training module and a model configuration module;

[0066] The model building module is used to build a large-core convolution multi-order gated aggregation model; wherein the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0067] The model training module is used to input the remote sensing image data set into the backbone network after preprocessing, and extract features through i stages to obtain features of different resolutions and dimensions; wherein the model training module includes:

[0068] The first feature extraction unit is used for each stage of the i stages. The large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image;

[0069] The second feature extraction unit is used to extract features based on long-range dependencies and large-scale features through N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0070] A third feature extraction unit is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network;

[0071] Feature pyramid operation unit, used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Conduct model training and obtain model parameters;

[0072] The model configuration module is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0073] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the remote sensing target detection method described in the first aspect and various possible designs of the first aspect.

[0074] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the remote sensing target detection method described in the first aspect and various possible designs of the first aspect is implemented.

[0075] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the remote sensing target detection method described in the first aspect and various possible designs of the first aspect.

[0076] The remote sensing target detection method, device, equipment and storage medium provided in this application can effectively integrate local and global information and significantly optimize computing efficiency, and are suitable for complex remote sensing image scenes. The main advantages are summarized as follows:

[0077] 1. Adaptive context aggregation to improve feature utilization efficiency:

[0078] This application designs a compact spatial aggregation module and channel aggregation module, which significantly improves the efficiency of feature integration through local perception and gated context aggregation mechanism. It optimizes information redundancy from the two dimensions of space and channel, and further explores the neglected potential interactions. Compared with traditional methods, this application achieves higher expressiveness and parameter efficiency under the same computational budget.

[0079] 2. Optimize computing efficiency and reduce model redundancy and overhead:

[0080] This application significantly reduces the computational complexity and number of parameters of the model through lightweight design and gated dynamic aggregation mechanism. Compared with the high computational cost of traditional methods in high-resolution remote sensing image processing, this application reduces resource consumption through multiple large-core deep separable convolutions and optimized feature aggregation strategies, making the model perform well in inference speed and video memory usage. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0082] Figure 1 A flowchart of a remote sensing target detection method provided in an embodiment of the present application;

[0083] Figure 2 The architecture diagram of the large-kernel convolution multi-order gated aggregation model provided in the embodiment of the present application;

[0084] Figure 3 A schematic diagram of the structure of a remote sensing target detection device provided in an embodiment of the present application.

[0085] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0088] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0089] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] An embodiment of the present application provides a remote sensing target detection method. Figure 1 A flowchart of a remote sensing target detection method provided in an embodiment of the present application. Figure 1 As shown, the remote sensing target detection method starts with step S10, building a large kernel convolution multi-order gated aggregation model.

[0091] In this embodiment, the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid, and a target detection head connected in sequence; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers.

[0092] Figure 2 The architecture diagram of the large-kernel convolution multi-order gated aggregation model provided in an embodiment of the present application, which is based on the input remote sensing image and uses a backbone network to perform feature extraction to obtain an output feature map, which is further input into the feature pyramid and target detection head to ultimately identify targets in the remote sensing image.

[0093] Step S20: After preprocessing, the remote sensing image data set is input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions.

[0094] In some embodiments, the remote sensing image data set is preprocessed and then input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions. The detailed steps are as follows:

[0095] Step S201, preprocessing the samples of the training data set, first setting the overlapping pixel distance gap between the two segmented images of the training sample image to 200, the resolution of the segmented image subsize to H×W to 1024*1024, and using the scale factors rates to obtain multi-scale training images as the data set;

[0096] Step S202: Arrange the training sample labels into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0097] Step S203: Input the sorted data set into the backbone network to extract features and gradually reduce the resolution H×W and channel dimension C. i In the backbone network, feature extraction is performed at stage i. The relationship between resolution and dimension for the i-th stage is as follows:

[0098] First stage:

[0099] Second stage:

[0100] The third stage:

[0101] Fourth stage:

[0102] Where C1, C2, C3, and C4 represent the dimensions of the first stage, the second stage, the third stage, and the fourth stage, respectively.

[0103] like Figure 1 As shown, step S20 is the process of feature extraction by the backbone network, and step S20 includes:

[0104] Step S21: For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image.

[0105] In some embodiments, the detailed steps of step S21 are as follows:

[0106] Step S211: Perform preliminary feature extraction on the input image X using two large kernel convolutions Conv1 and Conv2 of different sizes, where the expansion rate and kernel size of the i-th stage are k and i and d i In order to capture larger scale features layer by layer, k i and d i The relationship in formula (5) needs to be satisfied:

[0107] k i-1 ≤k i ; d1=1,d i-1 <d i (5)

[0108] In the formula, k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, and d1 represents the kernel size of the 1st stage.

[0109] Step S212: By combining two smaller large kernel convolutions, it is equivalent to obtaining a receptive field RF that is similar to the original large convolution kernel without significantly increasing the number of parameters and computational complexity. The calculation formula is as follows:

[0110] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0111] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage.

[0112] Step S213: Through two large kernel convolutions Conv1 and Conv2, a wider range of context information is captured for the input remote sensing image X, and the processed feature map is output. The process can be represented by equations (7) and (8):

[0113]

[0114]

[0115] In the formula, Represents the feature map after processing by the first large kernel convolution Conv1, Indicates the size of the convolution kernel in the i-th stage of the first large kernel convolution, represents the expansion rate of the first large kernel convolution in the i-th stage, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Represents the size of the convolution kernel of the second largest kernel convolution in the i-th stage, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0116] Step S22: Based on long-range dependency and large-scale features, N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v using element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation.

[0117] In some embodiments, the detailed steps of step S22 are as follows:

[0118] Step S221, calculate the gate signal g, and convolve the features output by the large kernel Through 1×1 convolution, a gate is generated to control the activation of certain feature channels, thereby realizing selective processing of input features. Further weighted aggregation is performed through the gating mechanism to weight features of different scales, and nonlinear transformation is performed on the projected features to better extract features. The process can be represented as formula (9):

[0119]

[0120] In the formula, SiLU represents the SiLU activation function, Conv g Represents a 1×1 convolution.

[0121] Step S222: The features output by the large kernel convolution Through k×k depth-wise separable convolution Conv v Convolution, through different expansion rates d i and channel split i Improve the ability to perceive target diversity. Split through different channel split ratios i The characteristics Divided into three channels C split1 , C split2 , C split3 , then the process can be expressed as formula (10):

[0122]

[0123] In the formula, C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 represents the second channel, split1 represents the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel.

[0124] Step S223: for different channels C split1 , C split2 , C split3 Using different depth separable convolution Conv DW , and through 1×1 convolution fusion, the process can be characterized as follows:

[0125]

[0126]

[0127]

[0128] x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0129] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, Conv represents convolution, Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels.

[0130] Step S224: Aggregation results x of different channels multi-order The calculated value signal v is generated by the SiLU activation function to determine the importance of each position in the feature map, indicating the feature content to be aggregated. The process can be represented by formula (15):

[0131] v=SiLU(x multi-order ) (15)

[0132] Step S225, gate operation The gate signal g and the value signal v are multiplied element by element after being processed by the activation function. Here, the gate signal g controls the degree of passing of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed and the weighted features are obtained. The process can be represented by formula (16):

[0133] out=g×v (16)

[0134] Wherein, out represents the second feature map, and the second feature map is used to characterize features of different scales.

[0135] Step S23, the average pooling and maximum pooling layers respond to the features of different scales of the input to obtain the output feature map of the backbone network.

[0136] In some embodiments, the detailed steps of step S23 are as follows:

[0137] Step S231, calculate the average pooling avg_attn and the maximum pooling max_attn of the channel dimension, extract the information of the feature map in the channel dimension, and capture the significant pattern in the feature map. Based on the output of the model, the process can be characterized as equations (17) and (18):

[0138] avg_out=mean(attn,dim=1) (17)

[0139] max_out=max(attn,dim=1) (18)

[0140] In the formula, avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result.

[0141] Step S232, concatenate the average pooling result and the maximum pooling result in the channel dimension, aggregate the information from different pooling methods, and enhance the diversity of feature maps. The process can be represented by formula (19):

[0142] attn=avg_out⊕max_out (19)

[0143] In the formula, ⊕ represents splicing.

[0144] Step S30: Use feature pyramid operation on the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Perform model training and obtain model parameters.

[0145] In some implementations, the detailed steps of step S30 are as follows:

[0146] Step S31: Use feature pyramid operation on the feature map attn to obtain multi-scale features. The process can be represented by formula (20):

[0147] F=FPN(attn) (20)

[0148] In the formula, F represents multi-scale features and FPN represents feature pyramid operation.

[0149] Step S32: Input the multi-scale features output by the feature pyramid into the model detection head, and use the classification loss Regression Loss The model parameters obtained by training the model based on large kernel convolution and multi-order gated aggregation are denoted as W.

[0150] Finally, in step S40, a large kernel convolution multi-order gated aggregation model is configured with model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0151] The present application also provides a remote sensing target detection device, such as Figure 3 As shown, the remote sensing target detection device includes a model building module 310, a model training module 320 and a model configuration module 330;

[0152] The model building module 310 is used to build a large-core convolution multi-order gated aggregation model; wherein the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers;

[0153] The model training module 320 is used to input the remote sensing image data set into the backbone network after preprocessing, and extract features through i stages to obtain features of different resolutions and dimensions; wherein, the model training module 320 includes:

[0154] The first feature extraction unit 321 is used for extracting long-range dependency information of the remote sensing image in response to the input remote sensing image for each stage of the i stages, and connecting each pixel with the surrounding receptive field to obtain long-range dependency and large-scale features in the remote sensing image;

[0155] The second feature extraction unit 322 is used to extract N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation;

[0156] A third feature extraction unit 323 is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network;

[0157] The feature pyramid operation unit 324 is used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Conduct model training and obtain model parameters;

[0158] The model configuration module 330 is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

[0159] In some embodiments, the model training module is further used to:

[0160] Preprocess the remote sensing images of the remote sensing image dataset. Separate the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where H and W represent the height and width of the segmented remote sensing images, respectively. Use the scale factors rates = 0.5, 1.0, 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset.

[0161] The labels of the remote sensing images are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target;

[0162] Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed in the backbone network for i stages. When i=4, the relationship between the resolution and dimension for the i-th stage is as follows:

[0163] First stage:

[0164] Second stage:

[0165] The third stage:

[0166] Fourth stage:

[0167] Where C1, C2, C3, and C4 represent the dimensions of the first stage, the second stage, the third stage, and the fourth stage, respectively.

[0168] In some embodiments, the first feature extraction unit is used to:

[0169] The input remote sensing image X is subjected to preliminary feature extraction using the first large kernel convolution Conv1 and the second large kernel convolution Conv2 of different sizes, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied:

[0170] k i-1 ≤k i ; d1=1,d i-1 <d i (5)

[0171] In the formula, k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, and d1 represents the kernel size of the 1st stage;

[0172] The receptive field is calculated using the following formula:

[0173] RF1=k1,RF i =d i (k i -1)+RF i-1 (6)

[0174] In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage;

[0175] Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as:

[0176]

[0177]

[0178] In the formula, Represents the feature map after processing by the first large kernel convolution Conv1, Indicates the size of the convolution kernel in the i-th stage of the first large kernel convolution, represents the expansion rate of the first large kernel convolution in the i-th stage, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Represents the size of the convolution kernel of the second largest kernel convolution in the i-th stage, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

[0179] In some embodiments, the second feature extraction unit:

[0180] The gating signal g is calculated by the following formula:

[0181]

[0182] In the formula, SiLU represents the SiLU activation function, Conv g Represents a 1×1 convolution;

[0183] For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as:

[0184] In the formula, C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 represents the second channel, split1 represents the number of copies divided into the second channel, C split3 represents the third channel, and C represents the total channel;

[0185] Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as:

[0186]

[0187]

[0188]

[0189] xmulti-order =Conv 1×1 (concat(x0,x1,x2)) (14)

[0190] In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, Conv represents convolution, Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels;

[0191] Aggregation results x for different channels multi-order The calculated value signal v is generated by the SiLU activation function to determine the importance of each position in the feature map. The calculation process is expressed as:

[0192] v=SiLU(x multi-order ) (15)

[0193] Gating operation The gating signal g and the value signal v are multiplied element by element after being processed by the activation function. The gating signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed to obtain features of different scales. The calculation process is expressed as:

[0194] out=g×v (16)

[0195] Wherein, out represents the second feature map, and the second feature map is used to characterize features of different scales.

[0196] The third feature extraction unit is used for:

[0197] Calculate the average pooling avg_attn and the maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, so as to capture the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as:

[0198] avg_out=mean(attn,dim=1) (17)

[0199] max_out=max(attn,dim=1) (18)

[0200] In the formula, avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result;

[0201] The average pooling result and the maximum pooling result are concatenated in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as:

[0202] attn=avg_out⊕max_out (19)

[0203] In the formula, ⊕ represents splicing.

[0204] In some embodiments, the feature pyramid operation unit is used to apply the feature pyramid operation to the output feature map attn to obtain multi-scale features through the following formula:

[0205] F=FPN(attn) (20)

[0206] In the formula, F represents multi-scale features and FPN represents feature pyramid operation.

[0207] An embodiment of the present application provides an electronic device, which may include: a processor and a memory, wherein the processor and the memory may communicate with each other; illustratively, the processor and the memory communicate with each other via a communication bus.

[0208] The processor executes the computer execution instructions stored in the memory, so that the processor executes the scheme in the above embodiment. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0209] The communication bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0210] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0211] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the remote sensing target detection method of the above embodiment.

[0212] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the remote sensing target detection method in the above embodiment can be implemented.

[0213] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0214] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0215] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0216] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0217] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0218] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0219] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0220] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0221] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0222] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote sensing target detection method, characterized in that: The method comprises: Construct a large-core convolution multi-order gated aggregation model; wherein the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid and an object detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers; After preprocessing, the remote sensing image data set is input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions; wherein: For each of the i stages, the large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image; Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation; The average pooling and maximum pooling layers respond to the features of different scales of the input to obtain an output feature map of the backbone network; Use feature pyramid operations on the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Conduct model training and obtain model parameters; The large kernel convolution multi-order gated aggregation model is configured with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

2. The remote sensing target detection method according to claim 1, characterized in that: After preprocessing, the remote sensing image dataset is input into the backbone network for feature extraction through i stages to obtain features of different resolutions and dimensions, including: Preprocess the remote sensing images of the remote sensing image dataset. Separate the remote sensing images based on the overlapping pixel distance between the two segmented images. The resolution of the segmented images is subsize = H × W, where H and W represent the height and width of the segmented remote sensing images, respectively. Use the scale factors rates = 0.5, 1.0, 1.5 to obtain multi-scale remote sensing images as the preprocessed dataset. The labels of the remote sensing images are organized into the format of (x1, y1, x2, y2, x3, y3, x4, y4, n, d), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the horizontal and vertical coordinates of the four points of the target frame, n represents the category of the target, and d represents the difficulty of distinguishing the target; Input the preprocessed data set into the backbone network to extract features and gradually reduce the resolution and channel dimension C i , feature extraction is performed in the backbone network for i stages. When i=4, the relationship between the resolution and dimension for the i-th stage is as follows: First stage: Second stage: The third stage: Fourth stage: Where C1, C2, C3, and C4 represent the dimensions of the first stage, the second stage, the third stage, and the fourth stage, respectively.

3. The remote sensing target detection method according to claim 1, characterized in that: The large kernel convolution module extracts the long-range dependency information of the remote sensing image in response to the input remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image, including: The input remote sensing image X is subjected to preliminary feature extraction using the first large kernel convolution Conv1 and the second large kernel convolution Conv2 of different sizes, where the expansion rate and kernel size of the i-th stage are k i and d i , k at different stages i and d i The relationship in formula (5) needs to be satisfied: k i-1 ≤k i ;d1=1,d i-1 <d i (5) In the formula, k i-1 represents the expansion rate of the i-1th stage, d i-1 represents the kernel size of the i-1th stage, and d1 represents the kernel size of the 1st stage; The receptive field is calculated using the following formula: RF1=k1,RF i =d i (k i -1)+RF i-1 (6) In the formula, RF1 represents the receptive field of the first stage, k1 represents the expansion rate of the first stage, and RF i represents the receptive field of the i-th stage, RF i-1 represents the receptive field of the i-1th stage; Based on the first large kernel convolution Conv1 and the second large kernel convolution Conv2, the context information of the input remote sensing image X is captured and the first feature map is output. The calculation process is expressed as: In the formula, Represents the feature map after processing by the first large kernel convolution Conv1, Indicates the size of the convolution kernel in the i-th stage of the first large kernel convolution, represents the expansion rate of the first large kernel convolution in the i-th stage, represents a first feature map, which is used to characterize long-range dependencies and large-scale features in remote sensing images. Represents the size of the convolution kernel of the second largest kernel convolution in the i-th stage, Represents the dilation rate of the i-th stage of the second largest kernel convolution.

4. The remote sensing target detection method according to claim 3, characterized in that: Based on long-range dependence and large-scale features, N i A multi-stage gated aggregation module calculates the gate signal g and the value signal The gate signal g and the value signal v are combined by element-by-element multiplication, and features of different scales are obtained by dynamic weighting and aggregation, including: The gating signal g is calculated by the following formula: In the formula, SiLU represents the SiLU activation function, Conv g Represents a 1×1 convolution; For the first feature map Through k×k depth separable convolution, the first feature map is divided into Divided into three channels, the calculation process is expressed as: In the formula, C split1 Indicates the first channel, split total Indicates the total number of channels divided, split0 indicates the number of channels divided into the first channel, C split2 represents the second channel, split1 represents the number of copies divided into the second channel, C split3 represents the third channel, C represents the total channel; Different depth-separable convolutions are applied to the three channels, and the aggregation results of different channels are obtained through convolution fusion. The calculation process is expressed as: x multi-order =Conv 1×1 (concat(x0,x1,x2)) (14) In the formula, x0 represents the feature obtained after convolution of the first channel, x1 represents the feature obtained after convolution of the second channel, x2 represents the feature obtained after convolution of the third channel, Conv represents convolution, Indicates the size of the convolution kernel for processing the first channel feature at the i-th stage, Conv 1×1 Indicates that concat means superimposing the features of the three channels together, x multi-order Represents the aggregation results of different channels; Aggregation results x for different channels multi-order The calculated value signal v is generated by the SiLU activation function to determine the importance of each position in the feature map. The calculation process is expressed as: v=SiLU(x multi-order ) (15) Gating operation The gating signal g and the value signal v are multiplied element by element after being processed by the activation function. The gating signal g controls the degree of passage of the value signal v. If the activation value of g is large, the corresponding features of v are more retained; otherwise, the corresponding features are suppressed to obtain features of different scales. The calculation process is expressed as: out=g×v (16) Wherein, out represents the second feature map, and the second feature map is used to characterize features of different scales.

5. The remote sensing target detection method according to claim 4, characterized in that: The average pooling and maximum pooling layers respond to the features of different scales of the input to obtain the output feature map of the backbone network, including: Calculate the average pooling avg_attn and the maximum pooling max_attn of the channel dimension to extract the information of the second feature map in the channel dimension, so as to capture the significant pattern in the second feature map. Based on the output of the model, the calculation process is expressed as: avg_out=mean(attn,dim=1) (17) max_out=max(attn,dim=1) (18) In the formula, avg_out represents the average pooling result, attn represents the output feature map, dim represents the number of channels, mean represents the average pooling, max_out represents the maximum pooling result, and max represents the maximum pooling result; The average pooling result and the maximum pooling result are concatenated in the channel dimension to aggregate information from different pooling methods to enhance the diversity of the output feature map. The calculation process is expressed as: In the formula, Indicates splicing.

6. The remote sensing target detection method according to claim 5, characterized in that: The output feature map attn is operated using a feature pyramid through the following formula to obtain multi-scale features: F=FPN(attn) (20) Where F represents multi-scale features and FPN represents feature pyramid operation.

7. A remote sensing target detection device, characterized in that: The device includes a model building module, a model training module and a model configuration module; The model building module is used to build a large-core convolution multi-order gated aggregation model; wherein the large-core convolution multi-order gated aggregation model includes a backbone network, a feature pyramid and a target detection head; wherein the backbone network includes a large-core convolution module, N i A multi-stage gated aggregation module and average pooling and maximum pooling layers; The model training module is used to input the remote sensing image data set into the backbone network after preprocessing, and extract features through i stages to obtain features of different resolutions and dimensions; wherein the model training module includes: The first feature extraction unit is used for each stage of the i stages. The large kernel convolution module responds to the input remote sensing image, extracts the long-range dependency information of the remote sensing image, connects each pixel with the surrounding receptive field, and obtains the long-range dependency and large-scale features in the remote sensing image; The second feature extraction unit is used to extract features based on long-range dependencies and large-scale features through N i A multi-stage gated aggregation module calculates the gated signal g and the value signal v, combines the gated signal g and the value signal v by element-by-element multiplication operation, and obtains features of different scales through dynamic weighting and aggregation; A third feature extraction unit is used to input features of different scales into the average pooling and maximum pooling layers to obtain an output feature map of the backbone network; Feature pyramid operation unit, used to apply feature pyramid operation to the output feature map and input it to the target detection head, using the classification loss of the model Regression Loss Perform model training to obtain model parameters; The model configuration module is used to configure the large kernel convolution multi-order gated aggregation model with the model parameters to obtain a remote sensing target detection model, and the remote sensing target detection model is used to realize target detection in remote sensing images.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the remote sensing target detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the remote sensing target detection method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the remote sensing target detection method according to any one of claims 1 to 6 is implemented.

Images