A training and recognition method for a remote sensing target detection model

Abstract

This invention discloses a training and recognition method for a remote sensing target detection model. The training method involves acquiring a remote sensing classification training set and a remote sensing detection training set; inputting the remote sensing classification training set into a first network framework for pre-training to obtain a pre-trained first network framework; inputting the remote sensing detection training set into the pre-trained first network framework for remote sensing feature extraction to obtain a fused feature set; inputting the fused feature set into a second network framework for remote sensing target detection to obtain target detection information; and performing joint fine-tuning training on the pre-trained first and second network frameworks based on the target detection information to obtain a trained remote sensing target detection model. This method can improve the robustness of the remote sensing target detection model in complex remote sensing scenarios, reduce the required computational resources, and improve the accuracy and efficiency of remote sensing target detection. This invention relates to the field of target detection technology.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a training method and recognition method for a remote sensing target detection model. Background Technology

[0002] In recent years, with the rapid development of deep learning and remote sensing technology, target detection in remote sensing images has received increasing attention.

[0003] Currently, existing deep learning-based remote sensing target detection methods can be broadly divided into two categories. The first category mainly uses the multi-layer convolutional structure of convolutional neural networks (CNNs) to detect remote sensing targets. This type of method has low robustness in dealing with highly complex remote sensing scenes and poor accuracy in remote sensing target detection. The second category mainly uses the self-attention mechanism of Transformer networks to detect remote sensing targets. This type of method has high computational complexity, requires more computational resources to process high-resolution remote sensing images, and has low efficiency in remote sensing target detection.

[0004] Therefore, the problems existing in the current technology still need to be solved and optimized. Summary of the Invention

[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one objective of this invention is to provide a training method and a recognition method for a remote sensing target detection model, wherein the method can improve the robustness of the remote sensing target detection model in the face of complex remote sensing scenarios, reduce the required computing resources, and improve the accuracy and efficiency of remote sensing target detection.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:

[0008] In a first aspect, embodiments of this application provide a method for training a remote sensing target detection model, including:

[0009] Obtain training sets for remote sensing classification and remote sensing detection;

[0010] The remote sensing classification training set is input into the first network framework for pre-training to obtain the pre-trained first network framework.

[0011] The remote sensing detection training set is input into the pre-trained first network framework for remote sensing feature extraction to obtain a fusion feature set. The fusion feature set includes several spatial fusion features of different spatial depths. Each spatial fusion feature is used to record the feature information of remote sensing features of two adjacent feature scales.

[0012] The fused feature set is input into the second network framework for remote sensing target detection to obtain target detection information;

[0013] Based on the target detection information, the pre-trained first network framework and the second network framework are jointly fine-tuned to obtain the trained remote sensing target detection model.

[0014] In addition, the method according to the above embodiments of this application may also have the following additional technical features:

[0015] Furthermore, in one embodiment of this application, obtaining the remote sensing classification training set and the remote sensing detection training set includes:

[0016] Obtain remote sensing classification datasets and remote sensing detection datasets;

[0017] Data augmentation is performed on the remote sensing classification dataset to obtain the remote sensing classification training set.

[0018] The remote sensing detection dataset is augmented to obtain the remote sensing detection training set.

[0019] Furthermore, in one embodiment of this application, the first network framework includes a plurality of cascaded state-space fusion units, and the step of inputting the remote sensing classification training set into the first network framework for pre-training to obtain a pre-trained first network framework includes:

[0020] The remote sensing classification training set is input into the plurality of cascaded state space fusion units for multi-level spatial feature extraction and fusion to obtain target classification fusion features. The target classification fusion features are used to record remote sensing scene feature information and target category feature information corresponding to the remote sensing classification training set.

[0021] Based on the target classification fusion features, the parameters of the first network framework are updated to obtain the pre-trained first network framework.

[0022] Furthermore, in one embodiment of this application, each state space fusion unit includes a state space modeling module and a spatial fusion module. Input features are input into the state space fusion unit for spatial feature extraction and fusion to obtain intermediate classification fusion features, including:

[0023] The input features are input into the state space modeling module to perform state space modeling, thereby obtaining spatial depth modeling features;

[0024] The spatial depth modeling features are input into the spatial fusion module to perform adjacent scale feature fusion, thereby obtaining the intermediate classification fusion features.

[0025] Furthermore, in one embodiment of this application, the step of inputting the input features into the state space modeling module for state space modeling to obtain spatial depth modeling features includes:

[0026] The input features are subjected to bi-branch linear normalization to obtain the first intermediate feature and the second intermediate feature;

[0027] The first intermediate feature is scanned in multiple directions to obtain a scanning feature set, which includes several scanning features, each of which corresponds to a different scanning direction;

[0028] Spatial modeling is performed on the scan feature set to obtain several state space modeling data, each of the state space modeling data corresponding to one scan feature;

[0029] The state-space modeling features are obtained by adding all the state-space modeling data element by element.

[0030] Based on the second intermediate feature and the input feature, the state space modeling feature is multiplied element-wise to obtain the spatial depth modeling feature.

[0031] Furthermore, in one embodiment of this application, the intermediate classification fusion feature includes a first classification fusion feature and a second classification fusion feature. The step of inputting the spatial depth modeling feature into the spatial fusion module for adjacent scale feature fusion to obtain the intermediate classification fusion feature includes:

[0032] The spatial depth modeling features are subjected to first feature channel separation to obtain third and fourth intermediate features;

[0033] Based on the spatial depth modeling features, frequency fusion is performed on the third intermediate features to obtain the first classification fusion feature and the intermediate classification fusion feature;

[0034] Based on the fourth intermediate feature, the intermediate classification fusion feature is subjected to deep convolution and channel concatenation to obtain the second classification fusion feature.

[0035] Secondly, embodiments of this application provide a method for identifying a remote sensing target detection model, including:

[0036] Acquire remote sensing images of the target to be detected;

[0037] The target remote sensing image is input into the trained remote sensing target detection model to perform target detection and obtain the remote sensing target detection result.

[0038] Thirdly, embodiments of this application provide a training system for a remote sensing target detection model, comprising:

[0039] The first processing unit is used to acquire remote sensing classification training sets and remote sensing detection training sets;

[0040] The second processing unit is used to input the remote sensing classification training set into the first network framework for pre-training to obtain the pre-trained first network framework.

[0041] The third processing unit is used to input the remote sensing detection training set into the pre-trained first network framework to extract remote sensing features and obtain a fused feature set. The fused feature set includes several spatial fused features of different spatial depths. Each spatial fused feature is used to record the feature information of remote sensing features of two adjacent feature scales.

[0042] The fourth processing unit is used to input the fused feature set into the second network framework for remote sensing target detection to obtain target detection information;

[0043] The fifth processing unit is used to perform joint fine-tuning training on the pre-trained first network framework and the second network framework based on the target detection information to obtain the trained remote sensing target detection model.

[0044] Fourthly, embodiments of this application also provide an electronic device, including:

[0045] At least one processor;

[0046] At least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0048] Fifthly, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by the processor, is used to implement the above-described method.

[0049] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:

[0050] This application discloses a training method and recognition method for a remote sensing target detection model. The training method involves acquiring a remote sensing classification training set and a remote sensing detection training set; inputting the remote sensing classification training set into a first network framework for pre-training to obtain a pre-trained first network framework; inputting the remote sensing detection training set into the pre-trained first network framework for remote sensing feature extraction to obtain a fused feature set, which includes several spatial fusion features at different spatial depths, each of which records feature information of remote sensing features at two adjacent feature scales; inputting the fused feature set into a second network framework for remote sensing target detection to obtain target detection information; and performing joint fine-tuning training on the pre-trained first network framework and the second network framework based on the target detection information to obtain the trained remote sensing target detection model. This training method, based on pre-training of the first network framework, can improve the first network framework's perception ability of remote sensing scenes and target categories, thereby enhancing the robustness of the remote sensing target detection model in complex remote sensing scenes. Furthermore, this training method extracts spatial fusion features at different spatial depths through the pre-trained first network framework. These spatial fusion features record the feature information of remote sensing features at two adjacent feature scales, which can improve the remote sensing target detection model's extraction of contextual semantic information, enhance the detection ability of small objects, and reduce the computational complexity of remote sensing target detection, thus effectively improving the detection efficiency and accuracy of remote sensing targets. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0052] Figure 1 A flowchart illustrating a training method for a remote sensing target detection model provided in an embodiment of this application;

[0053] Figure 2 A simplified structural diagram of a state-space modeling module provided in an embodiment of this application;

[0054] Figure 3 A simplified structural diagram of a spatial fusion module provided in an embodiment of this application;

[0055] Figure 4 A schematic diagram of the framework of a training system for a remote sensing target detection model provided in an embodiment of this application;

[0056] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Detailed Implementation

[0057] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] Currently, existing deep learning-based remote sensing target detection methods can be broadly divided into two categories. The first category mainly relies on the multi-layer convolutional structure of convolutional neural networks (CNNs) to detect remote sensing targets. Specifically, it extracts local features from remote sensing images through multi-layer convolutional structures. However, since the receptive field of the convolutional layer in the multi-layer convolutional structure is limited by the fixed-size convolutional kernel, CNN networks have difficulty capturing the correlation between objects that are far apart in space and cannot capture long-distance dependent features well. They also have low robustness in the face of highly complex remote sensing scenes of different scales, resulting in poor accuracy in remote sensing target detection.

[0060] Furthermore, the second type of approach mainly relies on the self-attention mechanism of the Transformer network to detect remote sensing targets. Specifically, it uses the self-attention mechanism to globally model the feature correlation in the remote sensing image. However, since the self-attention mechanism requires similarity calculation between each input feature, its computational complexity increases quadratically with the size of the remote sensing image. This results in a large amount of computational resources required for processing high-resolution remote sensing images, leading to low efficiency in remote sensing target detection.

[0061] In view of this, embodiments of the present invention provide a training method and a recognition method for a remote sensing target detection model. The training method is based on pre-training a first network framework, which can improve the first network framework's perception ability of remote sensing scenes and target categories, thereby improving the robustness of the remote sensing target detection model in detecting remote sensing targets in complex remote sensing scenes. Furthermore, this training method extracts spatial fusion features at different spatial depths through the pre-trained first network framework. These spatial fusion features record feature information of remote sensing features at two corresponding adjacent feature scales, which can improve the remote sensing target detection model's extraction of contextual semantic information, enhance the detection ability of small objects, and reduce the computational complexity of remote sensing target detection, thereby effectively improving the detection efficiency and accuracy of remote sensing targets.

[0062] Specifically, this application embodiment uses multiple cascaded state-space fusion units to achieve efficient feature extraction from remote sensing images, enabling the processing of large-scale remote sensing images with lower computational cost, thereby significantly improving detection efficiency. Simultaneously, the proposed spatial fusion module can extract and combine local and global features from different scales, allowing the remote sensing target detection model to adapt to highly complex remote sensing scenarios and efficiently and accurately detect targets. Furthermore, this application embodiment first pre-trains the first network framework using a remote sensing classification training set, a process that effectively improves the robustness of the remote sensing target detection model in handling complex remote sensing scenarios. The pre-trained first network framework possesses preliminary recognition capabilities for multiple target types, helping the remote sensing target detection model better handle remote sensing target detection tasks at different scales and against complex backgrounds in remote sensing images. Subsequently, fine-tuning the remote sensing target detection model using a remote sensing detection training set can further optimize the model's performance on specific target detection tasks.

[0063] Reference Figure 1 In this embodiment of the application, a training method for a remote sensing target detection model includes:

[0064] Step 110: Obtain the remote sensing classification training set and the remote sensing detection training set;

[0065] In this embodiment, several remote sensing images recording remote sensing scene classification information and remote sensing target category information can be acquired and used to determine a remote sensing classification training set. This training set is used for pre-training the first network framework. The remote sensing detection training set can be obtained by acquiring several remote sensing images recording remote sensing target category information and remote sensing target coordinate information. The target coordinate information is used to record the center coordinates of the location box of the remote sensing target in the remote sensing image. This remote sensing detection training set is used for joint training of the first network framework and the second network framework. The specific acquisition method of the training set can be to obtain publicly available remote sensing image datasets from the Internet, or to generate it based on previously acquired remote sensing data. This example is only for illustration.

[0066] In some embodiments, step 110, obtaining the remote sensing classification training set and the remote sensing detection training set, includes:

[0067] A1. Obtain remote sensing classification dataset and remote sensing detection dataset;

[0068] A2. Perform data augmentation on the remote sensing classification dataset to obtain the remote sensing classification training set;

[0069] A3. Perform data augmentation on the remote sensing detection dataset to obtain the remote sensing detection training set.

[0070] In this embodiment of the application, after obtaining the original remote sensing classification dataset and remote sensing detection dataset, step A2 may first unify the size of each remote sensing image in the remote sensing classification dataset. Specifically, the size of each remote sensing image in the remote sensing classification dataset may be set to 224×224. Then, data augmentation may be performed on all remote sensing images of the unified size in the remote sensing classification dataset. Specifically, at least one of the following operations may be performed on each remote sensing image: random cropping, flipping, color distortion, blending, distortion blending, etc., to obtain the remote sensing classification training set.

[0071] It is understandable that step A3 can be at least one of the following data augmentation operations: random cropping, scaling, horizontal flipping, brightness adjustment, etc., for each remote sensing image in the remote sensing detection dataset, so as to obtain the data augmented remote sensing detection dataset; then, by integrating the original remote sensing detection dataset and the data augmented remote sensing detection dataset, the remote sensing detection dataset is obtained.

[0072] Step 120: Input the remote sensing classification training set into the first network framework for pre-training to obtain the pre-trained first network framework;

[0073] In this embodiment, during the pre-training of the first network framework, multiple remote sensing images from a remote sensing classification training set can be acquired. Each remote sensing image includes image data of a remote sensing scene and a target, and a corresponding label is also acquired. This label characterizes the true type of the remote sensing scene and target in the remote sensing image. Then, each remote sensing image and its corresponding label can be used as a set of training data. The input data for the first network framework is the remote sensing image. The first network framework predicts the remote sensing image, and its output data is the remote sensing image recognition result. After obtaining the remote sensing image recognition result output by the first network framework, the accuracy of the prediction by the first network framework can be evaluated based on the remote sensing image recognition result and its corresponding label, thereby updating the model parameters.

[0074] Specifically, for machine learning models, the accuracy of model predictions can be measured by a loss function. The loss function is defined on a single training data point and measures the prediction error of that data point. Specifically, the loss value is determined by the label of the individual training data point and the model's prediction result for that data point. However, in actual training, a training dataset contains many data points. Therefore, a cost function is generally used to measure the overall error of the training dataset. The cost function is defined on the entire training dataset and calculates the average prediction error of all training data points, providing a better measure of the model's prediction performance. For general machine learning models, the aforementioned cost function, plus a regularization term to measure model complexity, serves as the training objective function. Based on this objective function, the loss value of the entire training dataset can be calculated. Many types of loss functions are commonly used, such as 0-1 loss, squared loss, absolute loss, logarithmic loss, and cross-entropy loss, which will not be elaborated upon here. In this embodiment, any one of these loss functions can be selected to determine the training loss value, such as the cross-entropy loss function. Based on the training loss value, the parameters of the first network framework are updated using the backpropagation algorithm. After several iterations, the pre-trained first network framework can be obtained. The specific number of iterations can be preset, or training can be considered complete when the accuracy requirement on the test set is met.

[0075] In some embodiments, step 120, the first network framework includes a plurality of cascaded state-space fusion units, and the step of inputting the remote sensing classification training set into the first network framework for pre-training to obtain a pre-trained first network framework includes:

[0076] B1. Input the remote sensing classification training set into the plurality of cascaded state space fusion units to perform multi-level spatial feature extraction and fusion to obtain target classification fusion features. The target classification fusion features are used to record remote sensing scene feature information and target category feature information corresponding to the remote sensing classification training set.

[0077] In the embodiments of this application, during the pre-training stage of the first network framework, the remote sensing classification training set can be input into several cascaded state space fusion units for multi-level spatial feature extraction and fusion. Specifically, each state space fusion unit can perform spatial feature extraction and fusion at a depth level on the input features. The depth level corresponding to the current state space fusion unit is greater than the depth level corresponding to the previous state space fusion unit, thereby obtaining the target classification fusion feature. The target classification fusion feature can be the classification fusion feature output by the last state space fusion unit, or it can be the classification fusion feature output by one of the state space fusion units among all the state space fusion units.

[0078] It is understood that the embodiments of this application do not limit the number of cascaded state space fusion units. For example, it can be any of 8, 12, 16, 24, etc. The specific number of cascaded units can be set according to the actual situation. This embodiment of the application takes 12 cascaded state space fusion units as an example. In this case, the depth level corresponding to the first state space fusion unit is 1, the depth level corresponding to the third state space fusion unit is 3, and the depth level corresponding to the last state space fusion unit is 12. The depth levels corresponding to the remaining state space fusion units are similar and can be easily deduced. This application will not elaborate further here.

[0079] Furthermore, each state-space fusion unit includes a state-space modeling module and a spatial fusion module. Input features are fed into the state-space fusion unit for spatial feature extraction and fusion to obtain intermediate classification fusion features, including:

[0080] B11. Input the input features into the state space modeling module to perform state space modeling and obtain spatial depth modeling features;

[0081] Reference Figure 2 Further, step B11, inputting the input features into the state space modeling module for state space modeling to obtain spatial depth modeling features, includes:

[0082] B111. Perform bi-branch linear normalization on the input features to obtain the first intermediate feature and the second intermediate feature;

[0083] B112. Perform multi-directional scanning on the first intermediate feature to obtain a scanning feature set, wherein the scanning feature set includes several scanning features, and each scanning feature corresponds to a different scanning direction;

[0084] B113. Perform spatial modeling on the scan feature set to obtain several state space modeling data, each of the state space modeling data corresponding to one of the scan features;

[0085] B114. Add all the state-space modeling data element by element to obtain the state-space modeling features;

[0086] B115. Based on the second intermediate feature and the input feature, the state space modeling feature is multiplied element-wise to obtain the spatial depth modeling feature.

[0087] In the embodiments of this application, the features output by a certain state space fusion unit can be determined as intermediate classification fusion features. Specifically, for the first cascaded state space fusion unit, its input features can be the remote sensing classification training set input to the first network framework; while for the second and subsequent cascaded state space fusion units, its input features can be the classification fusion features output by the previous state space fusion unit.

[0088] It is understandable that the two-branch linear normalization in step B111 can be achieved by inputting the input features into a normalization layer for data standardization, obtaining standardized input features, and then using two branches to perform linear normalization on the standardized input features. Specifically, the first branch includes a linear layer, a deep convolutional layer, and a GELU activation function layer. The input features are mapped sequentially through the linear layer, deep convolution, and activation function in the first branch to obtain the first intermediate feature. Furthermore, the second branch includes a linear layer and a GELU activation function layer. The input features are mapped sequentially through the linear layer and activation function in the second branch to obtain the second intermediate feature.

[0089] It should be noted that the multi-directional scanning in step B112 can be based on a multi-directional scanning method (such as a four-directional scanning strategy), obtaining the scanning results of multiple scanning directions from the feature map of the first intermediate feature, and determining the scanning result of each scanning direction as a scanning feature; then, after obtaining the scanning features of all scanning directions, spatial modeling can be performed on each scanning feature separately. Specifically, the scanning features can be input into the SSM selective state-space operator for modeling, with each scanning feature corresponding to one SSM selective state-space operator, thereby achieving comprehensive spatial coverage and effective modeling of local and global information, and obtaining state-space modeling data corresponding to each scanning feature.

[0090] It is worth mentioning that, in this embodiment, after performing element-wise addition on all state space model data to obtain state space modeling features, the obtained state space modeling features are multiplied element-wise with the second intermediate features of the second branch, and then convolutional processing with a kernel of 1 is performed to obtain convolutional feature data. Then, the convolutional feature data is residually connected with the original input features to obtain spatial depth modeling features.

[0091] B12. Input the spatial depth modeling features into the spatial fusion module to perform adjacent scale feature fusion to obtain the intermediate classification fusion features.

[0092] Reference Figure 3 Furthermore, the intermediate classification fusion feature includes a first classification fusion feature and a second classification fusion feature. Step B12, inputting the spatial depth modeling feature into the spatial fusion module for adjacent scale feature fusion to obtain the intermediate classification fusion feature, includes:

[0093] B121. Perform first feature channel separation on the spatial depth modeling features to obtain third and fourth intermediate features;

[0094] B122. Based on the spatial depth modeling features, frequency fusion is performed on the third intermediate features to obtain the first classification fusion feature and the intermediate classification fusion feature;

[0095] B123. Based on the fourth intermediate feature, perform deep convolution and channel concatenation on the intermediate classification fusion feature to obtain the second classification fusion feature.

[0096] In this embodiment of the application, step B121 may specifically involve downsampling and convolving the spatial depth modeling features to obtain low-resolution features, and then performing feature channel separation on the low-resolution features to separate the low-resolution features into third intermediate features and fourth intermediate features in the channel dimension. The feature scales corresponding to the third intermediate features and fourth intermediate features are adjacent to the feature scales of the spatial depth modeling features input to the spatial fusion module.

[0097] Understandably, step B122 can be based on the FreqFusion frequency fusion operator, performing complementary fusion of the original resolution feature map corresponding to the spatial depth modeling feature and the low-resolution feature map corresponding to the third intermediate feature at both high-resolution and low-resolution angles to obtain a first classification fusion feature and an intermediate classification fusion feature with enhanced feature detail information. The first classification fusion feature is obtained by fusing the spatial depth modeling feature and the third intermediate feature at high resolution, while the intermediate classification fusion feature is obtained by fusing the spatial depth modeling feature and the third intermediate feature at low resolution. Furthermore, step B123 can first perform a depthwise convolution on the intermediate classification fusion feature, and then concatenate the depthwise convolutioned first classification fusion feature with the fourth intermediate feature obtained by separating the feature channels along the channel dimension to obtain the second classification fusion feature.

[0098] It should be noted that if the current state space fusion unit is not the last state space fusion unit, the second classification fusion feature obtained in step B123 can be the input feature of the state space modeling module of the next state space fusion unit; or, if the current state space fusion unit is the last state space fusion unit, the second classification fusion feature obtained in step B123 can be determined as the final target classification fusion feature of the first network framework.

[0099] B2. Based on the target classification fusion features, update the parameters of the first network framework to obtain the pre-trained first network framework.

[0100] In this embodiment of the application, after obtaining the final target classification fusion feature of the first network framework, the target classification fusion feature can be input into the corresponding classification head for recognition, thereby obtaining the remote sensing image recognition result output by the first network framework. Then, the parameters of the first network framework are updated based on the remote sensing image recognition result. The specific parameter update content is similar to the content of the aforementioned step 120, and can be simply deduced by analogy. This application will not elaborate further here.

[0101] Step 130: Input the remote sensing detection training set into the pre-trained first network framework to extract remote sensing features and obtain a fusion feature set. The fusion feature set includes several spatial fusion features with different spatial depths. Each spatial fusion feature is used to record the feature information of remote sensing features at two adjacent feature scales.

[0102] In this embodiment of the application, the remote sensing detection training set can be input into a pre-trained first network framework. The remote sensing features contained in the remote sensing images in the remote sensing monitoring training set can be extracted through the pre-trained first network framework to obtain several spatial fusion features with different spatial depths.

[0103] It is understood that the spatial fusion feature record in this embodiment of the application records feature information of remote sensing features at two adjacent feature scales corresponding to the spatial depth, which may specifically be the first classification fusion feature obtained in step B122 above. In addition, the fusion feature set in this embodiment of the application may be composed of all spatial fusion features, or it may be composed of several spatial fusion features at a specific spatial depth.

[0104] For example, if the spatial depth corresponding to the last state space fusion unit is 12, the embodiments of this application can select spatial fusion features with spatial depths of 2, 4, 10, and 12 to construct a fusion feature set. Specifically, the fusion features can be determined by the first classification fusion features output by the second state space fusion unit, the fourth state space fusion unit, the tenth state space fusion unit, and the last state space fusion unit. This example is only for illustration and is not intended to limit the application. The spatial depth corresponding to the spatial fusion features in the fusion feature set can also be set according to the actual situation.

[0105] Step 140: Input the fused feature set into the second network framework for remote sensing target detection to obtain target detection information;

[0106] In this embodiment, step 140 may involve inputting the fused feature set into a second network framework, and then using the second network framework to fuse and classify all spatial fused features to obtain target detection information for the remote sensing training set. Specifically, the second network framework in this embodiment may include a bottleneck model (such as an FPN model) and a detection head. Spatial fused features of different spatial depths in the fused feature set can be input into the bottleneck model for fusion to obtain the final target fused features, which are then input into the detection head to obtain target detection information.

[0107] Step 150: Based on the target detection information, perform joint fine-tuning training on the pre-trained first network framework and the second network framework to obtain the trained remote sensing target detection model.

[0108] In this embodiment, the joint fine-tuning training in step 150 can determine a loss value based on the classification information in the target detection information and the corresponding classification label in the remote sensing detection dataset. This loss value is recorded as the first sub-loss value, which is the classification loss for target detection. The loss function used to determine the first sub-loss value can be based on the binary cross-entropy loss function. Specifically, the equivalent expression for the first sub-loss value can be:

[0109]

[0110] Where L is the first sub-loss value, and N is the total number of samples in the remote sensing detection training set; L i y is the classification loss for the i-th sample in the remote sensing training set; i p represents the sample category of the i-th sample. i It represents the predicted probability in the target detection information corresponding to the i-th sample.

[0111] Then, another loss value can be determined based on the predicted bounding box in the target detection information and the corresponding ground truth bounding box label in the remote sensing detection dataset. This second loss value is denoted as the second sub-loss value, which can be the target detection bounding box regression loss. Specifically, in the embodiments of this application, the second sub-loss value can be determined based on a combination of the DFL (Distribution Focal) loss function and the CIoU (Complete-IoU) loss function. The equivalent expression for the DFL loss part in the second sub-loss value can be:

[0112]

[0113] Among them, L DFL For DFL loss; S i Let be the predicted score corresponding to the i-th sample.

[0114] The equivalent expression for the CIoU loss component in the second sub-loss value can be:

[0115]

[0116] Among them, L CIoU CIoU loss; IoU is the ratio of the intersection and union of the areas of the ground truth bounding box and the predicted bounding box; box is the predicted bounding box; gbox is the ground truth bounding box; d 2 (box, gbox) is the Euclidean distance between the centers of the ground truth box and the predicted box; c is the diagonal length of the bounding box containing both the ground truth box and the predicted box; α is the correction factor; v is the aspect ratio difference between the predicted box and the ground truth box.

[0117] It is understandable that after determining the first sub-loss value and the second sub-loss value respectively, the target loss value can be easily determined based on the first sub-loss value and the second sub-loss value. Based on the target loss value, the pre-trained first network framework and the second network framework can be jointly fine-tuned and trained through the backpropagation algorithm to obtain the trained remote sensing target detection model.

[0118] This application also proposes a method for identifying remote sensing target detection models, including:

[0119] Step 160: Obtain the remote sensing image of the target to be detected;

[0120] Step 170: Input the target remote sensing image into the remote sensing target detection model trained as described above for target detection, and obtain the remote sensing target detection result.

[0121] In this embodiment of the application, the remote sensing image of the target to be detected can be input into a trained remote sensing target detection model. The trained remote sensing target detection model can then be used to detect the remote sensing scene and remote sensing target in the target remote sensing image, thereby obtaining the remote sensing target detection result output by the remote sensing target detection model.

[0122] The following describes in detail, with reference to the accompanying drawings, a training system for a remote sensing target detection model according to an embodiment of this application.

[0123] Reference Figure 4 The training system for a remote sensing target detection model proposed in this application includes:

[0124] The first processing unit 101 is used to acquire remote sensing classification training set and remote sensing detection training set;

[0125] The second processing unit 102 is used to input the remote sensing classification training set into the first network framework for pre-training to obtain the pre-trained first network framework.

[0126] The third processing unit 103 is used to input the remote sensing detection training set into the pre-trained first network framework to extract remote sensing features and obtain a fusion feature set. The fusion feature set includes several spatial fusion features with different spatial depths. Each spatial fusion feature is used to record the feature information of remote sensing features of two adjacent feature scales.

[0127] The fourth processing unit 104 is used to input the fused feature set into the second network framework for remote sensing target detection to obtain target detection information;

[0128] The fifth processing unit 105 is used to perform joint fine-tuning training on the pre-trained first network framework and the second network framework based on the target detection information to obtain the trained remote sensing target detection model.

[0129] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0130] Reference Figure 5 This application also provides an electronic device, including:

[0131] At least one processor 201;

[0132] At least one memory 202 is used to store at least one program;

[0133] When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the method embodiment described above.

[0134] Similarly, it can be understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0135] This application also provides a computer-readable storage medium storing a program executable by a processor 201, which, when executed by the processor 201, is used to implement the above-described method embodiments.

[0136] Similarly, the content of the above method embodiments is applicable to the present computer-readable storage medium embodiments. The specific functions implemented by the present computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0137] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0138] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0139] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0141] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0142] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0143] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0144] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0145] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A training method for a remote sensing target detection model, characterized in that, include: Obtain training sets for remote sensing classification and remote sensing detection; The remote sensing classification training set is input into the first network framework for pre-training to obtain the pre-trained first network framework. The remote sensing detection training set is input into the pre-trained first network framework to extract remote sensing features and obtain a fusion feature set. The fusion feature set includes several spatial fusion features with different spatial depths. Each spatial fusion feature is used to record the feature information of remote sensing features of two adjacent feature scales. The fused feature set is input into the second network framework for remote sensing target detection to obtain target detection information; Based on the target detection information, the pre-trained first network framework and the second network framework are jointly fine-tuned to obtain the trained remote sensing target detection model. The first network framework includes several cascaded state-space fusion units. Each state-space fusion unit includes a state-space modeling module and a spatial fusion module. Input features are input into the state-space fusion unit for spatial feature extraction and fusion to obtain intermediate classification fusion features, including: The input features are input into the state space modeling module to perform state space modeling, thereby obtaining spatial depth modeling features; The spatial depth modeling features are input into the spatial fusion module to perform adjacent scale feature fusion, thereby obtaining the intermediate classification fusion features; The step of inputting the input features into the state space modeling module for state space modeling to obtain spatial depth modeling features includes: The input features are subjected to bi-branch linear normalization to obtain the first intermediate feature and the second intermediate feature; The first intermediate feature is scanned in multiple directions to obtain a scanning feature set, which includes several scanning features, each of which corresponds to a different scanning direction; Spatial modeling is performed on the scan feature set to obtain several state space modeling data, each of the state space modeling data corresponding to one scan feature; The state-space modeling features are obtained by adding all the state-space modeling data element by element. Based on the second intermediate feature and the input feature, the state space modeling feature is multiplied element-wise to obtain the spatial depth modeling feature.

2. The method according to claim 1, characterized in that, The acquisition of the remote sensing classification training set and the remote sensing detection training set includes: Obtain remote sensing classification datasets and remote sensing detection datasets; Data augmentation is performed on the remote sensing classification dataset to obtain the remote sensing classification training set. The remote sensing detection dataset is augmented to obtain the remote sensing detection training set.

3. The method according to claim 1, characterized in that, The step of inputting the remote sensing classification training set into the first network framework for pre-training to obtain the pre-trained first network framework includes: The remote sensing classification training set is input into the plurality of cascaded state space fusion units for multi-level spatial feature extraction and fusion to obtain target classification fusion features. The target classification fusion features are used to record remote sensing scene feature information and target category feature information corresponding to the remote sensing classification training set. Based on the target classification fusion features, the parameters of the first network framework are updated to obtain the pre-trained first network framework.

4. The method according to claim 1, characterized in that, The intermediate classification fusion feature includes a first classification fusion feature and a second classification fusion feature. The step of inputting the spatial depth modeling feature into the spatial fusion module for adjacent scale feature fusion to obtain the intermediate classification fusion feature includes: The spatial depth modeling features are subjected to first feature channel separation to obtain third and fourth intermediate features; Based on the spatial depth modeling features, frequency fusion is performed on the third intermediate features to obtain the first classification fusion feature and the intermediate classification fusion feature; Based on the fourth intermediate feature, the intermediate classification fusion feature is subjected to deep convolution and channel concatenation to obtain the second classification fusion feature.

5. A method for identifying remote sensing target detection models, characterized in that, include: Acquire remote sensing images of the target to be detected; The target remote sensing image is input into the trained remote sensing target detection model as described in any one of claims 1-4 to perform target detection and obtain the remote sensing target detection result.

6. A training system for a remote sensing target detection model, characterized in that, include: The first processing unit is used to acquire remote sensing classification training sets and remote sensing detection training sets; The second processing unit is used to input the remote sensing classification training set into the first network framework for pre-training to obtain the pre-trained first network framework. The third processing unit is used to input the remote sensing detection training set into the pre-trained first network framework to extract remote sensing features and obtain a fused feature set. The fused feature set includes several spatial fused features of different spatial depths. Each spatial fused feature is used to record the feature information of remote sensing features of two adjacent feature scales. The fourth processing unit is used to input the fused feature set into the second network framework for remote sensing target detection to obtain target detection information; The fifth processing unit is used to perform joint fine-tuning training on the pre-trained first network framework and the second network framework based on the target detection information to obtain the trained remote sensing target detection model. The first network framework includes several cascaded state-space fusion units. Each state-space fusion unit includes a state-space modeling module and a spatial fusion module. Input features are input into the state-space fusion unit for spatial feature extraction and fusion to obtain intermediate classification fusion features, including: The input features are input into the state space modeling module to perform state space modeling, thereby obtaining spatial depth modeling features; The spatial depth modeling features are input into the spatial fusion module to perform adjacent scale feature fusion, thereby obtaining the intermediate classification fusion features; The step of inputting the input features into the state space modeling module for state space modeling to obtain spatial depth modeling features includes: The input features are subjected to bi-branch linear normalization to obtain the first intermediate feature and the second intermediate feature; The first intermediate feature is scanned in multiple directions to obtain a scanning feature set, which includes several scanning features, each of which corresponds to a different scanning direction; Spatial modeling is performed on the scan feature set to obtain several state space modeling data, each of the state space modeling data corresponding to one scan feature; The state-space modeling features are obtained by adding all the state-space modeling data element by element. Based on the second intermediate feature and the input feature, the state space modeling feature is multiplied element-wise to obtain the spatial depth modeling feature.

7. An electronic device, characterized in that, include: at least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1-5.

Images