In-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion

Abstract

The invention relates to an in-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion. The method comprises the following steps: acquiring multi-scale fusion features of a remote sensing image; carrying out space channel feature interaction processing; and carrying out model training based on an EWD joint loss function. According to the method, improvement is carried out on the basis of YOLOV5, a remote sensing image target detection model MF-YOLO based on multi-scale feature fusion is provided, the multi-level receptive field of the model is expanded, and the capacity of a backbone network for extracting multi-dimensional features and obtaining detailed information of a target area is improved; a space channel feature interaction module SCFAM is designed in a backbone network connection layer, and the positioning capability of the model to a salient region is improved under the condition that network parameters are not increased; in addition, an EWD loss function is further designed, the sensitivity of the model to slight changes of target positions is reduced, the model is guided to distinguish dense overlapping targets, and the detection performance of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of optical remote sensing technology application technology, and in particular to an on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion. Background Art

[0002] With the advancement of remote sensing technology and the increase in the number of satellites, the resolution of remote sensing images is constantly improving, and the data scale is becoming larger and larger. It has very important application value in geological exploration, agricultural mapping, disaster emergency response, urban construction and military target detection. Remote sensing image on-orbit target detection mainly processes the massive remote sensing data taken by satellites on-orbit, and transmits information such as the category and location of the target. At present, remote sensing sensors mainly include synthetic aperture radar, thermal infrared sensor and optical sensor. Among them, optical remote sensing images have an imaging mechanism similar to that of the human eye, which can intuitively present rich feature information such as color, texture, shape, etc. of the detected target, reducing the difficulty of image interpretation. It is an important research direction for remote sensing image target detection.

[0003] Remote sensing image target detection mainly solves two problems: positioning and identification. Common detection targets include aircraft, oil tanks, ships, etc. At present, there are two main solutions: traditional feature extraction and deep learning detection. The traditional method mainly relies on manual analysis of target images and construction of mathematical models, and uses expert knowledge to extract important information from images. By inputting a large number of low- and medium-level features such as shape, color, texture, etc. into the classifier or performing template matching, the target is detected. However, this method consumes a lot of manpower costs, and due to the particularity of remote sensing images, the representativeness of expert-extracted features is poor, which cannot meet the detection needs of large-scale multi-scenes. The target detection method based on deep learning mainly uses convolutional neural networks to extract image features, which is divided into two-stage and one-stage methods. The two-stage divides feature extraction and recognition into two stages. For example, the R-CNN series model mainly uses the region candidate network to determine the candidate region, extracts the target feature information and hands it over to the classifier for processing. It has a high accuracy rate, but the model is highly complex and has a large number of parameters, which is not suitable for rapid deployment of the edge end of the satellite in orbit. The one-stage method performs feature extraction and recognition at the same time, such as the YOLO series network, which directly divides the image into multiple grids, and detects the bounding box range and the probability of the existence of the target at the same time, thereby improving the speed of target detection. Zhu Ruifei (publication number: CN118196652A, publication date: 2024.06.14) et al. proposed a satellite image ship rapid detection method based on a large-core selective attention mechanism. On the basis of the traditional YOLOV8 model, a large-core selective attention mechanism module is added, and a large-scale convolution kernel is used to expand the network's receptive field and enhance the model's perception of the target area. This method uses a serial structure to obtain deep features of different receptive fields layer by layer, and adaptively selects different features for subsequent splicing and other processing. The process can retain all the information of the previous layer features while expanding the receptive field, but it will also increase the amount of calculation of the actual size of the corresponding convolution, and the fusion of features of different scales is not sufficient. The infrared dim small target detection method based on cross-connection and fusion attention mechanism proposed by Li Zhengzhou et al. (publication number: CN118172634A, publication date: 2024.06.11) extracts refined features of spatial and channel dimensions in the backbone network in turn by introducing the CBAM module to detect infrared dim small targets. The remote sensing image classification method and device based on spatial and channel feature extraction proposed by Wang Wei et al. (publication number: CN116188836A, publication date: 2023.05.30) mainly introduces channel attention module, local window attention module, linear attention module, pyramid pooling attention module to construct a multi-head self-attention structure, and uses 4 stages to complete the fusion of spatial and channel features for image classification. However, this type of processing architecture is relatively complex, and the model parameter scale and computational complexity are large, which is not conducive to the lightweight deployment of the model on orbit.Wang Xiaojuan (publication number: CN118212505A, publication date: 2024.06.18) et al. proposed a deep space small target detection method and system based on multi-scale feature pyramid, which solves the detection of deep space small targets by introducing squeeze and excitation (SE) attention mechanism and feature pyramid, but the model still uses CIOU as the loss function, which makes it difficult to distinguish the differences caused by changes in the length and width of the prediction box; Xu Xiaoping (publication number: CN118196757A, publication date: 2024.06.14) et al. proposed a persimmon fruit detection method based on an improved YOLOv5 model. The model detection ability is improved by introducing the squeeze and excitation (SE) attention mechanism and the EIOU loss function, but EIOU cannot solve the problem of unbalanced data samples; Yan Dongmei (publication number: CN118212545A, publication date: 2024.06.18) and others proposed a ship detection and recognition method, device and equipment based on the improved YOLOv5 model, which obtains small target feature information by setting different detection layers to detect ships on the dim sea surface at night. The SIOU cited in the method still relies on a single IOU loss to guide the model, and has poor discrimination for densely overlapping target areas.

[0004] The current target detection algorithms are mainly aimed at traditional images, with poor generalization in the field of remote sensing images and poor interpretation of multi-target information in dense areas. In addition, unlike ground-based remote sensing image target detection and processing, the scale and processing time of the on-orbit model will also be affected by multiple limitations such as power consumption and computing power of the on-orbit platform. Summary of the invention

[0005] The present invention aims to solve the technical problems in the prior art and provide an on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] An on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion includes the following steps:

[0008] Step 1: Obtain multi-scale fusion features of remote sensing images;

[0009] A set of hole convolutions of different scales are introduced to capture features under different receptive fields. At the same time, residual connections are used to avoid gradient explosion or gradient disappearance. Then, the features of each branch are averaged and merged to obtain multi-scale information of the image.

[0010] Group and batch split the features, build a parallel processing structure, and improve the efficiency of feature extraction;

[0011] Step 2: Interactive processing of spatial and channel features;

[0012] The high-dimensional features of the convolutional neural network are bilinearly upsampled and then fused with the low-dimensional features by feature concatenation to obtain the input features. Where C is the number of channels, H×W is the length and width of the feature map;

[0013] By calculating the mean and variance distribution of each channel in the channel direction, a three-dimensional attention weight is obtained Perform Hadamard product of input features and attention weights to obtain output weighted features

[0014] Then, the weighted features are convolved, mean normalized, and activated to fully integrate the acquired spatial and channel dimension information to obtain the final spatial channel joint feature. Used for subsequent feature processing;

[0015] Step 3: Model training based on the EWD joint loss function;

[0016] Model the two-dimensional Gaussian distribution of the coordinate frame B = (cx, cy, w, h)

[0017]

[0018] Where: (cx, cy)w, h are the coordinates of the center point of the anchor box, the box width and the box height respectively;

[0019] Measuring the prediction box under two-dimensional Gaussian distribution With real box The distribution distance between

[0020] Perform exponential normalization to obtain the normalized Wasserstein distance:

[0021]

[0022] Where: ||·|| F is the Frobenius norm, C is the data set size distribution constant;

[0023] The two evaluation criteria are combined in a weighted average manner to obtain the loss function EWD:

[0024]

[0025] Where: λ is the adjustment factor of the joint loss function; ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the two boxes, c represents the diagonal distance of the minimum enclosed area of ​​the two boxes, C wand C h They are the width and height of the smallest enclosed box containing the predicted box and the real box respectively; IOU represents the intersection-over-union ratio between the predicted box and the real box;

[0026] ρ 2 (w,w gt ), ρ 2 (h,h gt ) represent the Euclidean distance between the width and height of the two boxes respectively;

[0027] In the above technical solution, in step 1, a multi-scale feature fusion module is used to group and batch split the features to build a parallel processing structure;

[0028] The multi-scale feature fusion module includes: branch one, branch two and branch three;

[0029] Branch 1 adopts SA structure, obtains feature information along the channel dimension and spatial dimension and performs splicing, integrates the spatial position information of each group of features into the channel feature, and realizes feature fusion;

[0030] Branch 2 uses traditional convolutional pooling to further mine deep feature information;

[0031] Branch 1 and branch 2 interact locally across channels, aggregate multi-scale features using matrix multiplication, and finally combine with branch 3, fully combining the contextual information between channels and within space, realizing multi-scale feature fusion, and improving the feature mining and expression capabilities of the model.

[0032] In the above technical solution, in step 2, the weighted feature Y is calculated as follows:

[0033] Y=X⊙Ε

[0034] Each neuron Y in Y c,i,j It is expressed as:

[0035]

[0036]

[0037] Where: μ c is the mean value in channel c, is the variance within channel c, ε is a very small constant to prevent divisor, and its value is 0.

[0038] In the above technical solution, the EWD joint loss function in step 3 is as follows:

[0039]

[0040] Where: IOU represents the intersection-over-union ratio between the predicted box and the true box, α and γ are modulation factors, α is used to adjust the ratio between the positive and negative sample losses, γ is used to control the balance between simple and complex samples, and ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted box and the real box, c represents the diagonal distance of the minimum closure area of ​​the two boxes, C w and C h are the width and height of the smallest enclosing box containing the predicted box and the real box, respectively; ρ 2 (w,w gt ), ρ 2 (h,h gt ) represent the Euclidean distance between the width and height of the two boxes respectively.

[0041] The present invention has the following beneficial effects:

[0042] The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion of the present invention is improved on the basis of YOLOV5, and a remote sensing image target detection model MF-YOLO based on multi-scale feature fusion is proposed, which can extract more comprehensive and detailed feature information for on-orbit detection of remote sensing targets. The model mainly designs a multi-scale feature fusion module MSFFM based on dilated convolution, which expands the multi-level receptive field of the model and improves the ability of the backbone network to extract multi-dimensional features and obtain detailed information of the target area; a spatial channel feature interaction module SCFAM is designed in the connection layer of the backbone network, which improves the model's ability to locate significant areas without increasing network parameters; in addition, an EWD loss function is designed to reduce the model's sensitivity to slight changes in target position, guide the model to distinguish densely overlapping targets, and improve the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0044] Figure 1 It is a schematic diagram of the overall network structure of the MF-YOLO model in the on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion of the present invention.

[0045] Figure 2 Schematic diagram of the multi-scale feature fusion module structure.

[0046] Figure 3 Schematic diagram of the processing flow of the spatial and channel feature interaction module.

[0047] Figure 4 Schematic diagram of the PR curve of the algorithm model.

[0048] Figure 5Schematic diagram of dense target detection and heat map; (a) is the basic model and (b) is the improved model.

[0049] Figure 6 Schematic diagram of target detection results.

[0050] Figure 7 Schematic diagram of ocean target detection results.

[0051] Figure 8 Schematic diagram of offshore target detection results.

[0052] Fig. 9 This is a schematic diagram of the port target detection results. DETAILED DESCRIPTION

[0053] The inventive concept of the present invention is:

[0054] The invention proposes an on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion, and proposes a method MF-YOLO for on-orbit remote sensing target detection. Figure 1 Shown is the overall framework structure of the model.

[0055] After the image data is input into the model, it is first preprocessed and converted into training data with anchor frames. It then enters the backbone network to obtain deep feature information layer by layer. MSFFM is used at the deepest part of the network to expand the model's receptive field and enrich the global feature information. The feature information extracted by the model at different levels is then fused in the connection layer in the form of a feature pyramid. In the process, SCFAM is used to perform a spatial and channel feature interactive processing on the intermediate features based on the spatial domain inhibition effect of neurons, thereby enhancing the model's extraction of detail information in salient areas. After the features are output to the detection head, they mainly include three scales, and the target position, category, and confidence level are predicted according to the set classifier to obtain the final detection result.

[0056] The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion of the present invention comprises the following steps:

[0057] Step 1: Obtain multi-scale fusion features of remote sensing images

[0058] The scales of detected targets in remote sensing images are widely distributed, and the pixel areas covered vary greatly. Therefore, it is necessary to expand the range of images corresponding to different anchor boxes on the feature pyramid so that the model has stronger multi-scale detection capabilities. The size of the receptive field represents the ability of the features extracted by the model to capture the original image information. When the receptive field of the model increases, the single-dimensional deep features of the traditional feature pyramid cannot well reflect the semantic information of the target. Therefore, the present invention designs a multi-scale feature fusion module in the backbone network. The main structure of the module is as follows: Figure 2 As shown:

[0059] The upper part of the multi-scale feature fusion module mainly introduces a set of hole convolutions of different scales to capture features under different receptive fields, and uses residual connections to avoid gradient explosion or gradient disappearance. Then, the features of each branch are averaged and merged to obtain the multi-scale information of the image. After the feature merging, the spatial information is still independently distributed in the channel, and the channel features and spatial features need to be multi-scale fused. In order to quickly perform feature fusion, the lower part of the multi-scale feature fusion module first groups and batches the features, builds a parallel processing structure, and improves the efficiency of feature extraction. Among them, branch one adopts the SA structure, obtains feature information along the channel dimension and spatial dimension and splices it, integrates the spatial position information of each group of features into the channel feature, and realizes feature fusion; branch two adopts the traditional convolution pooling method to further mine deep feature information; at the same time, the two branches also perform local cross-channel interaction, and use matrix multiplication to aggregate multi-scale features, and finally combine with branch three to fully combine the contextual information between channels and in space, realize multi-scale feature fusion, and improve the feature mining and expression capabilities of the model.

[0060] Step 2: Interactive processing of spatial and channel features

[0061] After extracting multi-scale features of remote sensing images, it is necessary to process the images, especially the salient target areas, to improve the model's ability to extract and identify key features. The information obtained by traditional spatial attention mechanisms such as SAM and channel attention mechanisms such as CAM is relatively single, and some information may be lost during the processing process. Joint attention mechanisms such as CBAM process features twice, which will increase the complexity of the model and increase the weight parameters of the network, which is not conducive to the on-orbit deployment of the model. The present invention introduces the idea of ​​the SimAM attention mechanism and designs a spatial and channel feature interaction module in the middle layer of the network. While occupying fewer weight parameters, it highlights the important regional features. The module processing flow is as follows: Figure 3 As shown:

[0062] The spatial and channel feature interaction module first performs bilinear upsampling on the high-dimensional features of the convolutional neural network, and then fuses them with the low-dimensional features by feature splicing to obtain the input features. Where C is the number of channels, H×W is the length and width of the feature map; activated neurons usually inhibit surrounding neurons and should be given higher parameter weights. By calculating the mean and variance distribution of each channel in the channel direction, a three-dimensional attention weight is obtained The output weighted feature can be obtained by performing Hadamard product of the input feature and the attention weight Then, the weighted features are convolved, mean normalized, and activated to fully integrate the acquired spatial and channel dimension information to obtain the final spatial channel joint feature. Used for subsequent feature processing.

[0063] The calculation method of weighted feature Y in the process is as follows:

[0064] Y=X⊙Ε

[0065] Each neuron Y in Y c,i,j It can be expressed as:

[0066]

[0067] Where: μ c is the mean value in channel c, is the variance within channel c, ε is a very small constant to prevent divisor, and its value is 0.

[0068] Step 3: Model training based on EWD joint loss function

[0069] After building the overall framework of the model, we start training the model. During the training process, we need to design a reasonable loss function. The traditional loss function mainly includes three parts, namely classification loss, positioning loss and confidence loss. The positioning loss and confidence loss are both related to the bounding box loss. The CIOU loss used by YOLOV5 mainly reflects the aspect ratio change of the bounding box, and cannot express the width and height difference between the predicted box and the real box.

[0070] Therefore, the present invention proposes an EWD joint loss function to replace the CIOU loss function, and simultaneously calculates the IOU between boxes and the distance under the two-dimensional Gaussian distribution, guiding the model to distinguish small-sized targets in dense areas and improve detection performance. Unlike CIOU, which uniformly calculates the aspect ratio of the annotation box, the EIOU loss function calculates the length and width losses of the predicted box and the real box separately, while minimizing the difference between the bounding boxes, which can improve the convergence speed and positioning accuracy of the model. In the actual training process, there are still problems such as unbalanced distribution of positive and negative samples and large regression errors of difficult and easy samples. Combined with the idea of ​​Focal Loss, the intersection of the two boxes is used as the base, and a modulation factor is added to avoid the problem of uneven sample distribution. The improved EIOU loss function is as follows:

[0071]

[0072] Where: IOU represents the intersection-over-union ratio between the predicted box and the true box, α and γ are modulation factors, α is used to adjust the ratio between the positive and negative sample losses, γ is used to control the balance between simple and complex samples, and ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the two boxes, c represents the diagonal distance of the minimum enclosed area of ​​the two boxes, C w and C h are the width and height of the smallest enclosing box containing the predicted box and the real box, respectively; ρ2 (w,w gt ), ρ 2 (h,h gt ) represent the Euclidean distance between the width and height of the two boxes respectively.

[0073] There are a large number of small-sized targets in remote sensing images. If the intersection-over-union ratio is used alone to calculate the bounding box loss, the model will be more sensitive to the deviation of smaller targets, and a slight offset will produce a large error. When detecting overlapping targets in dense areas, when the offset scale is within a certain range, the intersection-over-union ratio of two frames of different scales will be constant. At this time, the offset of the annotation box is difficult to guide the model to converge, which will cause certain missed detections. However, under the two-dimensional Gaussian distribution, the weight of the center pixel of the detection box is increased, and the weight of the boundary background pixel is weakened. The similarity between bounding boxes of different sizes can be measured by the distribution distance, thereby reducing the interference of bounding boxes of different sizes on model regression and improving the model's detection ability for smaller targets. The calculation process is as follows:

[0074] First, the coordinate frame B = (cx, cy, w, h) needs to be modeled into a two-dimensional Gaussian distribution

[0075]

[0076] Where: (cx, cy), w, h are the coordinates of the center point of the anchor box, the box width and the box height respectively;

[0077] After that, measure the prediction box under the two-dimensional Gaussian distribution With real box The distribution distance between

[0078] Then, perform exponential normalization to obtain the normalized Wasserstein distance:

[0079]

[0080] Where: ||·|| F is the Frobenius norm, C is the data set size distribution constant; (cx a ,cy a ), w a ,h a are the coordinates of the center point of the prediction box, the box width and the box height respectively; (cx b ,cy b ), w b ,h b They are respectively the coordinates of the center point of the real frame, the frame width and the frame height.

[0081] The two evaluation criteria are combined in a weighted average manner to obtain the loss function EWD:

[0082]

[0083] Where: λ is the adjustment factor of the joint loss function.

[0084] Finally, the model is trained based on the loss function EWD, and the trained model is used for remote sensing target detection. Training the model based on the EWD loss function can improve the detection ability of the model and reduce the missed detection rate of dense targets.

[0085] The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion of the present invention is improved on the basis of YOLOV5, and a remote sensing image target detection model MF-YOLO based on multi-scale feature fusion is proposed, which can extract more comprehensive and detailed feature information for on-orbit detection of remote sensing targets. The model mainly designs a multi-scale feature fusion module MSFFM based on dilated convolution, which expands the multi-level receptive field of the model and improves the ability of the backbone network to extract multi-dimensional features and obtain detailed information of the target area; a spatial channel feature interaction module SCFAM is designed in the connection layer of the backbone network, which improves the model's ability to locate significant areas without increasing network parameters; in addition, an EWD loss function is designed to reduce the model's sensitivity to slight changes in target position, guide the model to distinguish densely overlapping targets, and improve the detection performance of the model.

[0086] The present invention is described in detail below with reference to the accompanying drawings.

[0087] The experimental data set used in the present invention is mainly composed of Jilin-1 series satellite annotation data, DOTA data set, HRRSD data set, NWPU VHR-10 data set and DIOR data set. Taking ship detection data as an example, a total of 17,691 images were collected and organized, with a size range of 400×400 to 800×800, and a resolution of mainly 2m. Data cleaning was carried out to address the phenomenon of labeling errors, and erroneous data such as mislabeling and anchor frame offset were removed; at the same time, in order to address the problem of uneven sample scale distribution, traditional data enhancement methods such as cropping, rotation, splicing and noise interference were used to expand small sample data, reducing the interference of data distribution on model training. At present, the overall scale of the experimental data set is 25,326, and it is randomly divided into training set, verification set and test set according to 6:2:2. After that, the data is loaded into the model for training. The CPU of the experimental workstation is 48 cores. Xeon(R)Silver

[0088] 4214CPU@2.20GHz, 128G memory, 2 NVIDIAGeForce RTX3090 GPUs, 48G video memory. The operating system is Ubuntu18.04, the computing platform is CUDA11.4, cuDNN8.2.4, and the deep learning framework is Pytorch1.8.1. Through loss function guidance and multiple rounds of training optimization iterations, the optimal weight of the model is obtained. According to the optimal weight, the specific performance indicators of the model can be obtained by evaluating on the test set. Finally, the trained model can be deployed to the edge after lightweight processing such as quantization and pruning for on-orbit reasoning business. Considering the computing power level of on-orbit hardware and the timeliness requirements of the business, it is only compared with the current One-stage algorithm with lower parameter quantity and complexity. The evaluation results are as follows.

[0089] Table 1 YOLO series algorithm performance

[0090]

[0091] It can be seen that YOLOV9m has a lower model complexity and the least number of parameters, but its detection performance is poor; YOLOV5l and YOLOV10x have high detection accuracy, but the computational complexity of YOLOV10x is relatively high, which affects the on-orbit processing speed; MF-YOLO proposed in the present invention has increased the detection mean average accuracy by 1.1 percentage points to 98.2% with only 3.4M more parameters and less changes in model computational complexity compared to YOLOV5. Comprehensive comparison shows that the new model proposed in the present invention performs better than several other YOLO algorithms on remote sensing data sets.

[0092] In order to verify the improvement of model performance by the multi-scale feature fusion module, the spatial and channel feature interactive processing module and the EWD joint loss function, an ablation experiment was conducted on the model, and the experimental results are shown in Table 2. AT1 to AT3 in the table are the spatial and channel feature joint processing modules inserted into different positions of the network.

[0093] Table 2 Algorithm module control experiment

[0094]

[0095]

[0096] At the same time, the PR curves of each group of model training are drawn, such as Figure 4 As shown in the figure, the PR curve and the mean average precision of the corresponding model show that the improvements in each part proposed in the present invention have achieved results and improved the accuracy of model detection.

[0097] Select typical dense target images for detection and draw the corresponding heat map, such as Figure 5As shown in the figure, by comparing the detection results and the heat map, it can be seen that the model's weight allocation to the target salient area has been further strengthened, the model is more sensitive to the deviation of the dense target detection box, and the detection accuracy has also been improved.

[0098] In summary, the MF-YOLO proposed in this invention has improved the model's multi-scale feature extraction capability and the detection performance for dense targets to a certain extent, while ensuring that the model complexity remains basically unchanged. The detection timeliness of targets in different scenes can meet the on-orbit requirements. The detection results of the model on the data set are as follows: Figure 6 shown.

[0099] The model is deployed to the edge on-orbit AI platform for inference. The actual detection results are as follows Figure 7-9 As shown, it can be seen that the model has good detection capabilities for targets in different sea areas, ports and of different sizes.

[0100] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. An on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion, It is characterized in that The following steps are involved: Step 1: Obtain multi-scale fusion features of remote sensing images; A set of hole convolutions of different scales are introduced to capture features under different receptive fields. At the same time, residual connections are used to avoid gradient explosion or gradient disappearance. Then, the features of each branch are averaged and merged to obtain multi-scale information of the image. Group and batch split the features, build a parallel processing structure, and improve the efficiency of feature extraction; Step 2: Interactive processing of spatial and channel features; The high-dimensional features of the convolutional neural network are bilinearly upsampled and then fused with the low-dimensional features by feature concatenation to obtain the input features. Where C is the number of channels, H×W is the length and width of the feature map; By calculating the mean and variance distribution of each channel in the channel direction, a three-dimensional attention weight is obtained Perform Hadamard product of input features and attention weights to obtain output weighted features Then, the weighted features are convolved, mean normalized, and activated to fully integrate the acquired spatial and channel dimension information to obtain the final spatial channel joint feature. Used for subsequent feature processing; Step 3: Model training based on the EWD joint loss function; Model the two-dimensional Gaussian distribution of the coordinate frame B = (cx, cy, w, h) Where: (cx, cy), w, h are the coordinates of the center point of the anchor box, the box width and the box height respectively; Measuring the prediction box under two-dimensional Gaussian distribution With real box The distribution distance between Perform exponential normalization to obtain the normalized Wasserstein distance: Where: ||·|| F is the Frobenius norm, C is the data set size distribution constant; (cx a ,cy a ), w a ,h a are the coordinates of the center point of the prediction box, the box width and the box height respectively; (cx b ,cy b ), w b ,h b They are the coordinates of the center point of the real frame, the frame width and the frame height respectively; The two evaluation criteria are combined in a weighted average manner to obtain the loss function EWD: Where: λ is the adjustment factor of the joint loss function; ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the two boxes, c represents the diagonal distance of the minimum enclosed area of ​​the two boxes, C w and C h They are the width and height of the smallest enclosing box containing the predicted box and the real box respectively; IOU represents the intersection-over-union ratio between the predicted box and the true box; ρ 2 (w,w gt ), ρ 2 (h,h gt ) represent the Euclidean distance between the width and height of the two boxes respectively; Finally, the model is trained based on the loss function EWD, and the trained model is used for remote sensing target detection.

2. The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion according to claim 1, It is characterized in that In step 1, a multi-scale feature fusion module is used to obtain multi-scale fusion features of remote sensing images, and the multi-scale feature fusion module includes: branch one, branch two, and branch three; Branch 1 adopts SA structure, obtains feature information along the channel dimension and spatial dimension and performs splicing, integrates the spatial position information of each group of features into the channel feature, and realizes feature fusion; Branch 2 uses traditional convolutional pooling to further mine deep feature information; Branch 1 and branch 2 interact locally across channels, aggregate multi-scale features using matrix multiplication, and finally combine with branch 3, fully combining the contextual information between channels and within space, realizing multi-scale feature fusion, and improving the feature mining and expression capabilities of the model.

3. The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion according to claim 1, It is characterized in that In step 2, the weighted feature Y is calculated as follows: Y=X⊙E Each neuron Y in Y c,i,j It is expressed as: Where: μ c is the mean value in channel c, is the variance within channel c, ε is a very small constant to prevent divisor, and its value is 0.

4. The on-orbit remote sensing target detection method based on deep learning and multi-scale feature fusion according to claim 1, It is characterized in that The EWD joint loss function in step 3 is as follows: Where: IOU represents the intersection-over-union ratio between the predicted box and the true box, α and γ are modulation factors, α is used to adjust the ratio between the positive and negative sample losses, γ is used to control the balance between simple and complex samples, and ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted box and the real box, c represents the diagonal distance of the minimum closure area of ​​the two boxes, C w and C h are the width and height of the smallest enclosing box containing the predicted box and the real box, respectively; ρ 2 (w,w gt ), ρ 2 (h,h gt ) represent the Euclidean distance between the width and height of the two boxes respectively.

Images