Target detection method and system for remote sensing image

Abstract

The invention discloses a target detection method and system for a remote sensing image. The method comprises the following steps: acquiring and preprocessing a to-be-detected remote sensing image; feature extraction is carried out on the preprocessed remote sensing image, and in the feature extraction process, at least one time of feature enhancement processing and at least one time of feature fusion processing are carried out; and outputting a prediction result of the to-be-detected target according to the features after the feature enhancement processing and the feature fusion processing. According to the scheme of the invention, the edge information of the target can be enhanced and the fine features can be captured through feature enhancement. Through feature fusion, feature maps of different levels can be combined, deep semantic information and shallow spatial detail information are complemented, and key detail loss caused by information mismatching is avoided. Through combination of feature enhancement and feature fusion processing, the feature expression ability of the model to various remote sensing targets under a complex background is significantly improved, so that the final accuracy and robustness of detection are improved.

Description

Technical Field

[0001] This application generally relates to the fields of computer vision and object detection technology. More specifically, this application relates to a method and system for object detection in remote sensing images. Background Technology

[0002] Target detection in remote sensing images, which aims to accurately identify and locate various targets of interest (such as aircraft, ships, and vehicles) from high-resolution remote sensing images, is a key technology in the field of computer vision. This technology has wide-ranging applications in many fields, including environmental protection, urban planning, agricultural management, disaster monitoring, and military reconnaissance.

[0003] With the rapid development of deep learning technology, object detection algorithms based on convolutional neural networks (CNNs) have become mainstream and have made significant progress in this field, greatly improving detection accuracy and efficiency. However, compared with natural images, remote sensing images exhibit inherent complex characteristics: targets often have arbitrary rotation angles, diverse categories, drastic size scale variations, complex and variable background environments, and are frequently densely arranged. These characteristics, especially the common problem of "uncertain angles" of targets, pose a significant challenge to object detection tasks.

[0004] For targets with variable angles (i.e., tilted), especially when the aspect ratio of the target varies greatly, traditional horizontal bounding box (HBB) detection methods tend to include a large number of irrelevant background regions, introducing background noise and interfering with feature extraction, thus affecting the final detection accuracy. To more accurately describe the position and orientation of the target, oriented bounding box (OBB) detection algorithms have emerged. These algorithms tightly enclose the target by adding directional information, effectively separating neighboring objects from the background, and have become the mainstream approach for target detection in remote sensing images. However, most existing OBB detection algorithms focus on angle constraints through loss functions, paying less attention to the feature distribution characteristics of the tilted target itself, especially the edge features that determine its orientation. Once edge information is lost or blurred during extraction, the model struggles to accurately determine the target angle, leading to a decrease in detection accuracy.

[0005] Furthermore, to address the challenges of varying target scales and complex backgrounds in remote sensing images, existing algorithms commonly employ Feature Pyramid Networks (FPNs) and their variants (such as BiFPNs), combined with attention mechanisms to enhance the extraction and fusion capabilities of multi-scale features. However, when fusing semantic information extracted from deep layers with spatial detail information extracted from shallow layers, inappropriate fusion strategies often lead to the loss of crucial detail information, particularly affecting the detection performance of small or densely packed targets.

[0006] In view of this, there is an urgent need to provide a target detection scheme for remote sensing images that can overcome the above-mentioned defects, effectively enhance the edge feature representation of tilted targets, and optimize the fusion strategy of multi-scale features to preserve key details. Summary of the Invention

[0007] In order to at least address one or more of the technical problems mentioned above, this application proposes target detection schemes for remote sensing images in several aspects.

[0008] In a first aspect, this application provides a target detection method for remote sensing images, comprising: acquiring and preprocessing the remote sensing image to be detected; performing feature extraction on the preprocessed remote sensing image, wherein at least one feature enhancement process and at least one feature fusion process are performed during the feature extraction process; and outputting a prediction result of the target to be detected based on the features after feature enhancement and feature fusion processes.

[0009] In some embodiments, during the preprocessing of the remote sensing image to be detected, the following steps are performed: cropping, correcting, and flipping the remote sensing image to be detected; processing the cropped, corrected, and flipped remote sensing image to a preset resolution; and inputting the processed remote sensing image to a pre-trained model for initialization.

[0010] In some embodiments, the feature enhancement process includes edge feature enhancement processing; during the edge feature enhancement processing, the Sobel operator is used to extract the edge features corresponding to the remote sensing image on the first branch path, the original features corresponding to the remote sensing image are extracted on the second branch path, and the extracted edge features are fused with the original features.

[0011] In some embodiments, the formula used in the edge feature enhancement process is: in, X is the input feature. This indicates a standard convolution operation with a 1×1 kernel. Sobel indicates a standard convolution operation with a 3×3 kernel.Convx For the horizontal Sobel operator, Sobel Convy For the Sobel operator in the vertical direction, f act The activation function is BN, which is a Batch Normalization layer. Concat represents the connection of channel-dimensional feature maps, and Y1 is the feature map after fusing edge features with the original features.

[0012] In some embodiments, the feature enhancement process further includes intra-layer multi-scale feature enhancement; during the intra-layer multi-scale feature enhancement process, the feature map after edge feature enhancement is sequentially passed through dilated convolutional layers with different dilation rates to obtain features of multiple receptive fields of different sizes, and the features of multiple receptive fields of different sizes are fused with the feature map after edge feature enhancement in the channel dimension; wherein, the formula used in the intra-layer multi-scale feature enhancement process is: F2 = AtrousConv3(F1), F3 = AtrousConv5(F2), Y1 is the feature map after edge feature enhancement processing, AtrousConv1 is a dilated convolution with a dilation rate of 1, AtrousConv3 is a dilated convolution with a dilation rate of 3, and AtrousConv5 is a dilated convolution with a dilation rate of 5. This indicates a standard convolution operation with a 1×1 kernel. Y1 represents a standard convolution operation with a 3×3 kernel, and Y2 represents a feature map that fuses features from multiple receptive fields of different sizes with the feature map after edge feature enhancement in the channel dimension.

[0013] In some embodiments, the feature enhancement process further includes wide-area attention enhancement processing; during the wide-area attention enhancement processing, local attention features and global attention features are extracted from the feature map obtained after the intra-layer multi-scale feature enhancement processing.

[0014] In some embodiments, the formula used in the wide-area attention enhancement process is: F out =F Local +F Global F Local For local attention features, F Global For global attention features; the expression for local attention features is: Avg(·) represents global average pooling, Max(·) represents global max pooling, Shufftle(·) represents channel shuffling, Y2 is the feature map obtained after multi-scale feature enhancement within the layer, represents a standard convolution operation with a 1×1 kernel, and exp(·) represents an exponential function.

[0015] In some embodiments, the expression for the global attention feature is: in, Q, K, and V are all weight matrices, dk is the dimension of weight matrix K, and Y2 is the feature map obtained after intra-layer multi-scale feature enhancement processing. This indicates a standard convolution operation with a 1×1 kernel, and exp(·) represents an exponential function.

[0016] In some embodiments, the feature fusion process includes inter-layer multi-scale attention dynamic fusion processing. During this process, the feature map after wide-area attention enhancement is fed into a weight allocator, assigning corresponding weights to the shallow and deep features after wide-area attention enhancement, and then performing weighted fusion processing on the shallow and deep features based on these weights. The formula used in this inter-layer multi-scale attention dynamic fusion processing is: W = Attention[BSAE(F)+F], F = YLow+YDeep, YLow is the shallow feature after wide-area attention enhancement, YDeep is the deep feature after wide-area attention enhancement, W is the weight corresponding to the shallow feature after wide-area attention enhancement, Y is the feature map after inter-layer multi-scale attention dynamic fusion processing, Attention(·) is the selected attention mechanism, and BSAE(·) indicates that wide-area attention enhancement processing is performed.

[0017] In a second aspect, this application provides a target detection system for remote sensing images, employing the target detection method for remote sensing images as described in any embodiment of the first aspect. The system includes: an image data preprocessing module for acquiring and preprocessing the remote sensing image to be detected; a feature extraction module for extracting features from the preprocessed remote sensing image, wherein at least one feature enhancement process and at least one feature fusion process are performed during feature extraction; and a target prediction module for outputting a prediction result of the target to be detected based on the features after feature enhancement and feature fusion processes.

[0018] The target detection scheme for remote sensing images provided above, in this application embodiment, enhances the edge information of targets and captures subtle features through feature enhancement, which is crucial for identifying targets with uncertain orientation, varying scale, and dense arrangement. Feature fusion combines feature maps from different levels, complementing deep semantic information with shallow spatial details, avoiding the loss of key details due to information mismatch. By combining feature enhancement and feature fusion during feature extraction, the model's ability to represent the features of various remote sensing targets in complex backgrounds is significantly improved, thereby enhancing the final detection accuracy and robustness.

[0019] Furthermore, in some embodiments, during the edge feature enhancement process, one branch utilizes the Sobel operator to specifically extract the edge contour features of the target, while another branch preserves the rich spatial and textural information of the original image. Fusing these two types of features enhances the model's ability to perceive target edge details while retaining complete contextual information. By highlighting the target's contour and boundaries, the algorithm's sensitivity to image details, especially the direction of rotating targets, is significantly improved. While enhancing edges, the integrity of key information such as coordinates and texture is ensured, enabling the algorithm to understand image content more comprehensively and accurately. Combining edge precision and spatial richness effectively enhances the ability to extract target features in complex backgrounds, ultimately improving the accuracy of target detection.

[0020] Furthermore, in some embodiments, during the intra-layer multi-scale feature enhancement process, by employing dilated convolutions with different dilation rates and fusing the feature maps after edge feature enhancement, the algorithm can effectively capture fine-grained target information at different scales within the same feature layer, thereby enhancing the feature representation capability for dense targets. Secondly, during the wide-area attention enhancement process, by combining local and global feature extraction methods, dual attention to key local details and long-range global structure is achieved. Local attention feature extraction utilizes global pooling and channel shuffling to enhance information interaction and feature representation between channels. Global attention feature extraction utilizes a self-attention mechanism to capture long-range dependencies between features. This complementarity of local and global attention features optimizes the feature fusion effect, especially improving the detection accuracy for irregularly shaped targets such as slender ones.

[0021] Furthermore, in some embodiments, intelligent weighted fusion of different feature maps is achieved through inter-layer multi-scale attention dynamic fusion processing. First, a weight is dynamically generated using an attention mechanism. Based on this weight, the two feature maps are weighted and summed to generate the final fused feature map. This is not a simple merging of the two feature maps, but rather allows the model to intelligently determine which feature is more important in the current context based on the actual situation of the input features, thus assigning it a higher weight. This enables the model to flexibly balance and utilize edge information and multi-scale detail information, achieving complementary advantages and ultimately improving the overall performance and accuracy of target detection in complex remote sensing scenarios. Attached Figure Description

[0022] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0023] Figure 1 An exemplary flowchart of a target detection method for remote sensing images according to an embodiment of this application is shown;

[0024] Figure 2 A structural diagram of the overall detection model according to an embodiment of this application is shown;

[0025] Figure 3 A structural diagram of the edge feature enhancement module according to an embodiment of this application is shown;

[0026] Figure 4 A structural diagram of an intra-layer multi-scale feature enhancement extractor according to an embodiment of this application is shown;

[0027] Figure 5 A structural diagram of the wide-area attention enhancement module according to an embodiment of this application is shown;

[0028] Figure 6 A structural diagram of the multi-scale attention mechanism dynamic fusion module according to an embodiment of this application is shown;

[0029] Figure 7 The following is a comparison chart of detections on the HRSC2016 dataset based on an embodiment of this application;

[0030] Figure 8 This paper shows a visualization of the detection on the DIOR-R dataset according to an embodiment of this application;

[0031] Figure 9 A thermal comparison diagram of the EIMYOLO model and the YOLO11 model according to an embodiment of this application is shown;

[0032] Figure 10 An iterative comparison diagram of embodiments of this application is shown;

[0033] Figure 11 An exemplary structural block diagram of a target detection system for remote sensing images according to an embodiment of this application is shown. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0036] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0037] Figure 1 An exemplary flowchart of a target detection method 100 for remote sensing images according to an embodiment of this application is shown.

[0038] like Figure 1 As shown, in step S110, the remote sensing image to be detected is acquired and preprocessed.

[0039] In the embodiments of this application, during the acquisition of the remote sensing image to be detected, two remote sensing image datasets with common rotated bounding boxes were obtained: the HRSC2016 dataset and the DIOR-R dataset. First, the datasets were analyzed. The HRSC2016 dataset contains 1061 images and 2976 instances across 3 major classes and 27 minor classes. These instances are labeled with rotated bounding boxes, allowing for the validation of the rotated bounding box object detection algorithm. Image sizes range from 300×300 to 1500×900 pixels. The DIOR-R dataset contains 23,463 images and 192,472 instances, covering 20 different object categories, including airplanes (AP), airports (AT), baseball fields (BF), basketball courts (BC), bridges (BD), chimneys (CM), dams (DAM), highway service areas (ESA), highway toll booths (ETS), ports (HB), golf courses (GF), ground runways (GTF), overpasses (OP), ships (SH), stadiums (STA), storage tanks (STO), tennis courts (TC), train stations (TS), vehicles (VE), and windmills (WM). Each image is fixed at 800×800 pixels. Both datasets contain images with typical remote sensing image characteristics, such as variable angles, diverse categories, varying scales, complex backgrounds, and dense arrangement.

[0040] In the embodiments of this application, during the preprocessing of the remote sensing image to be detected, firstly, the remote sensing image to be detected is cropped, corrected, and flipped. Next, the remote sensing image after cropping, correction, and flipping is processed to a preset resolution. Then, the remote sensing image processed to the preset resolution is input into a pre-trained model for initialization.

[0041] In some embodiments of this application, remote sensing images are uniformly cropped to a resolution of 1024. In other embodiments of this application, remote sensing images may also be cropped according to actual needs, and this application does not impose any restrictions here.

[0042] By cropping, correcting, and flipping remote sensing images, the diversity of the training dataset is greatly enriched. This can effectively improve the generalization ability of the model, enabling it to adapt to targets with different angles and poses, thereby enhancing the robustness of the model and preventing overfitting.

[0043] After step S110 is completed, in step S120, feature extraction is performed on the preprocessed remote sensing image, wherein at least one feature enhancement process and at least one feature fusion process are performed during the feature extraction process.

[0044] In the embodiments of this application, the feature enhancement processing includes edge feature enhancement processing, intra-layer multi-scale feature enhancement processing, and wide-area attention enhancement processing. In the edge feature enhancement processing, an edge feature enhancement module is used. In the intra-layer multi-scale feature enhancement processing, an intra-layer multi-scale feature enhancement extractor is used. In the wide-area attention enhancement processing, a wide-area attention enhancement module is used.

[0045] In the embodiments of this application, the feature fusion processing includes inter-layer multi-scale attention dynamic fusion processing, and a multi-scale attention mechanism dynamic fusion module is used in the process of inter-layer multi-scale attention dynamic fusion processing.

[0046] The specific process involved in step S120 in the embodiments of this application can be found in [reference needed]. Figure 2 .

[0047] Figure 2 A structural diagram of the overall detection model according to an embodiment of this application is shown.

[0048] like Figure 2 As shown, the overall detection model adopts EIMYOLO, which is an improvement on the YOLO11-obb algorithm and is mainly divided into three parts: backbone network, neck structure and head structure.

[0049] The backbone network is responsible for extracting shallow and deep features from the preprocessed remote sensing images. The backbone network consists of multiple stages (p1 to p5). As the network deepens, the size of the feature maps gradually decreases (e.g., p3 stage is 160×160, representing shallow features, while p5 stage is 40×40, representing deep features), but the semantic information becomes richer. Simultaneously, the EFRB (Edge Feature Enhancement Module) is embedded in the backbone network. This module uses the Sobel operator to obtain enhanced edge features, which are then combined with the originally extracted spatial features, improving the backbone network's boundary perception capability. This helps in identifying remote sensing targets with complex backgrounds and arbitrary orientations.

[0050] The neck structure serves to fuse features extracted from different layers of the backbone network to obtain a more robust multi-scale feature representation. EIMYOLO introduces two core improved modules here: IFEE (Intra-Layer Multi-Scale Feature Enhancement Extractor) and MSADF (Multi-Scale Attention Dynamic Fusion Module). The output layers at three different scales (p3, p4, and p5) of the backbone network all point to an IFEE. The IFEE enhances the single-scale feature map before it enters the feature pyramid for fusion. It captures multi-scale information within the same layer without changing the feature map size through dilated convolutions with different dilation rates, which is particularly effective for detecting densely packed small targets. At the end of the feature fusion path in the neck structure (which includes upsampling and fusion operations), the features are fed into the MSADF. The MSADF replaces the simple concatenation or addition operations found in traditional YOLO.

[0051] MSADF integrates a wide-area attention enhancement module (BASE), which runs in parallel with local and global attention branches. The global branch uses a self-attention mechanism to capture long-distance dependencies, while the local branch aggregates key local information through operations such as channel shuffling and pooling. These two branches complement each other, optimizing feature weights and improving the detection accuracy for slender targets. By assigning different weights to features from different levels (e.g., shallow detail features from p3 and deep semantic features from p5) through the wide-area attention enhancement module (BASE), features are then fused. This resolves the information conflict that can arise from directly fusing features from different levels, resulting in a more powerful fused feature representation.

[0052] In the embodiments of this application, a comparative experiment on the attention hyperparameters of MSADF (Multi-Scale Attention Mechanism Dynamic Fusion Module) was conducted, and the results are shown in Table 1:

[0053] Table 1 Comparison of Attention Hyperparameters

[0054] MSADF FLOPS(G) Params(M) mAP(%) No attention 29.1 11.2 90.00 +XA 30.3 12.3 90.08 +EMA 31.8 14.2 90.40 +SA 29.4 11.3 90.64 +CA 29.9 11.2 90.80

[0055] According to Table 1, the attention selection channel attention mechanism (CA) in the wide-area attention enhancement module (BASE) is described.

[0056] The head structure is used to predict the position, class, and rotation angle of an object on a fused feature map at three different scales using an OBB (Oriented Bounding Box) detector. The OBB detector is specifically designed to predict the bounding box of rotated objects, which is crucial for non-horizontal and non-vertical targets commonly found in remote sensing images, such as aircraft and ships.

[0057] The specific process of edge feature enhancement processing in the embodiments of this application can be found in [reference needed]. Figure 3 .

[0058] Figure 3 A structural diagram of the edge feature enhancement module according to an embodiment of this application is shown.

[0059] like Figure 3 As shown, the Edge Feature Enhancement (EFRB) module divides the input image into two paths for processing. In the first branch path, the Sobel operator is used to extract edge features corresponding to the remote sensing image. In the second branch path, the original features corresponding to the remote sensing image are extracted to maintain rich spatial information. Then, the extracted edge features are fused with the original features.

[0060] Specifically, the first branch path uses the Sobel operator to extract edge information from the image. It obtains the gradient magnitude of each pixel in the horizontal and vertical directions using two 3×3 convolution kernels, SobelConvx and SobelConvy, respectively. The edge strength and direction are then calculated based on the gradient magnitude changes. Furthermore, Sobel convolution reduces noise in the image, and its symmetrical kernels, with the central kernel set to 0, ensure that the overall brightness of the image is not altered when calculating the gradient.

[0061] The edge feature enhancement module aims to preserve the original features of the image while enhancing edge information, thereby improving the algorithm's ability to capture image details. It not only highlights contours and boundaries in the image but also ensures the integrity of key elements such as contextual information and target coordinates, enabling the algorithm to understand image content more comprehensively and accurately. By replacing the Bottleneck portion in the original YOLO11-obb C3K2 with the Edge Feature Enhancement Module (EFRB), the C3K2EFRB module is generated. Utilizing the enhanced edge features, the algorithm's orientation sensitivity can be better improved, making it more compatible with rotating targets.

[0062] In the embodiments of this application, the formula used in the edge feature enhancement process is: in, X is the input feature. This indicates a standard convolution operation with a 1×1 kernel. Sobel indicates a standard convolution operation with a 3×3 kernel. Convx For the horizontal Sobel operator, Sobel Convy For the Sobel operator in the vertical direction, f act The activation function is BN, which is a Batch Normalization layer. Concat represents the connection of channel-dimensional feature maps, and Y1 is the feature map after fusing edge features with the original features.

[0063] The specific process of performing intra-layer multi-scale feature enhancement processing in the embodiments of this application can be found in [reference needed]. Figure 4 .

[0064] Figure 4 A structural diagram of an intra-layer multi-scale feature enhancement extractor according to an embodiment of this application is shown.

[0065] like Figure 4 As shown, the intra-layer multi-scale feature enhancement extractor receives the input feature map after edge feature enhancement processing. It first passes this input feature map sequentially through dilated convolutional layers with dilation rates of 1, 3, and 5, generating three feature maps of different sizes: a basic receptive field feature map F1, a wide receptive field feature map F2, and an ultra-wide receptive field feature map F3. This sequential dilated convolution operation not only maintains the original resolution of the feature maps but also effectively captures receptive fields of different scales, ensuring that each layer's feature map can fully receive all the information passed from the previous layer, minimizing the loss of effective features. Subsequently, these three multi-scale receptive field feature maps are fused with the edge feature enhancement-processed feature map along the channel dimension to enhance the expressive power of the features. By integrating the intra-layer multi-scale feature enhancement extractor after the three layers of downsampled feature maps at different scales output by the Backbone network, this design provides local attention feature enhancement for the portion fed into the Neck network. Combined with subsequent cross-layer multi-scale feature fusion operations, this effectively improves the algorithm's ability to extract features over short and long distances.

[0066] In the embodiments of this application, the formula used in the intra-layer multi-scale feature enhancement process is: in, F2 = AtrousConv3(F1), F3 = AtrousConv5(F2), Y1 is the feature map after edge feature enhancement processing, AtrousConv1 is a dilated convolution with a dilation rate of 1, AtrousConv3 is a dilated convolution with a dilation rate of 3, and AtrousConv5 is a dilated convolution with a dilation rate of 5. This indicates a standard convolution operation with a 1×1 kernel. Y1 represents a standard convolution operation with a 3×3 kernel, and Y2 represents a feature map that fuses features from multiple receptive fields of different sizes with the feature map after edge feature enhancement in the channel dimension.

[0067] The specific process of performing wide-area attention enhancement processing in the embodiments of this application can be found in [reference needed]. Figure 5 .

[0068] Figure 5 A structural diagram of the wide-area attention enhancement module according to an embodiment of this application is shown.

[0069] like Figure 5 As shown, during the wide-area attention enhancement process, local attention features and global attention features are extracted from the feature map obtained after the multi-scale feature enhancement process within the layer.

[0070] Specifically, a self-attention mechanism is employed for global attention feature extraction. First, a three-dimensional weight matrix Q, K, and V with dimensions W×H×C is obtained through depthwise separable convolution. Then, the attention score is calculated using the weight matrices Q, K, and V, and the number of channels is adjusted through 1×1 convolution to obtain a feature map containing global information.

[0071] Specifically, in extracting local attention features, the feature maps obtained after multi-scale feature enhancement within the layer are first grouped through a channel-dimensional shuffling operation, and then convolutional operations are performed on each group. This process not only enhances the information interaction between different channels but also further improves the expressive power of the features. Then, global average pooling and global max pooling are used to aggregate spatial information, and dimensionality reduction and expansion operations are performed through standard 1×1 convolutions. Finally, the weights are normalized using the softmax activation function to obtain feature maps rich in local attention.

[0072] In the embodiments of this application, the formula used in the wide-area attention enhancement process is: F out =F Local +F Global F Local For local attention features, F Global This is a global attention feature.

[0073] Specifically, the expression for the local attention feature is: Avg(·) represents global average pooling, Max(·)

[0074] This indicates global max pooling, represented by `Shufftle(·)`.

[0075] This indicates a channel shuffling operation, where Y2 is the feature map obtained after in-layer multi-scale feature enhancement processing, and Y represents a 1× convolution kernel.

[0076] The standard convolution operation is 1, where exp(·) represents the exponential function.

[0077] Specifically, the expression for the global attention feature is: in,

[0078] Q, K, and V are all weight matrices, dk is the dimension of weight matrix K, and Y2 is the result of in-layer multi-scale feature enhancement processing. This indicates a standard convolution operation with a 1×1 kernel, and exp(·) represents an exponential function.

[0079] For details on the specific process of performing inter-layer multi-scale attention dynamic fusion processing in the embodiments of this application, please refer to [link / reference]. Figure 6 .

[0080] Figure 6 A structural diagram of the multi-scale attention mechanism dynamic fusion module according to an embodiment of this application is shown.

[0081] like Figure 6 As shown, during the inter-layer multi-scale attention dynamic fusion process, the feature map after wide-area attention enhancement is fed into the weight allocator to assign corresponding weights to the shallow and deep features after wide-area attention enhancement.

[0082] Weights are assigned, and the shallow and deep features after wide-area attention enhancement are weighted and fused based on these weights.

[0083] Specifically, a channel attention mechanism (CA) is employed for dynamic weight calculation, assigning complementary weight values ​​to the shallow and deep features after wide-area attention enhancement, thus achieving multi-scale weighted fusion. In the overall model, the multi-scale attention mechanism...

[0084] The Dynamic Fusion Module (MSADF) is integrated before the detection head. This module first applies the wide-area attention signal before it is fed into the detection head.

[0085] Multi-scale fusion of enhanced shallow and deep features not only integrates intra-layer features but also effectively...

[0086] It integrates cross-level features, forming a complementary information flow.

[0087] In the embodiments of this application, the formula used in the inter-layer multi-scale attention dynamic fusion processing is as follows: Where W = Attention[BSAE(F)+F], F = YLow+YDeep, YLow is the shallow feature after wide-area attention enhancement, YDeep is the deep feature after wide-area attention enhancement, W is the weight corresponding to the shallow feature after wide-area attention enhancement, Y is the feature map after inter-layer multi-scale attention dynamic fusion processing, Attention(·) is the selected attention mechanism, and BSAE(·) indicates that wide-area attention enhancement processing is performed.

[0092] After step S120 is completed, in step S130, the prediction result of the target to be detected is output based on the features after feature enhancement and feature fusion processing.

[0094] In the embodiments of this application, the features after feature enhancement and feature fusion are input into the OBB (Oriented Bounding Box) detection head of the overall detection model to obtain the predicted bounding box and predicted category information of the target to be detected.

[0096] In the embodiments of this application, during the training of the overall detection model, the initial learning rate is set to 0.001, the minimum learning rate is 0.0001, the training IoU threshold is set to 0.7, and the momentum and weight decay are set to 0.937 and 0.0005, respectively. Simultaneously, based on the model convergence graph comparison experiment analysis, the epoch is set to 400. The Adam optimizer is used to optimize the algorithm parameters during neural network training.

[0097] In the embodiments of this application, the detection results of this application on the HRSC2016 dataset and the DIOR-R dataset are analyzed. Based on... Figure 7 and Figure 8For the HRSC2016 dataset, this algorithm ignores the influence of noise factors such as water flow in ship images with interference, accurately locating the target itself with an accuracy 59% higher than the baseline. For ship images with uncertain orientation, this algorithm optimizes the false detection caused by background similarity and the missed detection caused by large angle differences. For port images with complex backgrounds, the baseline algorithm is more affected by background interference, resulting in a detection rate of only 44%, while this algorithm improves the detection rate by 45%. Faced with blurred images of docks and ships with very similar features, this algorithm improves the accuracy by 3% compared to the baseline, and there are no false detections. For the DIOR-R dataset, it can detect various types of targets well, such as targets with high inter-class similarity like tennis courts and basketball courts, slender targets like dams, airports, and train stations, and densely overlapping targets like ships and docks. It is evident that this algorithm still has excellent target detection performance for complex and diverse remote sensing images. Experimental results analysis shows the robustness of the proposed method to images with diverse categories.

[0098] In the embodiments of this application, the heatmaps corresponding to the EIMYOLO model and the YOLO11 model of this application are compared respectively. Figure 9 From two datasets, we selected dense, slender, small, and complex background targets with high recognition difficulty and conducted comparative experiments on image feature heatmap visualization. It can be seen that for remote sensing images of docks with dense ship targets, the baseline algorithm YOLO11s focuses more on environmental factors such as the dock when dealing with dense targets, while this algorithm mainly focuses on the target itself, accurately locating even the smaller interiors of densely packed ships and avoiding interference from irrelevant factors. For remote sensing images of slender overpasses, it better captures long-distance continuous features, fully focusing on key features of the bridge structure, effectively solving the problem of incomplete targets caused by the baseline algorithm mainly focusing on the bridgehead area. For remote sensing images of small aircraft, the baseline algorithm can predict the approximate location of the target, while this algorithm captures the aircraft target more accurately and with stronger coherence, avoiding missed targets. For remote sensing images of ports with complex backgrounds, this algorithm reduces background noise interference and better covers all the targets to be detected, not just the ship bows, enabling better overall target localization. Through the analysis of these four types of heatmap experiments, it is evident that this algorithm has a better target feature capture capability.

[0099] In the embodiments of this application, the iteration processes corresponding to the EIMYOLO model, YOLO11 model and YOLOV8 model of this application are compared. Compared with the YOLO11 model and YOLOV8 model, the EIMYOLO model of this application has a higher prediction accuracy as the number of iterations increases.

[0100] In summary, the target detection scheme for remote sensing images provided above enhances the edge information of targets and captures subtle features through feature enhancement, which is crucial for identifying targets with variable orientations, scales, and dense arrangement. Feature fusion combines feature maps from different levels, complementing deep semantic information with shallow spatial details, thus avoiding the loss of crucial details due to information mismatch. By combining feature enhancement and feature fusion during feature extraction, the model's ability to represent the features of various remote sensing targets in complex backgrounds is significantly improved, thereby enhancing the final detection accuracy and robustness.

[0101] Furthermore, in some embodiments, during the edge feature enhancement process, one branch utilizes the Sobel operator to specifically extract the edge contour features of the target, while another branch preserves the rich spatial and textural information of the original image. Fusing these two types of features enhances the model's ability to perceive target edge details while retaining complete contextual information. By highlighting the target's contour and boundaries, the algorithm's sensitivity to image details, especially the direction of rotating targets, is significantly improved. While enhancing edges, the integrity of key information such as coordinates and texture is ensured, enabling the algorithm to understand image content more comprehensively and accurately. Combining edge precision and spatial richness effectively enhances the ability to extract target features in complex backgrounds, ultimately improving the accuracy of target detection.

[0102] Furthermore, in some embodiments, during the intra-layer multi-scale feature enhancement process, by employing dilated convolutions with different dilation rates and fusing the feature maps after edge feature enhancement, the algorithm can effectively capture fine-grained target information at different receptive field scales within the same feature layer, thereby enhancing the feature representation capability for dense targets. Secondly, during the wide-area attention enhancement process, by combining local and global feature extraction methods, dual attention to key local details and long-range global structure is achieved. Local attention feature extraction utilizes global pooling and channel shuffling to enhance information interaction and feature representation between channels. Global attention feature extraction utilizes a self-attention mechanism to capture long-range dependencies between features. This complementarity of local and global attention features optimizes the feature fusion effect, especially improving the detection accuracy for irregularly shaped targets such as slender ones.

[0103] Furthermore, in some embodiments, intelligent weighted fusion of different feature maps is achieved through inter-layer multi-scale attention dynamic fusion processing. First, a weight is dynamically generated using an attention mechanism. Based on this weight, the two feature maps are weighted and summed to generate the final fused feature map. This is not a simple merging of the two feature maps, but rather allows the model to intelligently determine which feature is more important in the current context based on the actual situation of the input features, thus assigning it a higher weight. This enables the model to flexibly balance and utilize edge information and multi-scale detail information, achieving complementary advantages and ultimately improving the overall performance and accuracy of target detection in complex remote sensing scenarios.

[0104] This application also provides a target detection system for remote sensing images, which can use the aforementioned target detection method 100 for remote sensing images to perform target detection for remote sensing images, or other methods can be used for target detection for remote sensing images. This application does not impose any limitations on this.

[0105] Figure 11 An exemplary structural block diagram of a target detection system for remote sensing images according to an embodiment of this application is shown.

[0106] like Figure 11 As shown, the system 1100 includes an image data preprocessing module 1110, a feature extraction module 1120, and a target prediction module 1130. In the embodiments of this application, the image data preprocessing module 1110, the feature extraction module 1120, and the target prediction module 1130 may be separate units or integrated in the same controller; this application does not impose any restrictions here.

[0107] Specifically, the image data preprocessing module 1110 is used to acquire and preprocess the remote sensing image to be detected.

[0108] Specifically, the feature extraction module 1120 is used to extract features from the preprocessed remote sensing image, wherein at least one feature enhancement process and at least one feature fusion process are performed during the feature extraction process.

[0109] Specifically, the target prediction module 1130 is used to output the prediction result of the target to be detected based on the features after feature enhancement and feature fusion processing.

[0110] When system 1100 performs target detection on remote sensing images using the aforementioned target detection method 100 for remote sensing images, the image data preprocessing module 1110 executes the aforementioned step S110, the feature extraction module 1120 executes the aforementioned step S120, and the target prediction module 1130 executes the aforementioned step S130. The specific execution process can be found above and will not be repeated here.

[0111] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A target detection method for remote sensing images, characterized in that, include: Acquire and preprocess the remote sensing image to be detected; Feature extraction is performed on the preprocessed remote sensing image, wherein at least one feature enhancement process and at least one feature fusion process are performed during the feature extraction process; Based on the features after feature enhancement and feature fusion processing, the prediction result of the target to be detected is output.

2. The target detection method for remote sensing images according to claim 1, characterized in that, The following steps are performed during the preprocessing of the remote sensing image to be detected: The remote sensing images to be detected are processed by cropping, correction, and flipping. The remote sensing image, after being cropped, corrected, and flipped, is processed to a preset resolution; The remote sensing image, processed to a preset resolution, is input into the pre-trained model for initialization.

3. The target detection method for remote sensing images according to claim 1, characterized in that, The feature enhancement process includes edge feature enhancement processing; During the edge feature enhancement process, the Sobel operator is used to extract the edge features corresponding to the remote sensing image on the first branch path, and the original features corresponding to the remote sensing image are extracted on the second branch path. The extracted edge features are then fused with the original features.

4. The target detection method for remote sensing images according to claim 3, characterized in that, The formula used in the edge feature enhancement process is: in, X is the input feature. This indicates a standard convolution operation with a 1×1 kernel. Sobel indicates a standard convolution operation with a 3×3 kernel. Convx For the horizontal Sobel operator, Sobel Convy For the Sobel operator in the vertical direction, f act The activation function is BN, which is a Batch Normalization layer. Concat represents the connection of channel-dimensional feature maps, and Y1 is the feature map after fusing edge features with the original features.

5. The target detection method for remote sensing images according to claim 3, characterized in that, The feature enhancement process also includes intra-layer multi-scale feature enhancement processing; In the process of performing multi-scale feature enhancement within the layer, the feature map after edge feature enhancement is passed through dilated convolutional layers with different dilation rates to obtain features of multiple receptive fields of different sizes. The features of multiple receptive fields of different sizes are then fused with the feature map after edge feature enhancement in the channel dimension. The formula used in the intra-layer multi-scale feature enhancement process is as follows: F2 = AtrousConv3(F1), F3 = AtrousConv5(F2), Y1 is the distance from F1 to F2. The feature maps after edge feature enhancement processing: AtrousConv1 is a dilated convolution with a dilation rate of 1, AtrousConv3 is a dilated convolution with a dilation rate of 3, and AtrousConv5 is a dilated convolution with a dilation rate of 5. This indicates a standard convolution operation with a 1×1 kernel. Y1 represents a standard convolution operation with a 3×3 kernel, and Y2 represents a feature map that fuses features from multiple receptive fields of different sizes with the feature map after edge feature enhancement in the channel dimension.

6. The target detection method for remote sensing images according to claim 5, characterized in that, The feature enhancement process also includes wide-area attention enhancement processing; During the wide-area attention enhancement process, local attention features and global attention features are extracted from the feature map obtained after the intra-layer multi-scale feature enhancement process.

7. The target detection method for remote sensing images according to claim 6, characterized in that, The formula used in the wide-area attention enhancement process is: F out =F Local +F Global F Local For local attention features, F Global This is a global attention feature; The expression for the local attention feature is: Avg(·) represents global average pooling, Max(·) represents global max pooling, Shufftle(·) represents channel shuffling, and Y2 is the feature map obtained after in-layer multi-scale feature enhancement. This indicates a standard convolution operation with a 1×1 kernel, and exp(·) represents an exponential function.

8. The target detection method for remote sensing images according to claim 7, characterized in that, The expression for global attention features is: in, Q, K, and V are all weight matrices, d k Let K be the dimension of the weight matrix, and Y2 be the feature map obtained after in-layer multi-scale feature enhancement processing. This indicates a standard convolution operation with a 1×1 kernel, and exp(·) represents an exponential function.

9. The target detection method for remote sensing images according to claim 6, characterized in that, The feature fusion process includes inter-layer multi-scale attention dynamic fusion processing; During the inter-layer multi-scale attention dynamic fusion process, the feature map after wide-area attention enhancement is passed to the weight allocator, and corresponding weights are assigned to the shallow and deep features after wide-area attention enhancement. Then, the shallow and deep features after wide-area attention enhancement are weighted and fused based on the weights. The formula used in the inter-layer multi-scale attention dynamic fusion process is as follows: W = Attention[BSAE(F)+F], F = YLow+YDeep, YLow is the shallow feature after wide-area attention enhancement, YDeep is the deep feature after wide-area attention enhancement, W is the weight corresponding to the shallow feature after wide-area attention enhancement, Y is the feature map after inter-layer multi-scale attention dynamic fusion processing, Attention(·) is the selected attention mechanism, and BSAE(·) indicates that wide-area attention enhancement processing is performed.

10. A target detection system for remote sensing images, characterized in that, The system employs the target detection method for remote sensing images as described in any one of claims 1-9 to perform target detection for remote sensing images, the system comprising: The image data preprocessing module is used to acquire and preprocess the remote sensing images to be detected; The feature extraction module is used to extract features from the preprocessed remote sensing image. During the feature extraction process, at least one feature enhancement process and at least one feature fusion process are performed. The target prediction module is used to output the prediction result of the target to be detected based on the features after feature enhancement and feature fusion processing.

Images