Target detection method and device for remote sensing image
By extracting multi-scale contextual features and performing adaptive modulation and spatial guidance enhancement, the problem of low accuracy in small target detection in remote sensing images is solved, and higher target recognition accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202510777358.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-09-19
AI Technical Summary
The detection of small targets in existing remote sensing images faces the problems of limited size, susceptibility to downsampling and feature loss, complex background and numerous interference factors, resulting in unclear feature expression, difficulty in category discrimination and low detection accuracy.
By extracting multi-scale contextual features from remote sensing images and performing adaptive modulation and spatially guided enhancement, target-related areas are dynamically screened, irrelevant background activations are suppressed, and residual connections are used to identify targets.
It improves the accuracy and robustness of target detection in remote sensing images, enhances the stability of the training process, reduces feature aliasing problems, and improves the accuracy of target recognition.
Smart Images
Figure CN120673274A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a target detection method and device for remote sensing images. Background Art
[0002] Remote sensing target detection is a fundamental task in remote sensing image analysis, widely used in fields such as military reconnaissance, environmental monitoring, disaster assessment, and urban planning. With the rapid development of Earth observation technology, the amount of satellite and aerial imagery acquired has increased exponentially. Therefore, it is crucial to quickly and accurately locate and classify various targets in complex remote sensing imagery. In recent years, deep learning-based methods have become the mainstream technical solution for remote sensing target detection due to their superior feature representation capabilities and good scalability. Representative methods include two-stage detectors (such as Faster R-CNN) and one-stage detectors (such as the YOLO series and RetinaNet), which have achieved excellent results in large-scale and medium-scale target detection.
[0003] However, existing methods still face significant challenges in remote sensing small target detection. Small targets are typically limited in size and occupy a very small area, making them susceptible to downsampling and feature loss. Furthermore, remote sensing images feature complex backgrounds and numerous interfering factors. Small targets often closely resemble their surroundings in color, texture, and shape, blurring the boundaries between foreground and background. This leads to unclear feature representation and difficulty in classifying objects, severely impacting detection accuracy.
[0004] Therefore, there is an urgent need for an effective mechanism to specifically enhance foreground saliency and suppress background interference to improve the accuracy of target detection. Summary of the Invention
[0005] The present application provides a method and device for target detection in remote sensing images, which can improve the accuracy of target detection in remote sensing images.
[0006] In a first aspect, an embodiment of the present application provides a method for detecting an object in a remote sensing image, characterized in that the method includes:
[0007] Acquisition of remote sensing images;
[0008] Extract multi-scale context features from remote sensing images. Context features are used to indicate the associated information and environmental clues of targets in remote sensing images at different dimensions.
[0009] Performing context-driven adaptive modulation on multi-scale context features to obtain first context-enhanced features of remote sensing images;
[0010] performing a spatially guided feature enhancement operation on the first context enhancement feature to generate a spatially guided enhancement feature, where the spatially guided enhancement feature is used to indicate that a spatial feature of a foreground of the remote sensing image is stronger than a spatial feature of a background of the remote sensing image;
[0011] Performing double residual connection processing on the spatial guidance enhancement feature, the remote sensing image and the first context enhancement feature to obtain the output feature;
[0012] Based on the output features, the targets in the remote sensing images are located and identified.
[0013] According to the above scheme, by extracting multi-scale contextual features from remote sensing images and adaptively modulating the contextual features at different scales, the complementarity of information at different scales can be fully utilized to obtain context-enhanced features. In this way, by adjusting the spatial attention of the first context-enhanced feature, a spatially guided enhancement feature is obtained. In the spatially guided enhancement feature, the spatial features of the foreground are stronger than those of the background. This allows for dynamic screening of target-related regions and suppresses activation of irrelevant background, thereby increasing the separability between foreground and background in feature space and effectively alleviating feature aliasing. Next, by performing a residual connection on the remote sensing image, the spatially guided enhancement feature, and the first context-enhanced feature, an output feature is obtained, and targets in the remote sensing image are identified based on the output feature. This preserves the underlying information flow of the input features in the remote sensing image while enhancing the features, avoiding information distortion caused by excessive feature enhancement. This not only improves the stability of the training process (making it easier to converge), but also enhances the separability of the target from the complex background, improving target recognition accuracy and robustness.
[0014] In one possible implementation, performing context-driven adaptive modulation on multi-scale context features to obtain first context-enhanced features of the remote sensing image includes:
[0015] The multi-scale context features are spliced together to obtain aggregated features;
[0016] Based on the aggregated features, the weights of the multi-scale contextual features are generated;
[0017] According to the respective weights of the multi-scale context features, the multi-scale context features are modulated respectively to obtain the multi-scale context modulation features;
[0018] The multi-scale context modulation features are spliced in pairs in the channel dimension to obtain the spatial modulation masks corresponding to the multi-scale context modulation features;
[0019] According to the spatial modulation masks corresponding to the multi-scale context modulation features, the multi-scale context modulation features are dynamically weighted and fused to obtain the multi-scale second context enhancement features;
[0020] A first context enhancement feature is generated according to the multi-scale second context enhancement feature.
[0021] In this way, more fine-grained information can be retained.
[0022] In a possible implementation, performing a spatially guided feature enhancement operation on the first contextual enhancement feature to generate a spatially guided enhancement feature includes:
[0023] The first context enhancement feature is processed through the spatial gated attention mechanism to obtain the spatially guided enhancement feature.
[0024] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0025] In one possible implementation, the first context enhancement feature is processed by a spatial gated attention mechanism to obtain a spatially guided enhancement feature, including:
[0026] Separating the first context-enhanced feature to obtain a gated branch feature and a context-branch feature;
[0027] According to the gated branch features and contextual branch features, spatially guided enhanced features are generated.
[0028] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0029] In one possible implementation, locating and identifying a target in the remote sensing image based on the output feature includes:
[0030] Based on the output features, the target in the remote sensing image is located and the type of the target is determined.
[0031] In a second aspect, the present application provides a target detection device for remote sensing images, the device comprising:
[0032] Acquisition module, used to acquire remote sensing images;
[0033] The extraction module is used to extract multi-scale context features from remote sensing images. The context features are used to indicate the associated information and environmental clues of the target in the remote sensing image at different dimensions;
[0034] A modulation module, configured to perform context-driven adaptive modulation on the multi-scale context features to obtain a first context-enhanced feature of the remote sensing image;
[0035] an adjustment module, configured to adjust the spatial attention of the first context enhancement feature to obtain a spatial guidance enhancement feature, wherein the spatial guidance enhancement feature is configured to indicate that the spatial features of the foreground of the remote sensing image are stronger than the spatial features of the background of the remote sensing image;
[0036] a processing module, configured to perform double residual connection processing on the spatial guidance enhancement feature, the remote sensing image, and the first context enhancement feature to obtain an output feature;
[0037] The recognition module is used to identify targets in remote sensing images based on output features.
[0038] According to the above scheme, multi-scale contextual features are extracted from remote sensing images and adaptively modulated at different scales, thereby fully leveraging the complementarity of information at different scales to obtain a first contextual enhancement feature. Similarly, by adjusting the spatial attention of the first contextual enhancement feature, a spatial guidance enhancement feature is obtained. In the spatial guidance enhancement feature, the spatial features of the foreground are stronger than those of the background. This allows for dynamic selection of target-related regions and suppresses activation of irrelevant background, thereby increasing the separability between foreground and background in feature space and effectively alleviating feature aliasing. Next, a residual connection is performed on the remote sensing image, the spatial guidance enhancement feature, and the first contextual enhancement feature to obtain an output feature, which is then used to identify targets in the remote sensing image. This enhances the features while preserving the underlying information flow of the input features in the remote sensing image, avoiding information distortion caused by excessive feature enhancement. This not only improves the stability of the training process (making it easier to converge), but also enhances the separability of the target from the complex background, improving target recognition accuracy, and enhancing both the accuracy and robustness of target recognition.
[0039] In a possible implementation, the modulation module is configured to:
[0040] Performing splicing processing on the multi-scale context features to obtain aggregated features;
[0041] Generating weights of the multi-scale context features according to the aggregated features;
[0042] Modulating the multi-scale context features according to their respective weights to obtain the multi-scale context modulation features;
[0043] splicing the multi-scale context modulation features in pairs in the channel dimension to obtain spatial modulation masks corresponding to the multi-scale context modulation features;
[0044] Performing dynamic weighted fusion on the multi-scale context modulation features according to the spatial modulation masks corresponding to the multi-scale context modulation features, respectively, to obtain the multi-scale second context enhancement features;
[0045] The first context enhancement feature is generated according to the multi-scale second context enhancement feature.
[0046] In this way, the amount of computation can be reduced and more fine-grained information can be retained.
[0047] In one possible implementation, the adjustment module is configured to process the first context enhancement feature through a spatial gated attention mechanism to obtain a spatially guided enhancement feature.
[0048] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0049] In a possible implementation, the processing module is configured to:
[0050] Separating the first context-enhanced feature to obtain a gated branch feature and a context-branch feature;
[0051] According to the gated branch features and contextual branch features, spatially guided enhanced features are generated.
[0052] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0053] In a possible implementation, the recognition module is used to locate the target in the remote sensing image and determine the type of the target based on the output features.
[0054] In a third aspect, an embodiment of the present application provides a computing device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the method provided in the first aspect or any possible implementation of the first aspect is implemented.
[0055] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 A schematic diagram of a process for detecting an object in a remote sensing image according to an embodiment of the present invention is shown;
[0057] Figure 2 A schematic diagram of the architecture of an HCMB model provided in an embodiment of the present application is shown;
[0058] Figure 3 A schematic structural diagram of a target detection device for remote sensing images provided in an embodiment of the present application is shown;
[0059] Figure 4 A schematic structural diagram of a computing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.
[0061] In the description of the embodiments of the present application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of the present application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.
[0062] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more. For example, "multiple systems" refers to two or more systems, and "multiple screen terminals" refers to two or more screen terminals.
[0063] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly identifying the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.
[0064] Remote sensing target detection is a fundamental task in remote sensing image analysis, widely used in fields such as military reconnaissance, environmental monitoring, disaster assessment, and urban planning. With the rapid development of Earth observation technology, the amount of satellite and aerial imagery acquired has increased exponentially. Therefore, it is crucial to quickly and accurately locate and classify various targets in complex remote sensing imagery. In recent years, deep learning-based methods have become the mainstream technical solution for remote sensing target detection due to their superior feature representation capabilities and good scalability. Representative methods include two-stage detectors (such as Faster R-CNN) and one-stage detectors (such as the YOLO series and RetinaNet), which have achieved excellent results in large-scale and medium-scale target detection.
[0065] However, existing methods still face significant challenges in remote sensing small target detection. Small targets are typically limited in size and occupy a very small area, making them susceptible to downsampling and feature loss. Furthermore, remote sensing images feature complex backgrounds and numerous interfering factors. Small targets often closely resemble their surroundings in color, texture, and shape, blurring the boundaries between foreground and background. This leads to unclear feature representation and difficulty in classifying objects, severely impacting detection accuracy.
[0066] Currently, context modeling methods based on single-scale or fixed convolutions use fixed-size convolution kernels (such as 3×3 or 5×5) to extract spatial context information and expand the receptive field through static stacking of convolution operations. However, these methods lack the adaptive selection and dynamic adjustment of spatial features at different scales, making it difficult to fully capture rich, layered context information. Furthermore, even when multi-scale receptive fields are introduced, static convolution kernels are generally used for fixed fusion, lacking the ability to adaptively adjust the importance of features at different scales. This results in insufficient recognition of small objects, especially those in complex backgrounds.
[0067] Moreover, current foreground enhancement methods based on spatial attention use simple spatial attention modules (such as SE and CBAM) to enhance high-response areas and suppress low-response areas. They mainly perform static screening based on feature intensity and lack the ability to model fine-grained foreground details in complex backgrounds and perform dynamic modulation.
[0068] In addition, some current attention mechanisms (such as channel attention and spatial attention) have been introduced to suppress background noise, but most of them only work on a single scale or fixed receptive field, failing to effectively combine the complementarity of multi-scale contextual features, and also lack specialized designs for enhancing target saliency.
[0069] Based on this, an embodiment of the present application provides a method and device for target detection in remote sensing images, wherein, in this scheme, by extracting multi-scale context features from the remote sensing image and adaptively modulating the context features of different scales, the complementarity of information at different scales can be fully utilized to obtain context enhancement features. In this way, by adjusting the spatial attention of the context enhancement features, a spatial guidance enhancement feature is obtained. In the spatial guidance enhancement feature, the spatial features of the foreground are stronger than the spatial features of the background. In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and the background in the feature space, effectively alleviating the problem of feature aliasing. Then, by performing residual connection on the remote sensing image, the spatial guidance enhancement feature and the first context enhancement feature, an output feature is obtained, and the target in the remote sensing image is identified based on the output feature. In this way, while the feature is enhanced, the basic information flow of the input feature is retained in the remote sensing image, avoiding information distortion caused by excessive feature enhancement. In this way, not only the stability of the training process is improved (easier to converge), but also the separability of the target and the complex background is improved, the target recognition accuracy is improved, and the accuracy and robustness of target recognition are improved.
[0070] Figure 1 This is a target detection method for remote sensing images provided by an embodiment of the present application. The target detection method for remote sensing images provided by an embodiment of the present application can be applied to devices or platforms with computing and storage functions, such as computing devices (such as servers, etc.) and computing device clusters (such as cloud computing platforms, etc.). Figure 1 As shown, the target detection method for remote sensing images provided in the embodiment of the present application includes the following steps S101 to S106.
[0071] S101, acquiring remote sensing images.
[0072] Satellites, aircraft, and other devices can generate remote sensing images, which can then be transmitted to devices or platforms equipped with computing and storage capabilities. Remote sensing images contain a wealth of information, such as buildings, vehicles, and people—also known as objects in these images. Applications such as military reconnaissance, environmental monitoring, disaster assessment, and urban planning require the integration of this information to achieve their objectives. Therefore, object detection in remote sensing images is essential.
[0073] S102, extracting multi-scale context features from the remote sensing image, where the context features are used to indicate associated information and environmental clues of the target in the remote sensing image at different dimensions.
[0074] Contextual features at multiple scales can be extracted from remote sensing images, wherein the multiple scales can be set according to one's own needs and are not specifically limited in the embodiments of this application.
[0075] Exemplarily, S102 is described using context features at three scales as an example. Multi-scale context features are extracted from the remote sensing image to obtain a first context feature, a second context feature, and a third context feature; wherein the first context feature, the second context feature, and the third context feature have different scales.
[0076] After acquiring the remote sensing image, context features are extracted from the remote sensing image based on multiple scales.
[0077] like Figure 2 As shown, the hierarchical context modulation (HCMB) model provided in the embodiment of the present application includes a context-driven adaptive modulation (CAM) module. The CAM module obtains three sets of spatial features of different scales, namely the first context feature, the second context feature and the third context feature, through three depth-separable convolutions with different receptive fields. s Extract fine-grained local spatial features, the second context feature X m Use dilated convolution with a dilation rate of 2 to capture the mid-scale context. The third context feature X l Further expand the receptive field and extract large-scale background information. Each convolution operation is a depthwise separable convolution, that is, each channel is convolved independently, which reduces the computational complexity and retains more fine-grained information.
[0078] Among them, the first context feature X s Satisfying the following formula (1), the second context feature X m Satisfying the following formula (2), the third context feature X l The following formula (3) is satisfied.
[0079] X s =DWConv 3×3 (X) (1)
[0080]
[0081] It should be noted that the multi-scale extraction part can use other receptive field combinations, such as 1×1 convolution expansion + dilated convolution combination, to achieve similar scale information capture effects. Figure 2 This is only a possible implementation example and should be based on actual needs.
[0082] In some embodiments, as Figure 2As shown, the HCMB model also includes a normalization (Norm) module. In order to improve the stability of features and accelerate training convergence, before S101, the remote sensing image can also be normalized by the normalization module (such as Batch Norm, Group Norm and other normalization methods). For example, the normalization module can perform layer normalization (LN) on the remote sensing image. Layer normalization acts on all channel features of each spatial position, standardizing their mean to 0 and variance to 1, and improving the consistency of feature distribution. The normalized remote sensing image is used as input information of the CAM module for multi-scale feature extraction.
[0083] S103 , performing context-driven adaptive modulation on the multi-scale context features to obtain a first context-enhanced feature of the remote sensing image.
[0084] Context-driven adaptive modulation is performed on multi-scale context features to obtain the first context-enhanced features of remote sensing images.
[0085] Specifically, in S103, first, the multi-scale context features are spliced to obtain aggregated features; based on the aggregated features, the weights of the multi-scale context features are generated; based on the weights of the multi-scale context features, the multi-scale context features are modulated respectively to obtain multi-scale context modulation features; the multi-scale context modulation features are spliced in pairs in the channel dimension to obtain the spatial modulation masks corresponding to the multi-scale context modulation features; based on the spatial modulation masks corresponding to the multi-scale context modulation features, the multi-scale context modulation features are dynamically weighted and fused to obtain multi-scale second context enhancement features; based on the multi-scale second context enhancement features, the first context enhancement features are generated.
[0086] Exemplarily, S103 is described by taking context features at three scales as an example: context-driven adaptive modulation is performed on the first context feature, the second context feature, and the third context feature to obtain a first context-enhanced feature of the remote sensing image.
[0087] Specifically, first, the first context feature, the second context feature, and the third context feature are concatenated to obtain an aggregate feature. A deep convolution is performed on the aggregate feature to obtain a first weight of the first context feature, a second weight of the second context feature, and a third weight of the third context feature. Figure 2 As shown in the figure, the CAM module performs multi-scale feature concatenation and attention weight generation on the first context feature, the second context feature, and the third context feature. The generator module of the attention mechanism concatenates the above three sets of scale features in the channel dimension (Concat operation) to obtain feature Xcat Feature X cat Satisfies the following formula (4):
[0088] X cat =Concat(X s ,X m ,X l ) (4)
[0089] like Figure 2 As shown, the concatenated feature X cat The input is fed into a lightweight convolution module, which includes a 3×3 depthwise convolution (DWConv), a GeLU activation function, and a Sigmoid normalization function. Finally, the importance weight vector corresponding to each scale feature is generated, and the weight vector satisfies the following formula (5):
[0090] [w0,w1,w2]=sigmoid(GeLU(DWConv(X cat ))) (5)
[0091] Where: w0 represents the first context feature X s The first weight of w1 represents the second context feature X m The second weight of w2 represents the third context feature X l The third weight.
[0092] Then, scale adaptive processing is performed on the first context feature, the second context feature, and the third context feature according to the first weight, the second weight, and the third weight to obtain the first feature, the second feature, and the third feature.
[0093] Among them, the CAM module performs the first context feature X s , the second context feature X m and the third context feature X l Importance weighting is performed. Each scale feature is multiplied by its corresponding importance weight to complete preliminary scale adaptation. Among them, the first context modulation feature, the second context modulation feature and the third context modulation feature satisfy the following formula (6):
[0094] X0=w0·X s ,X1=w1·X m ,X2=w2·X l (6)
[0095] Next, the first context modulation feature, the second context modulation feature, and the third context modulation feature are concatenated in pairs in the channel dimension to obtain a first spatial modulation mask, a second spatial modulation mask, and a third spatial modulation mask.
[0096] like Figure 2 As shown in Figure 1, the CAM module includes an Adaptive Scale-wise Modulation (ASM) module. The three weighted features obtained in the previous step are combined in pairs through the ASM module to perform fine-grained fusion of scale features. Specifically, for each pair of features (such as (X0, X1)), the following steps are performed to generate a spatial modulation mask. The two features X0 and X1 are concatenated in the channel dimension, then pass through a 7×7 depthwise convolution (DWConv), and then pass through a Sigmoid activation to generate a first spatial modulation mask M1. The first spatial modulation mask M1 satisfies the following formula (7):
[0097] M1=sigmoid(DWConv 7×7 (Concat(X0,X1))) (7)
[0098] Similarly, the second spatial modulation mask and the third spatial modulation mask satisfy the following formulas (8) and (9), respectively:
[0099] M2=sigmoid(DWConv 7×7 (Concat(X0,X2))) (8)
[0100] M3=sigmoid(DWConv 7×7 (Concat(X1,X2))) (9)
[0101] Again, the first context modulation feature, the second context modulation feature and the third context modulation feature are dynamically weighted and fused according to the first spatial modulation mask, the second spatial modulation mask and the third spatial modulation mask, respectively, to obtain the third context enhancement feature, the fourth context enhancement feature and the fifth context enhancement feature, that is, the multi-scale second context enhancement feature.
[0102] According to the first spatial modulation mask M1, the first context modulation feature and the second context modulation feature are dynamically weighted and fused to obtain the first context enhancement feature U0, wherein the third context enhancement feature U0 satisfies the following formula (10):
[0103] U0=X0+X1+M0·X0+(1-M0)·X1 (10)
[0104] Similarly, U1 and U2 are calculated for (X1, X2) and (X0, X2) respectively.
[0105] The fourth context enhancement feature U1 and the fifth context enhancement feature U2 satisfy the following formulas (11) and (12), respectively:
[0106] U1=X0+X2+M1·X0+(1-M1)·X2 (11)
[0107] U2=X1+X2+M2·X1+(1-M2)·X2 (12)
[0108] Finally, the first context enhancement feature is generated based on the third context enhancement feature, the fourth context enhancement feature, and the fifth context enhancement feature. This can reduce the amount of calculation and retain more fine-grained information.
[0109] Among them, the first context enhancement feature X CAM The following formula (13) is satisfied:
[0110] X CAM =U0+U1+U2 (13)
[0111] S104 , adjusting the spatial attention of the first context enhancement feature to obtain a spatial guidance enhancement feature, where the spatial guidance enhancement feature is used to indicate that the spatial features of the foreground of the remote sensing image are stronger than the spatial features of the background of the remote sensing image.
[0112] In some embodiments, the first contextual enhancement feature is processed through a spatially gated attention mechanism to generate a spatially guided enhancement feature. Specifically, the first contextual enhancement feature is separated to obtain a gated branch feature and a contextual branch feature; based on the gated branch feature and the contextual branch feature, a spatially guided enhancement feature is generated. This allows for dynamic screening of relevant regions of the target and suppresses irrelevant background activation, thereby increasing the separability between foreground and background in the feature space and effectively alleviating feature aliasing.
[0113] For example, Figure 2 As shown, first, the spatial-gating attention unit (SGAU) enhances the first context feature X CAM Separate. CAM Input a 1×1 convolution, the output is divided into two branches: gated branch feature X gate and context branch feature X content , both have the same spatial size and number of channels. The above process satisfies the following formula (14):
[0114] X gate ,X content =Conv 1×1 (X CAM ) (14)
[0115] Next, the SGAU module converts X gateInput a 7×7 depth convolution (DWConv) to obtain the spatial gating branch G, which is used to dynamically filter the target-related positions in the image. content Perform pixel-by-pixel multiplication with the gated branch G. Finally, perform channel adjustment through 1×1 convolution and output the spatially enhanced features, namely the spatially guided enhanced features X SGAU Among them, the spatial guidance enhancement feature X SGAU The following formula (15) is satisfied:
[0116] X SGAU =Conv 1×1 (X content DWConv 7×7 (X gate )) (15)
[0117] Here, the spatial gating unit can be replaced by other spatial attention mechanisms (such as coordinate attention, local adaptive gating module), but the core idea is to dynamically adjust the spatial attention to highlight the foreground area. The specific spatial attention mechanism used in this application embodiment is not specifically limited.
[0118] S105 , performing double residual connection processing on the spatial guidance enhancement feature, the remote sensing image and the first context enhancement feature to obtain an output feature.
[0119] For example, Figure 2 As shown, the input feature X (the original feature of the remote sensing image) and the CAM module output the first context enhanced feature X CAM , SGAU module outputs spatial guidance enhancement feature X SGAU Perform double residual connection to obtain the output feature Y. The output feature Y satisfies the following formula (16):
[0120] Y=X SGAU +X CAM +X (16)
[0121] Among them, the first context enhancement feature X CAM Maintain the basic feature information of context modeling. The original feature X of the remote sensing image retains the original input feature, enhancing training stability and feature consistency.
[0122] S106: Identify the target in the remote sensing image according to the output features.
[0123] Based on the output features, the target in the remote sensing image is located and the type of the target is determined.
[0124] According to the above scheme, by extracting multi-scale contextual features from remote sensing images and adaptively modulating the contextual features at different scales, the complementarity of information at different scales can be fully utilized to obtain context-enhanced features. In this way, by adjusting the spatial attention of the first context-enhanced feature, a spatially guided enhancement feature is obtained. In the spatially guided enhancement feature, the spatial features of the foreground are stronger than those of the background. This allows for dynamic screening of target-related regions and suppresses activation of irrelevant background, thereby increasing the separability between foreground and background in feature space and effectively alleviating feature aliasing. Next, by performing a residual connection on the remote sensing image, the spatially guided enhancement feature, and the first context-enhanced feature, an output feature is obtained, and targets in the remote sensing image are identified based on the output feature. This enhances the features while preserving the underlying information flow of the input features in the remote sensing image, avoiding information distortion caused by excessive feature enhancement. This not only improves the stability of the training process (making it easier to converge), but also enhances the separability of the target from the complex background, improving target recognition accuracy, and enhancing both the accuracy and robustness of target recognition.
[0125] Based on the same concept as the embodiment of the method of the present application, the embodiment of the present application also provides a target detection device for remote sensing images. The target detection device for remote sensing images includes several modules, each of which is used to perform each step of the target detection method for remote sensing images provided in the embodiment of the present application. The division of modules is not limited here. Those skilled in the art will clearly understand that in actual applications, the various steps of the target detection method for remote sensing images provided in the embodiment of the present application can be assigned to different modules as needed, that is, the internal structure of the device can be divided into different modules to complete all or part of the functions described above. The modules in the embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more modules can be integrated into a single unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional units. In addition, the specific names of the modules are only for the purpose of distinguishing them from each other and are not intended to limit the scope of protection of this application. The specific working processes of the modules in the above-mentioned device can refer to the corresponding processes in the aforementioned method embodiment and will not be repeated here.
[0126] Figure 3 This is a target detection device for remote sensing images provided by an embodiment of the present application. Figure 3 As shown, the target detection device for remote sensing images provided by the embodiment of the present application includes:
[0127] An acquisition module 301 is used to acquire remote sensing images;
[0128] An extraction module 302 is configured to extract multi-scale context features from the remote sensing image to obtain a first context feature, a second context feature, and a third context feature; wherein the first context feature, the second context feature, and the third context feature have different scales;
[0129] A modulation module 303 is configured to perform context-driven adaptive modulation on the first context feature, the second context feature, and the third context feature to obtain a first context-enhanced feature of the remote sensing image;
[0130] An adjustment module 304 is configured to perform a spatially guided feature enhancement operation on the first context enhancement feature to generate a spatially guided enhancement feature, where the spatially guided enhancement feature is used to indicate that a spatial feature of a foreground of the remote sensing image is stronger than a spatial feature of a background of the remote sensing image;
[0131] A processing module 305 is configured to perform a double residual connection process on the spatial guidance enhancement feature, the remote sensing image, and the first context enhancement feature to obtain an output feature;
[0132] The recognition module 306 is used to recognize targets in the remote sensing image based on the output features.
[0133] According to the above scheme, multi-scale contextual features are extracted from remote sensing images and adaptively modulated at different scales, thereby fully leveraging the complementarity of information at different scales to obtain a first contextual enhancement feature. Similarly, by adjusting the spatial attention of the first contextual enhancement feature, a spatial guidance enhancement feature is obtained. In the spatial guidance enhancement feature, the spatial features of the foreground are stronger than those of the background. This allows for dynamic selection of target-related regions and suppresses activation of irrelevant background, thereby increasing the separability between foreground and background in feature space and effectively alleviating feature aliasing. Next, a residual connection is performed on the remote sensing image, the spatial guidance enhancement feature, and the first contextual enhancement feature to obtain an output feature, which is then used to identify targets in the remote sensing image. This enhances the features while preserving the underlying information flow of the input features in the remote sensing image, avoiding information distortion caused by excessive feature enhancement. This not only improves the stability of the training process (making it easier to converge), but also enhances the separability of the target from the complex background, improving target recognition accuracy, and enhancing both the accuracy and robustness of target recognition.
[0134] In a possible implementation, the modulation module is configured to:
[0135] The multi-scale context features are spliced together to obtain aggregated features;
[0136] Based on the aggregated features, the weights of the multi-scale contextual features are generated;
[0137] According to the respective weights of the multi-scale context features, the multi-scale context features are modulated respectively to obtain the multi-scale context modulation features;
[0138] The multi-scale context modulation features are spliced in pairs in the channel dimension to obtain the spatial modulation masks corresponding to the multi-scale context modulation features;
[0139] According to the spatial modulation masks corresponding to the multi-scale context modulation features, the multi-scale context modulation features are dynamically weighted and fused to obtain the multi-scale second context enhancement features;
[0140] A first context enhancement feature is generated according to the multi-scale second context enhancement feature.
[0141] In this way, the amount of computation can be reduced and more fine-grained information can be retained.
[0142] In one possible implementation, the adjustment module is configured to process the first context enhancement feature through a spatial gated attention mechanism to obtain a spatially guided enhancement feature.
[0143] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0144] In a possible implementation, the processing module is configured to:
[0145] Separating the first context-enhanced feature to obtain a gated branch feature and a context-branch feature;
[0146] According to the gated branch features and contextual branch features, spatially guided enhanced features are generated.
[0147] In this way, the relevant areas of the target can be dynamically screened and the activation of irrelevant background can be suppressed, thereby widening the separability between the foreground and background in the feature space and effectively alleviating the problem of feature aliasing.
[0148] In a possible implementation, the recognition module is used to locate the target in the remote sensing image and determine the type of the target based on the output features.
[0149] The following introduces a computing device provided by an embodiment of the present application.
[0150] Figure 4 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. Figure 4As shown, the computing device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computing device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display computing device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computing devices can be connected, and each computing device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 401 is taken as an example.
[0151] Memory 402 is a non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the method provided in this application.
[0152] Memory 402, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present application. Processor 401 executes the non-transitory software programs, instructions, and modules stored in memory 402 to execute various functional applications and data processing of the server, thereby implementing the methods in the above-mentioned method embodiments.
[0153] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computing device, etc. In addition, the memory 402 may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located relative to the processor 401, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0154] The computing device may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.
[0155] Input device 403 can receive input digital or character information and generate key signal input related to user settings and function control of the computing device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, a pointer, one or more mouse buttons, a trackball, a joystick, etc. Output device 404 can include a display computing device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display computing device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display computing device can be a touch screen.
[0156] In addition, in combination with the above embodiments, the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the methods in the above embodiments is implemented.
[0157] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0158] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0159] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.
[0160] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.
Claims
1. A target detection method for remote sensing images, characterized in that: The method comprises: Acquisition of remote sensing images; Extracting multi-scale context features from the remote sensing image, wherein the context features are used to indicate association information and environmental clues of the target in the remote sensing image at different dimensions; performing context-driven adaptive modulation on the multi-scale context features to obtain a first context-enhanced feature of the remote sensing image; performing a spatially guided feature enhancement operation on the first contextual enhancement feature to generate a spatially guided enhancement feature, wherein the spatially guided enhancement feature is used to indicate that a spatial feature of a foreground of the remote sensing image is stronger than a spatial feature of a background of the remote sensing image; performing double residual connection processing on the spatial guidance enhancement feature, the remote sensing image, and the first context enhancement feature to obtain an output feature; The target in the remote sensing image is located and identified according to the output features.
2. The method according to claim 1, characterized in that The performing context-driven adaptive modulation on the multi-scale context features to obtain a first context-enhanced feature of the remote sensing image includes: Performing splicing processing on the multi-scale context features to obtain aggregated features; Generating weights of the multi-scale context features according to the aggregated features; Modulating the multi-scale context features according to their respective weights to obtain the multi-scale context modulation features; splicing the multi-scale context modulation features in pairs in the channel dimension to obtain spatial modulation masks corresponding to the multi-scale context modulation features; Performing dynamic weighted fusion on the multi-scale context modulation features according to the spatial modulation masks corresponding to the multi-scale context modulation features, respectively, to obtain the multi-scale second context enhancement features; The first context enhancement feature is generated according to the multi-scale second context enhancement feature.
3. The method according to claim 1 or 2, characterized in that The performing a spatially guided feature enhancement operation on the first contextual enhancement feature to generate a spatially guided enhancement feature includes: The first context enhancement feature is processed through a spatial gated attention mechanism to obtain a spatial guided enhancement feature.
4. The method according to any one of claims 1 to 3, characterized in that The processing of the first context enhancement feature by a spatial gated attention mechanism to obtain a spatially guided enhancement feature includes: Separating the first context-enhanced feature to obtain a gated branch feature and a context-branch feature; A spatially guided enhanced feature is generated according to the gated branch feature and the contextual branch feature.
5. The method according to any one of claims 1 to 4, characterized in that Positioning and identifying the target in the remote sensing image according to the output features includes: The target in the remote sensing image is located according to the output features, and the type of the target is determined.
6. A target detection device for remote sensing images, characterized in that: The device comprises: Acquisition module, used to acquire remote sensing images; An extraction module is used to extract multi-scale context features from the remote sensing image, wherein the context features are used to indicate association information and environmental clues of the target in the remote sensing image at different dimensions; a modulation module, configured to perform context-driven adaptive modulation on the multi-scale context features to obtain a first context-enhanced feature of the remote sensing image; an adjustment module, configured to perform a spatially guided feature enhancement operation on the first contextual enhancement feature to generate a spatially guided enhancement feature, wherein the spatially guided enhancement feature is used to indicate that a spatial feature of a foreground of the remote sensing image is stronger than a spatial feature of a background of the remote sensing image; a processing module, configured to perform double residual connection processing on the spatial guidance enhancement feature, the remote sensing image, and the first context enhancement feature to obtain an output feature; The recognition module is used to locate and recognize the target in the remote sensing image according to the output features.
7. The device according to claim 5, characterized in that The modulation module is used to: Performing splicing processing on the multi-scale context features to obtain aggregated features; Generating weights of the multi-scale context features according to the aggregated features; Modulating the multi-scale context features according to their respective weights to obtain the multi-scale context modulation features; splicing the multi-scale context modulation features in pairs in the channel dimension to obtain spatial modulation masks corresponding to the multi-scale context modulation features; Performing dynamic weighted fusion on the multi-scale context modulation features according to the spatial modulation masks corresponding to the multi-scale context modulation features, respectively, to obtain the multi-scale second context enhancement features; The first context enhancement feature is generated according to the multi-scale second context enhancement feature.
8. The device according to claim 5 or 6, characterized in that The adjustment module is used to process the first context enhancement feature through a spatial gated attention mechanism to obtain a spatially guided enhancement feature.
9. The device according to any one of claims 5 to 8, characterized in that: The processing module is used for: Separating the first context-enhanced feature to obtain a gated branch feature and a context-branch feature; A spatially guided enhanced feature is generated according to the gated branch feature and the contextual branch feature.
10. The device according to any one of claims 5 to 9, characterized in that: The recognition module is configured to locate the first region in the remote sensing image and determine the type of the first region based on the output features.