Advanced scale perception attention enhancement method for remote sensing image small target detection
By introducing advanced scale perceived attention enhancement methods in remote sensing image object detection, using branch attention extraction networks and enhancement networks, the problem of performance differences in traditional methods in micro-object detection is solved, and high-precision small object detection is achieved.
Patent Information
- Application Number
- CN202411469546.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-02-11
AI Technical Summary
The existing remote sensing image object detection methods have performance differences when detecting small targets, and the traditional CNN method is limited by locality and weight sharing characteristics, making it difficult to effectively capture different scales and complex details of small targets.
It proposes an advanced scale perceived attention enhancement method for small object detection of remote sensing images. Through branch attention extraction networks and enhancement networks, use space and channel attention in different receptive fields to improve feature representation, and use advanced scale perceived attention enhancement method. Refine feature representation.
It significantly improves the accuracy of small object detection, can effectively handle features of different scales and directions, reduces calculation overhead, and improves model efficiency.
Smart Images
Figure CN119418205A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to an advanced scale-aware attention enhancement method for small target detection in remote sensing images. Background Art
[0002] Remote sensing image target detection focuses on accurate target classification and positioning, and has important application significance in military reconnaissance, maritime rescue, environmental monitoring, and urban planning. Thanks to the rapid progress of deep learning, remote sensing image target detection has achieved remarkable results in the past decade. However, typical methods have huge performance differences when detecting tiny targets and normal-sized targets, mainly because of the lack of feature representation of small targets and the complex background of remote sensing scenes, which poses new challenges to existing methods.
[0003] At present, deep learning algorithms have been widely used in remote sensing target detection tasks due to their powerful feature representation capabilities. Researchers have proposed a variety of solutions to address the challenges of small target detection, such as multi-scale fusion, attention mechanism, and super-resolution assistance. However, the performance of traditional CNN-based target detection algorithms is limited by the intrinsic locality and weight sharing characteristics of CNN. The locality of CNN weakens the ability of feature extractors to capture long-range dependencies. In addition, traditional CNN-based detection methods require manual creation of anchor boxes or prediction proposals, as well as complex post-processing techniques such as non-maximum suppression, which incurs additional computation and delay, significantly affecting the performance of the detector.
[0004] In contrast, the Transformer-based detector DETR treats the object detection task as a direct set prediction problem. Variants of DETR have quickly gained attention due to their outstanding performance in object detection tasks. On the one hand, Transformer has a strong ability to extract long-range features, which can effectively capture the unique shapes and textures in remote sensing images. On the other hand, the end-to-end pipeline design of Transformer eliminates the need for manual design and reduces the reliance on post-processing, thereby improving model efficiency. Despite the numerous advantages of DETR variants, researchers still encounter challenges in tiny objects. Limited information about tiny objects may be masked during the self-attention operation, causing the features of tiny objects to be contaminated by the background, i.e., foreground-background confusion.
[0005] Traditional methods improve detection performance by improving the distinguishability of features through channel or spatial attention mechanisms. However, these traditional methods usually rely on a single-scale representation, which limits their ability to capture different scales and complex details of small objects, especially in complex backgrounds. This limitation hinders their effectiveness because single-scale attention cannot fully distinguish small objects from their surroundings. Therefore, in order to obtain high-precision detection models for small objects in remote sensing images, more advanced methods are urgently needed that can effectively utilize multi-scale information to enhance feature recognition without incurring a large amount of computational overhead. Summary of the invention
[0006] Based on this, it is necessary to provide an advanced scale-aware attention enhancement method for small target detection in remote sensing images to address the above technical issues.
[0007] An advanced scale-aware attention enhancement method for small target detection in remote sensing images includes the following steps:
[0008] Obtaining input features, and classifying the input features according to preset standards to obtain a number of sub-features;
[0009] Obtaining a preset tensor shape, and dividing the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups;
[0010] Obtain a branch attention extraction network, and use the sub-feature group as input through the branch attention extraction network to obtain a spatial attention map;
[0011] An enhanced network is obtained, and the spatial attention map is used as input through the enhanced network to obtain an enhanced target.
[0012] In one embodiment, input features are obtained, and the input features are classified according to preset standards to obtain several sub-features including:
[0013] Acquire an input remote sensing image, and input the remote sensing image into a feature extraction generator to obtain input features;
[0014] The input features are grouped under the channel dimension to obtain several sub-features.
[0015] In one embodiment, it also includes:
[0016] The feature extraction generator is a ResNet50 backbone feature extractor.
[0017] In one embodiment, it also includes:
[0018] The preset tensor shape is as follows:
[0019] (C / / G,H,W)
[0020] Among them, C / / G represents the number of color channels, H represents the height of the image, and W represents the width of the image.
[0021] In one embodiment, the branch attention extraction network includes: a first branch structure, a second branch structure, and a third branch structure;
[0022] The first branch structure includes a first subpath and a second subpath, the first subpath applies a multi-directional attention gate in the (H, C) direction, and the second subpath applies a multi-directional attention gate in the (W, C) direction; the features of the first subpath and the second subpath are connected through the attention gate, and then the output is decomposed into two vectors, and two sigmoid functions are used on the linear convolution for fitting, so as to obtain a first output spatial attention map;
[0023] The second branch structure processes the sub-feature group using a 3x3 convolution kernel to obtain a second output spatial attention map;
[0024] The third branch structure processes the sub-feature group using a 5x5 convolution kernel to obtain a third output spatial attention map;
[0025] The output spatial attention map includes the first output spatial attention map, the second output spatial attention map and the third output spatial attention map.
[0026] In one embodiment, obtaining an enhanced network, taking the spatial attention map as input and passing it through the enhanced network, before obtaining an enhanced target, further includes:
[0027] The output spatial attention map is feature enhanced by high-level scale-aware attention enhancement to obtain an enhanced spatial attention map:
[0028] F3,F4,F5=ASA(s3,s4,s5)
[0029] Among them, the ASA() represents the advanced scale-aware attention enhancement method, s3 represents the first output spatial attention map, s4 represents the second output spatial attention map, s5 represents the third output spatial attention map, F3 represents the enhanced first output spatial attention map, F4 represents the enhanced second output spatial attention map, and F5 represents the enhanced third output spatial attention map.
[0030] In one embodiment, obtaining an enhancement network, taking the spatial attention map as input and passing it through the enhancement network to obtain an enhancement target comprises:
[0031] Taking the spatial attention map as input, combining the first output spatial attention map, the second output spatial attention map, and the third output spatial attention map by element-wise multiplication;
[0032] The aggregated features of the first output spatial attention map, the second output spatial attention map, and the third output spatial attention map are combined in pairs, and an output result is obtained by applying a Sigmoid function;
[0033] The output result is multiplied by the sub-feature to obtain an enhanced target.
[0034] The advanced scale-aware attention enhancement system for small target detection in remote sensing images is used to implement the advanced scale-aware attention enhancement method for small target detection in remote sensing images as described above, including:
[0035] A classification module is used to obtain input features and classify the input features according to preset standards to obtain a number of sub-features;
[0036] A grouping module, used for obtaining a preset tensor shape, and dividing the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups;
[0037] An extraction module, used to obtain a branch attention extraction network, and obtain a spatial attention map by taking the sub-feature group as input through the branch attention extraction network;
[0038] The enhancement module is used to obtain an enhancement network, and obtain an enhancement target by taking the spatial attention map as input through the enhancement network.
[0039] A device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the high-level scale-aware attention enhancement method for small target detection in remote sensing images described in the above-mentioned embodiments are implemented.
[0040] A storage medium stores a computer program, which, when executed by a processor, implements the steps of the advanced scale-aware attention enhancement method for small target detection in remote sensing images described in the above-mentioned embodiments.
[0041] Compared with the prior art, the advantages and beneficial effects of the present invention are: the present invention can propose an advanced scale-aware attention enhancement method, which improves feature representation by utilizing spatial and channel attention of different receptive fields, and effectively detects small targets in remote sensing images. Through a large number of experimental verifications, the method of the present invention has shown significant improvements in detection performance, demonstrating the potential to meet the challenges of remote sensing small target detection. The present invention minimizes computational overhead while retaining basic channel information by reorganizing channels into batch dimensions and subdividing them into multiple sub-features. Multiple branches are designed to interact collaboratively across dimensions to accurately capture pixel-level relationships. This innovative method can effectively process features of different scales and directions, thereby improving the detection accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A schematic diagram of a process flow of an advanced scale-aware attention enhancement method for small target detection in remote sensing images in one embodiment;
[0043] Figure 2 A diagram of an advanced scale-aware attention-enhanced overall detection framework for small target detection in remote sensing images in one embodiment;
[0044] Figure 3 A schematic diagram of an advanced scale-aware attention enhancement network structure for small target detection in remote sensing images in one embodiment;
[0045] Figure 4 Schematic diagram of the structure of an advanced scale-aware attention enhancement system for small target detection in remote sensing images in one embodiment;
[0046] Figure 5 Schematic diagram of the internal structure of a device in one embodiment. DETAILED DESCRIPTION
[0047] Before describing the specific embodiments of the present invention, the overall concept of the present invention is described as follows:
[0048] The present invention is mainly developed from the remote sensing image processing process. Currently, small target detection has the problems of difficult feature extraction and easy confusion between foreground and background.
[0049] Therefore, the present invention proposes an advanced scale-aware attention enhancement method for small target detection in remote sensing images, in which a parallel processing strategy is adopted. By reorganizing the channels into batch dimensions and subdividing them into multiple sub-features, the computational overhead is minimized while retaining the basic channel information. Multiple branches are designed to interact collaboratively across dimensions to accurately capture pixel-level relationships. This innovative method can effectively process features of different scales and directions, thereby improving the detection accuracy of small targets.
[0050] After introducing the overall concept of the present invention, in order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific implementation methods in conjunction with the accompanying drawings.
[0051] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of this specification should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of this specification do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0052] In one embodiment, Figure 1 As shown in Figure 1, a flow chart of an advanced scale-aware attention enhancement method for small target detection in remote sensing images is provided. Figure 2 As shown in Figure 1, an advanced scale-aware attention-enhanced overall detection framework for small target detection in remote sensing images is provided. Figure 3 As shown, a schematic diagram of an advanced scale-aware attention enhancement network structure for small target detection in remote sensing images is provided. The advanced scale-aware attention enhancement method for small target detection in remote sensing images includes the following steps:
[0053] Step S101, obtaining input features, and classifying the input features according to preset standards to obtain a plurality of sub-features.
[0054] Specifically, first, the input feature X∈R C×H×W Group X in the channel dimension and divide it into G features as follows:
[0055] X=[X0,X i ,...X G-1 ]
[0056] Get a smaller sub-feature map X i ∈R C / / G×H×W .
[0057] On this basis, the input features are obtained, and the input features are classified according to preset standards to obtain several sub-features including:
[0058] Acquire an input remote sensing image, and input the remote sensing image into a feature extraction generator to obtain input features;
[0059] The input features are grouped under the channel dimension to obtain several sub-features.
[0060] On this basis, it also includes:
[0061] The feature extraction generator is a ResNet50 backbone feature extractor.
[0062] Specifically, we obtain an input remote sensing image, and generate a multi-level feature map through the ResNet50 backbone feature extractor, where the backbone network ResNet50 has been pre-trained on ImageNet to ensure robust initialization. The generated features are described as follows:
[0063] s3,s4,s5=B(input)
[0064] Among them, input represents the input remote sensing image, B represents the ResNet50 backbone feature extractor, and s3, s4 and s5 are the extracted third-level features.
[0065] On this basis, a branch attention extraction network is obtained, and the sub-feature group is taken as input through the branch attention extraction network to obtain a spatial attention map, and the following steps are also included before the spatial attention map is obtained:
[0066] The sub-feature group is enhanced by high-level scale-aware attention enhancement to obtain an enhanced sub-feature group:
[0067] F3,F4,F5=ASA(s3,s4,s5)
[0068] Among them, the ASA() represents the advanced scale-aware attention enhancement method, s3 represents the first sub-feature group, s4 represents the second sub-feature group, s5 represents the third sub-feature group, F3 represents the enhanced first sub-feature group, F4 represents the enhanced second sub-feature group, and F5 represents the enhanced third sub-feature group.
[0069] Specifically, the advanced scale-aware attention enhancement method is used to refine and enhance the three-level features extracted by the backbone network to obtain F3, F4, and F5:
[0070] F3,F4,F5=ASA(s3,s4,s5)
[0071] ASA stands for Advanced Scale-Aware Attention Augmentation.
[0072] Step S102, obtaining a preset tensor shape, and dividing the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups.
[0073] Specifically, for each feature X∈R extracted by the backbone network extractor C×H×W Group X in the channel dimension and divide it into G features as follows:
[0074] X=[X0,X i ,...X G-1 ],
[0075] Where (X i ∈R C / / G×H×W ).
[0076] On this basis, it also includes:
[0077] The preset tensor shape is obtained as follows:
[0078] (C / / G,H,W)
[0079] Among them, C / / G represents the number of color channels, H represents the height of the image, and W represents the width of the image.
[0080] Specifically, reshape G to the batch size and redefine the tensor shape to (C / / G,H,W).
[0081] Step S103, obtaining a branch attention extraction network, taking the sub-feature group as input through the branch attention extraction network, and obtaining a spatial attention map.
[0082] Specifically, the grouped sub-feature X i Enter a three-branch structure to extract attention weights and obtain a spatial attention map.
[0083] On this basis, the branch attention extraction network includes: a first branch structure, a second branch structure and a third branch structure;
[0084] The first branch structure includes a first subpath and a second subpath, the first subpath applies a multi-directional attention gate in the (H, C) direction, and the second subpath applies a multi-directional attention gate in the (W, C) direction; the features of the first subpath and the second subpath are connected through the attention gate, and then the output is decomposed into two vectors, and two sigmoid functions are used on the linear convolution for fitting, so as to obtain a first output spatial attention map;
[0085] The second branch structure processes the sub-feature group using a 3x3 convolution kernel to obtain a second output spatial attention map;
[0086] The third branch structure processes the sub-feature group using a 5x5 convolution kernel to obtain a third output spatial attention map;
[0087] The output spatial attention map includes the first output spatial attention map, the second output spatial attention map and the third output spatial attention map.
[0088] Specifically, the branch attention extraction network includes: a first branch structure, a second branch structure and a third branch structure.
[0089] The first branch structure contains two subpaths, the first subpath and the second subpath, which apply multi-directional attention gates in the (H, C) and (W, C) directions respectively, focusing on encoding weights through channel and directional scale relationships.
[0090] The features from the two branches are connected through the attention gate, and then the output is decomposed into two vectors, and two sigmoid functions are used to fit on the linear convolution to obtain the encoded features. Specifically, the attention gate calculates the average and maximum values of the input features respectively, and concatenates them on the dimension of feature dimension 1, then obtains the vector through convolution, batch normalization and rectified linear unit ReLU, and finally obtains the first output spatial attention map through the activation function sigmoid, which is as follows:
[0091] att_gate=Sig(ConvBNReLu(cat(mean(x),max(x),dim=1)))
[0092] In order to realize different cross-channel interaction features between the two parallel paths in the first branch structure, the two sub-paths are concatenated and multiplied with the original sub-features to optimize the feature representation.
[0093] The processed features are group-normalized via group-channel normalization to stabilize the training. This helps to learn effectively across sub-feature maps and reduce the impact of different grouping scales.
[0094] The second branch structure uses a 3x3 convolution kernel to process the sub-feature group and obtains the second output spatial attention map to capture local detail features.
[0095] The third branch structure uses a 5x5 convolution kernel to process the sub-feature group to obtain a third output spatial attention map, using a larger 5x5 convolution kernel to capture larger spatial information and have a larger receptive field of view.
[0096] Step S104, obtaining an enhanced network, taking the spatial attention map as input through the enhanced network, and obtaining an enhanced target.
[0097] On this basis, an enhanced network is obtained, and the spatial attention map is used as input through the enhanced network to obtain enhanced targets including:
[0098] Taking the spatial attention map as input, the attention is weighted by two-dimensional global average pooling and SoftMax to generate attention weights;
[0099] The first output spatial attention map, the second output spatial attention map, and the third output spatial attention map are combined by element-wise multiplication;
[0100] The aggregated features of the first output spatial attention map, the second output spatial attention map, and the third output spatial attention map are combined in pairs, and an output result is obtained by applying a Sigmoid function;
[0101] The output result is multiplied by the sub-feature to obtain an enhanced target.
[0102] Specifically, the spatial attention maps output by the first branch structure, the second branch structure, and the third branch structure are used as input. Attention weighting is performed by two-dimensional global average pooling and SoftMax respectively to generate attention weights to encode cross-channel interaction spatial information. The outputs of the three branches are combined by element-wise multiplication. Each branch contributes a spatial attention map, which is used to adjust the corresponding feature representations of different spatial resolutions. The aggregated features of the three branches are combined in pairs, and the final output is obtained by applying the Sigmoid function, which is used as a gating mechanism to refine the feature representation across channels and spatial dimensions. Finally, it is combined with the input feature X i Multiply to get the result F5 feature output.
[0103] The refined F5 feature is input into the Transformer encoder for position encoding and self-attention mechanism processing to obtain the A5 feature. A5 is input into the cross-scale feature fusion module together with F4 and F3 for feature fusion to obtain the enhanced target P out .
[0104] The IOU-aware query selection mechanism is used to select a fixed number of image features as the initial target query of the decoder, which is compared with the feature map P output by the encoder. out Interact to output a prediction containing the target category and the corresponding bounding box coordinates. A linear layer is used to map the decoder output to the category prediction and bounding box regression.
[0105] The predicted bounding boxes are matched with the true labels, the Hungarian algorithm is used to achieve the best match, the loss is calculated, and the model parameters are optimized through back-propagation to obtain the final model.
[0106] Taking the actual application scenario of remote sensing small target detection as an example, a variety of small target detection methods are compared on a typical public remote sensing small target dataset. As shown in Table 1, it can be seen that the method proposed in the embodiment has better detection accuracy.
[0107] Table 1 Comparison of detection accuracy of various existing small target detection methods and this patent on the AITOD dataset
[0108]
[0109] The present invention proposes an advanced scale-aware attention enhancement method, which improves feature representation by utilizing spatial and channel attention of different receptive fields, and effectively detects small targets in remote sensing images. Through a large number of experimental verifications, the method of the present invention shows significant improvement in detection performance, demonstrating the potential to meet the challenges of remote sensing small target detection. The present invention minimizes computational overhead while retaining basic channel information by reorganizing channels into batch dimensions and subdividing them into multiple sub-features. Multiple branches are designed to interact collaboratively across dimensions to accurately capture pixel-level relationships. This innovative method can effectively process features of different scales and directions, thereby improving the detection accuracy of small targets.
[0110] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.
[0111] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0112] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present invention also provides an advanced scale-aware attention enhancement system for small target detection in remote sensing images.
[0113] refer to Figure 4 The advanced scale-aware attention enhancement system for small target detection in remote sensing images includes:
[0114] The classification module 401 is used to obtain input features and classify the input features according to preset standards to obtain a plurality of sub-features;
[0115] A grouping module 402 is used to obtain a preset tensor shape, and to group the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups;
[0116] An extraction module 403 is used to obtain a branch attention extraction network, and obtain a spatial attention map by taking the sub-feature group as input through the branch attention extraction network;
[0117] The enhancement module 404 is used to obtain an enhancement network, and obtain an enhancement target by taking the spatial attention map as input through the enhancement network.
[0118] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0119] The system of the above-mentioned embodiment is used to implement the corresponding advanced scale-aware attention enhancement method for small target detection in remote sensing images in any of the aforementioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0120] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the high-level scale-aware attention enhancement method for small target detection in remote sensing images as described in any of the above embodiments is implemented.
[0121] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0122] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0123] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0124] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0125] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0126] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0127] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0128] The electronic device of the above embodiment is used to implement the corresponding advanced scale-aware attention enhancement method for small target detection in remote sensing images in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0129] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present invention also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the advanced scale-aware attention enhancement method for small target detection in remote sensing images as described in any of the above embodiments.
[0130] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0131] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the advanced scale-aware attention enhancement method for small target detection in remote sensing images as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0132] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0133] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0134] Where specific details have been set forth to describe exemplary embodiments of the present invention, it will be apparent to those skilled in the art that embodiments of the present invention may be implemented without these specific details or with variations thereto. Therefore, these descriptions should be considered illustrative rather than restrictive. Although the present invention has been described in conjunction with specific embodiments of the present invention, many replacements, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description.
[0135] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.
Claims
1. An advanced scale-aware attention enhancement method for small target detection in remote sensing images, characterized in that: include: Obtaining input features, and classifying the input features according to preset standards to obtain a number of sub-features; Obtaining a preset tensor shape, and dividing the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups; Obtain a branch attention extraction network, and use the sub-feature group as input through the branch attention extraction network to obtain a spatial attention map; An enhanced network is obtained, and the spatial attention map is used as input through the enhanced network to obtain an enhanced target.
2. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 1, characterized in that: The input features are obtained, and the input features are classified according to preset standards to obtain several sub-features including: Acquire an input remote sensing image, and input the remote sensing image into a feature extraction generator to obtain input features; The input features are grouped under the channel dimension to obtain several sub-features.
3. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 2, characterized in that: Also includes: The feature extraction generator is a ResNet50 backbone feature extractor.
4. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 1, characterized in that: Also includes: The preset tensor shape is as follows: (C / / G,H,W) Among them, C / / G represents the number of color channels, H represents the height of the image, and W represents the width of the image.
5. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 1, characterized in that: The step of obtaining a branch attention extraction network and taking the sub-feature group as input and passing the branch attention extraction network to obtain a spatial attention map further includes: The sub-feature group is enhanced by high-level scale-aware attention enhancement to obtain an enhanced sub-feature group: F3,F4,F5=ASA(s3,s4,s5) Among them, the ASA() represents the advanced scale-aware attention enhancement method, s3 represents the first sub-feature group, s4 represents the second sub-feature group, s5 represents the third sub-feature group, F3 represents the enhanced first sub-feature group, F4 represents the enhanced second sub-feature group, and F5 represents the enhanced third sub-feature group.
6. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 1, characterized in that: The branch attention extraction network includes: a first branch structure, a second branch structure and a third branch structure; The first branch structure includes a first subpath and a second subpath, the first subpath applies a multi-directional attention gate in the (H, C) direction, and the second subpath applies a multi-directional attention gate in the (W, C) direction; the features of the first subpath and the second subpath are connected through the attention gate, and then the output is decomposed into two vectors, and two sigmoid functions are used on the linear convolution for fitting, so as to obtain a first output spatial attention map; The second branch structure processes the sub-feature group using a 3x3 convolution kernel to obtain a second output spatial attention map; The third branch structure processes the sub-feature group using a 5x5 convolution kernel to obtain a third output spatial attention map; The output spatial attention map includes the first output spatial attention map, the second output spatial attention map and the third output spatial attention map.
7. The high-level scale-aware attention enhancement method for small target detection in remote sensing images according to claim 1, characterized in that: The obtaining of the enhancement network, taking the spatial attention map as input through the enhancement network, and obtaining the enhancement target comprises: Taking the spatial attention map as input, combining the first output spatial attention map, the second output spatial attention map, and the third output spatial attention map by element-wise multiplication; The aggregated features of the first output spatial attention map, the second output spatial attention map, and the third output spatial attention map are combined in pairs, and an output result is obtained by applying a Sigmoid function; The output result is multiplied by the sub-feature to obtain an enhanced target.
8. An advanced scale-aware attention enhancement system for small target detection in remote sensing images, characterized by: The method for implementing the advanced scale-aware attention enhancement method for small target detection in remote sensing images as claimed in any one of claims 1 to 7 comprises: A classification module is used to obtain input features and classify the input features according to preset standards to obtain a number of sub-features; A grouping module, used for obtaining a preset tensor shape, and dividing the plurality of sub-features into a plurality of groups according to the preset tensor shape to obtain sub-feature groups; An extraction module, used to obtain a branch attention extraction network, and obtain a spatial attention map by taking the sub-feature group as input through the branch attention extraction network; The enhancement module is used to obtain an enhancement network, and obtain an enhancement target by taking the spatial attention map as input through the enhancement network.
9. A device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.