Remote sensing image target detection method, device and equipment fusing attention mechanism

Abstract

The invention relates to a remote sensing image target detection method, device and equipment fusing an attention mechanism, and the method comprises the steps: firstly extracting a multi-scale feature map of a preprocessed remote sensing image through a feature extraction network, adding the self-attention information for each scale feature map through an attention fusion unit, obtaining a self-attention feature map, and carrying out the recognition of a remote sensing image target. Performing cross-scale fusion through a multi-scale bidirectional feature pyramid network, performing channel-level weighted fusion through a standardized weighted fusion unit in the fusion, adding multi-scale information by means of a multi-scale bottleneck unit to obtain a multi-scale fusion feature map, and finally enhancing a detection head by channel attention to obtain a multi-scale fusion feature map. And after the important information of the fusion feature map of each scale is enhanced by a plurality of channel attention enhancement units, target detection is carried out, and the detection precision is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image detection, in particular to a remote sensing image target detection method, device and equipment fusing attention mechanism. BACKGROUND

[0002] Remote sensing image target detection, as an important branch of target detection, has gone through stages of hand-crafted features, deep learning driven and Transformer architecture. In the early stage, it relied on artificial design features and statistical learning methods, which were difficult to meet the detection needs of multi-angle and multi-scale targets in remote sensing images due to poor adaptability to complex backgrounds and scale changes. After AlexNet made a breakthrough in 2012, models based on convolutional neural networks were applied to remote sensing scenes, which were divided into two-stage (such as improved Faster R-CNN) and one-stage (such as improved YOLO) algorithms according to the detection strategy. With the development of Transformer in the field of computer vision, it is introduced into the field of remote sensing target detection. Researchers improve the DETR, ViT and other architectures to adapt to the remote sensing scene. The existing mature TPH-YOLOv5 algorithm combines multi-scale detection, attention mechanism and Transformer architecture, and has a significant performance in unmanned aerial vehicle image detection, but the existing technology still has some shortcomings. For example, in the feature extraction stage, the backbone network is difficult to extract enough small target information and distinguish targets from backgrounds. In the feature fusion stage, the feature pyramid structure has limitations in information transmission. In the detection head stage, the input information channels are given the same attention, which is easy to be disturbed by redundant information. In addition, in the design of the loss function, the CIoU loss has the problem of excessive punishment in small target detection, and the angle regression loss has the boundary problem. These problems restrict the accuracy and efficiency of remote sensing image target detection. SUMMARY

[0003] Therefore, it is necessary to provide a remote sensing image target detection method, device and equipment fusing attention mechanism, which can effectively provide target detection accuracy.

[0004] A remote sensing image target detection method fusing attention mechanism, the method comprises: acquiring a remote sensing image and pre-processing the remote sensing image; extracting multi-scale features of the pre-processed remote sensing image by using a feature extraction network to obtain a multi-scale feature map; adding self-attention information to the feature map of each scale by a plurality of attention fusion units to obtain a self-attention feature map corresponding to the scale; The multi-scale bidirectional feature gold tower network is used for cross-scale fusion of multi-scale self-attention feature maps. In the process of transmission and fusion of each self-attention feature map, the channel-level weighted information fusion is performed through a standardized weighted fusion unit, and then multi-scale information is added through a multi-scale bottleneck unit to obtain a multi-scale fused feature map. The channel attention is used to strengthen the detection head to perform target detection according to the multi-scale fused feature map. In the process of target detection, the important information in the fused feature map of each scale is enhanced through a plurality of channel attention strengthening units, and then target detection is performed.

[0005] In one embodiment, the attention fusion unit includes a local branch and a global branch. In the local branch, the feature map is subjected to preliminary feature transformation through a convolution layer, then the channel dimension is shuffled through a channel shuffle block, then the feature is refined through a depth-wise convolution layer, and finally the feature dimension is adjusted through a convolution layer to obtain a local feature. In the global branch, the feature map is processed through a convolution layer to generate a query vector, a key vector and a value vector respectively. The query vector and the key vector are operated to obtain an attention weight through Softmax, and then the value vector is weighted and fused to realize global context information mining and obtain a global feature. The local feature and the global feature are spliced to obtain a self-attention feature map.

[0006] In one embodiment, in the multi-scale bidirectional feature gold tower network, the standardized weighted fusion unit performs channel-level weighted information fusion on two adjacent feature maps of different scales. In the standardized weighted fusion unit: The two feature maps of different scales are spliced in the channel dimension to obtain a channel-dimensionally fused feature map. After the trainable weight coefficient is standardized, the standardized weight coefficient is assigned to each channel of the channel-dimensionally fused feature map for weighted fusion to obtain a standardized weighted fusion feature map. The standardized weighted fusion feature map is subjected to point-by-point convolution to obtain output data of the standardized weighted fusion unit.

[0007] In one embodiment, the multi-scale bottleneck unit includes a multi-scale convolution branch and an ordinary convolution branch. In the multi-scale convolution branch, the input feature is subjected to parallel multiple dilated convolution operations after a convolution layer, and then the operation results are element-by-element added and fused. The fused data is processed through a bottleneck block to obtain multi-scale convolution branch output data. In the ordinary convolution branch, the input feature sequentially passes through three convolution layers to obtain ordinary convolution branch output data. The multi-scale branch output data and the ordinary convolution branch output data are added together and then passed through a convolutional layer to obtain the output data of the multi-scale bottleneck unit.

[0008] In one embodiment, residual connectivity is controlled within the bottleneck block via a “TRUE / FALSE” parameter; When the parameter is "TRUE", the input features are processed by two concatenated convolutional blocks, while the input features are retained and fused with the output of the convolutional blocks through residual branches; When the parameter is "FALSE", the residual branch is not enabled, and the input features are processed by passing through two convolutional blocks in sequence.

[0009] In one embodiment, in the channel attention enhancement unit: The spatial dimension of the fused feature map at each scale is compressed by global average pooling. Each compressed channel interacts with multiple neighboring channels to generate a weight vector for each channel. Feature enhancement is achieved by multiplying the weight vector of each channel with the corresponding original information according to the channel dimension.

[0010] In one embodiment, a remote sensing image target detection network is constructed based on the feature extraction network, multiple attention fusion units, a multi-scale bidirectional feature pyramid, and a channel attention-enhanced detection head; The remote sensing image target detection network is used to detect targets in remote sensing images.

[0011] In one embodiment, the remote sensing image target detection network is trained using classification loss, confidence loss, regression loss, and angle classification loss.

[0012] This application also provides a remote sensing image target detection device that integrates an attention mechanism, the device comprising: The image acquisition module is used to acquire remote sensing images and preprocess the remote sensing images; The feature extraction module is used to extract multi-scale features from the preprocessed remote sensing image using a feature extraction network, thereby obtaining a multi-scale feature map. The self-attention information addition module is used to add self-attention information to feature maps at each scale through multiple attention fusion units to obtain self-attention feature maps at the corresponding scale. The multi-scale feature fusion module is used to perform cross-scale fusion of multi-scale self-attention feature maps using a multi-scale bidirectional feature pyramid network. During the transmission and fusion process of the respective attention feature maps, after channel-level weighted information fusion is performed by the standardized weighted fusion unit, multi-scale information is added by the multi-scale bottleneck unit to obtain the multi-scale fused feature map. The feature enhancement detection module is used to perform target detection based on the multi-scale fused feature map using the channel attention enhancement detection head. During the target detection process, multiple channel attention enhancement units are used to enhance important information in the fused feature map at each scale before target detection is performed.

[0013] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the steps in the remote sensing image target detection method with the fusion attention mechanism described above.

[0014] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a remote sensing image target detection method with a fusion attention mechanism.

[0015] The aforementioned remote sensing image target detection method, apparatus, and device based on the fusion attention mechanism extracts multi-scale features from preprocessed remote sensing images using a feature extraction network to obtain multi-scale feature maps. Multiple attention fusion units then add self-attention information to each scale's feature map, resulting in a corresponding scale's self-attention feature map. A multi-scale bidirectional feature pyramid network is used to perform cross-scale fusion of these multi-scale self-attention feature maps. During the fusion process, each attention feature map undergoes channel-level weighted information fusion via a standardized weighted fusion unit, followed by multi-scale information addition via a multi-scale bottleneck unit, resulting in a multi-scale fused feature map. Finally, a channel attention-enhanced detection head performs target detection based on the multi-scale fused feature map. During target detection, multiple channel attention enhancement units enhance important information in each scale's fused feature map before target detection. This method effectively improves the accuracy of target detection. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a remote sensing image target detection method that incorporates an attention mechanism in one embodiment. Figure 2 This is a schematic diagram illustrating the specific implementation process of a remote sensing image target detection method that incorporates an attention mechanism in one embodiment. Figure 3 This is a schematic diagram of the attention fusion unit in one embodiment; Figure 4This is a schematic diagram of the structure of a multi-scale bidirectional feature pyramid network in one embodiment; Figure 5 This is a schematic diagram of the structure of a standardized weighted fusion unit in one embodiment; Figure 6 This is a schematic diagram of the structure of a multi-scale bottleneck unit in one embodiment; Figure 7 for Figure 6 Schematic diagram of the bottleneck block; Figure 8 This is a schematic diagram of the channel attention enhancement unit in one embodiment; Figure 9 This is a structural block diagram of a remote sensing image target detection device that incorporates an attention mechanism in one embodiment. Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] To address the problems existing in current remote sensing image target detection networks, such as Figure 1 As shown, this application proposes a remote sensing image target detection method that incorporates an attention mechanism, specifically including the following steps: Step S100: Acquire remote sensing images and preprocess them.

[0019] Step S110: Use a feature extraction network to extract multi-scale features from the preprocessed remote sensing image to obtain a multi-scale feature map.

[0020] Step S120: Self-attention information is added to the feature maps at each scale through multiple attention fusion units to obtain the self-attention feature maps at the corresponding scales.

[0021] Step S130: Use a multi-scale bidirectional feature pyramid network to perform cross-scale fusion of multi-scale self-attention feature maps. During the transmission and fusion process of each attention feature map, after channel-level weighted information fusion is performed by a standardized weighted fusion unit, multi-scale information is added by a multi-scale bottleneck unit to obtain a multi-scale fused feature map.

[0022] Step S140: The channel attention enhancement detection head is used to perform target detection based on the multi-scale fusion feature map. During the target detection process, multiple channel attention enhancement units are used to enhance the important information in the fusion feature map at each scale before target detection is performed.

[0023] In this application, based on the single-stage target detection algorithm YOLOv5 framework, improvements are made to address the shortcomings of current algorithms at each stage, and a YOLOv5 remote sensing image target detection algorithm incorporating an attention mechanism is designed. For example... Figure 2 The diagram shows the overall architecture, which can be divided into a feature extraction network, an attention fusion unit, a multi-scale bidirectional feature pyramid network, and a channel attention-enhanced detection head. For an input remote sensing image, it is first preprocessed and then fed into the feature extraction network for feature extraction. The resulting multiple feature maps at different scales are then passed through an attention fusion unit to add self-attention information before being output to the multi-scale bidirectional feature pyramid network. In this network, the feature maps undergo channel-level weighted information fusion via a normalization weighted fusion unit and multi-scale information is added via a multi-scale bottleneck unit. The feature maps output from the multi-scale bidirectional feature pyramid network are then enhanced by a channel attention enhancement module to strengthen important channel information and suppress the influence of background and noise before finally being output to the detection head for detection.

[0024] In this method, a feature extraction network is used to obtain multi-scale feature maps, covering targets of varying sizes in remote sensing images. An attention fusion unit adds self-attention information, highlighting key features of smaller targets and suppressing irrelevant background, allowing subsequent processing to focus on effective information. Then, in the multi-scale bidirectional feature pyramid network, a standardized weighted fusion unit rationally allocates the weights of features from different channels, enabling better fusion of useful features. Combined with a multi-scale bottleneck unit to supplement multi-scale information, this alleviates the problem of varying target scales in remote sensing images, improves adaptability to targets of different scales, and reduces false negatives due to scale differences. Finally, a channel attention enhancement unit strengthens important channels and suppresses background noise, reducing the adverse effects of complex remote sensing scenes (such as noisy or congested environments), allowing the detection head to receive cleaner input with more prominent key features, aiding in accurate target identification, reducing false positives, and improving the overall accuracy and robustness of target detection in remote sensing images.

[0025] In step S100, the purpose of preprocessing is twofold: first, to enable images of different sizes to be input into the model, to output images to a fixed size through scaling operations, and to accelerate the training process and improve the stability of the model through normalization operations; and second, to enhance the diversity of data and the robustness of the model through data augmentation methods.

[0026] In step S110, the feature extraction network adopts the same architecture as the YOLOv5 network, which realizes feature extraction through the stacking of convolutional layers and C3 layers, and inputs the output multi-scale feature map into the attention fusion unit.

[0027] In step S120, the attention fusion unit includes a local branch and a global branch. In the local branch, the feature map undergoes preliminary feature transformation via convolutional layers, then the channel dimensions are shuffled by a channel shuffling block, followed by feature refinement via depthwise convolutional layers, and finally, the feature dimensions are adjusted by convolutional layers to obtain local features. In the global branch, the feature map is processed by convolutional layers to generate query vectors, key vectors, and value vectors. The query vector and key vector are operated on and then processed by Softmax to obtain attention weights, which are then weighted and fused with the value vectors to achieve global context information mining and obtain global features. Finally, the local and global features are concatenated to obtain a self-attention feature map.

[0028] like Figure 3 As shown, the attention fusion unit is divided into local and global branches. In the local branch, to better integrate features, the feature map is first enhanced by increasing the channel dimension through a 1×1 convolution to increase feature diversity. Then, the feature map after increasing the channel dimension is shuffled by a channel shuffling operation, and finally, a 1×1 convolution is used to restore the channel dimension. The channel shuffling operation in the local branch is mainly a residual structure. One path connects to the original information, and the other path first shuffles the channel dimension order through channel shuffling, and then uses depthwise convolution to process each channel individually.

[0029] Specifically, channel shuffling has many advantages. First, it can avoid overfitting to some extent while enhancing the model's generalization ability and robustness. Second, it can be done through tensor operations, reducing memory access costs. Then, the original information and the channel-shuffled information are concatenated along the channel dimension. Finally, a 1×1 convolution is used to restore the output feature map of the local branch to the original channel dimension.

[0030] Furthermore, in the global branch, the feature maps are first enlarged by 1×1 convolutions to enhance feature diversity, and then passed through three independent convolution kernels. These kernels generate three special feature maps, denoted as Query, Key, and Value. Next, the similarity between feature maps Q and K is calculated by multiplying the transposes of the Q and K matrices (multidimensional vector dot product), and the result is denoted as... By performing dot products between elements in the matrix, the similarity between elements at different positions in the original feature map can be calculated, representing the relationship between the two. The resulting... Each element in the matrix is ​​divided by a number; in attention mechanisms, this operation is called scaling. This is done to avoid numerical inflation caused by matrix multiplication. Then, a softmax operation is performed on the result. This is similar to why softmax is often used at the end of classification tasks to reduce the number of elements in the matrix. Transform the matrix into a probability matrix The higher the probability of an element, the higher the correlation between the two corresponding elements. This is the attention weight matrix, and its specific process is as follows: (1) In formula (1), This represents the dimension of matrix K.

[0031] Next, using the obtained relational probability matrix and By continuing matrix multiplication, we can obtain the correlation matrix between all global elements, and finally return to the original dimension through 1×1 convolution.

[0032] In step S130, the structure of the multi-scale bidirectional feature pyramid network is as follows: Figure 4 As shown, in the traditional network structure, the splicing unit is replaced with a standardized weighted fusion unit, and the C3 unit in the top-down branch is replaced with a multi-scale bottleneck unit. The former can achieve channel-dimensional information fusion, while the latter can add multi-scale information to the output feature map. These two units will be described in detail below.

[0033] In this embodiment, in the multi-scale bidirectional feature pyramid network, the standardized weighted fusion unit performs channel-level weighted information fusion on two adjacent feature maps of different scales. In the standardized weighted fusion unit, the feature maps of two different scales are concatenated along the channel dimension to obtain a channel-dimensional fused feature map. After standardizing the trainable weight coefficients, the standardized weight coefficients are assigned to each channel of the channel-dimensional fused feature map for weighted fusion to obtain a standardized weighted fused feature map. Finally, the standardized weighted fused feature map is convolved pointwise to obtain the output data of the standardized weighted fusion unit.

[0034] like Figure 5The diagram shows the specific structure of the standardized weighted fusion unit. In this unit, multiple input feature maps are concatenated along their channel dimensions to obtain a channel-dimensional fused feature map. Each channel in the fused map is then assigned a trainable weight coefficient. Before fusion, the model standardizes the trainable weight coefficients. This scales the weight vector to a distribution with a mean of 0 and a standard deviation of 1, unifying the feature scale, preventing numerical overflow or oscillations during gradient updates, making gradient descent more stable, and improving the model's generalization ability. After standardization, the weight coefficients are multiplied by their corresponding channels, and finally, pointwise convolutions are used to reduce the dimensionality of the fused feature map and output it.

[0035] like Figure 6 The diagram illustrates the specific structure of a multi-scale bottleneck unit, which includes a multi-scale convolutional branch and a regular convolutional branch. In the multi-scale convolutional branch, the input features undergo multiple parallel dilation convolution operations after passing through convolutional layers. The results are then element-wise summed and fused. The bottleneck block processes the fused data to obtain the multi-scale convolutional branch output. In the regular convolutional branch, the input features pass through three convolutional layers sequentially to obtain the regular convolutional branch output. The multi-scale branch output and the regular convolutional branch output are summed and then passed through one more convolutional layer to obtain the multi-scale bottleneck unit output.

[0036] In this embodiment, the ordinary convolutional branch structure in the multi-scale bottleneck unit is relatively simple, using only two 1×1 convolutions and one convolutional layer. The feature map is first increased in channel dimension through 1×1 convolution to improve feature complexity, thereby increasing the amount of information extracted by the subsequent convolutional operation. Then, it is reduced in dimension to the original channel dimension through another 1×1 convolution.

[0037] In this embodiment, in the multi-scale convolution branch of the multi-scale bottleneck unit, the input feature map is first increased in dimensionality through a 1×1 convolution, and then local context information at different scales is extracted through several parallel dilated convolution modules. The number of these parallel branches can be manually set, typically to two or three branches. When set to two branches (e.g....) Figure 6 As shown in the diagram, the dilation rates of the two branches are 2 and 3, respectively; when set to three branches, the dilation rates of the three branches are 2, 3, and 5, respectively. The feature maps generated after dilated convolution are fused together through a concatenation operation. It should be noted that in order to reduce the number of parameters used in subsequent convolutions, it is usually necessary to reduce the dimensionality to the dimension of the original input features. Generally, a 1×1 convolution is used to complete the dimensionality reduction.

[0038] Furthermore, after completing the multi-scale feature fusion, the feature map is input into the bottleneck block (NxBottleneck), whose structure is as follows: Figure 7 As shown, its internal structure is a residual structure. One branch consists of two convolutional modules connected in series, which enhances the original features. The other branch is the residual branch, responsible for incorporating the original information. The residual branch is controlled by a parameter: "TRUE" indicates its use, and "FALSE" indicates its disuse. This bottleneck module primarily enhances feature representation and promotes gradient flow, allowing gradients to be directly backpropagated and preventing degradation issues during deep network training. The number of bottleneck blocks can be controlled by a parameter; a parameter other than 1 indicates multiple bottleneck blocks connected in series.

[0039] In this embodiment, considering that traditional multi-scale bidirectional feature pyramid networks only fuse features by channel stacking without considering the differences in importance of different input features, which can easily lead to redundant information interference, the standardized weighted fusion module learns dynamic weights to perform channel-level weighted fusion of features from different sources (such as same-scale features and upsampling / downsampling features), achieving "weight allocation on demand." The standardized weighted fusion unit proposed in this method automatically strengthens the contribution of key features (such as target contours and details) and suppresses the interference of noise or background features through weight learning, solving the feature redundancy problem in complex scenes of remote sensing images. At the same time, compared with the surge in the number of channels brought about by the stitching module (requiring additional convolution to compress dimensions), weighted fusion maintains a stable number of channels through element-level addition, reducing subsequent computation and model parameters. The standardization operation can balance the distribution differences of different features to avoid feature conflicts caused by different scales or sources, and improve the stability of the fused features.

[0040] Furthermore, considering that the C3 module in the traditional multi-scale bidirectional feature pyramid network focuses on feature refinement at a single scale but has limited ability to capture multi-scale information, the multi-scale bottleneck module proposed in this method can actively embed multi-scale receptive fields in feature propagation through dynamic control of multi-branch convolution (such as parallel convolution with different kernel sizes) or residual connections, adapting to the characteristics of remote sensing targets with "variable scale" (such as from small vehicles to large building complexes).

[0041] Therefore, the multi-scale bidirectional feature pyramid network proposed in this method replaces the splicing unit with the standardized weighted fusion unit and replaces the C3 unit in the top-down branch with the multi-scale bottleneck unit. This enables efficient fusion of cross-scale features (highlighting key information through weighted fusion) and more accurately captures the features of targets at different scales (enhancing adaptability through the multi-scale bottleneck module). Ultimately, this improves the accuracy (especially for small targets and targets with varying scales) and robustness (resistance to background interference) of target detection in remote sensing images, while controlling model complexity and balancing detection efficiency.

[0042] In step S140, a channel attention enhancement unit is added before the basic detection head in the channel attention enhancement detection head to enhance important channel information and reduce the influence of background, noise and redundant information.

[0043] In this embodiment, in the channel attention enhancement unit, the spatial dimension of the fused feature map at each scale is compressed by global average pooling operation. Each compressed channel interacts with multiple neighboring channels to generate a weight vector for each channel. The weight vector of each channel is multiplied with the corresponding original information according to the channel dimension to achieve feature enhancement.

[0044] like Figure 8 The diagram shows the network structure of the channel attention enhancement unit. For the input feature map, this unit first compresses the spatial dimension of the input information to 1×1 using global average pooling, achieving spatial dimensional information fusion and interaction. Then, each compressed channel interacts with its k nearest neighbors to generate weight information for each channel, calculating the channel-dimensional weights. Finally, the channel weight vector is multiplied by the original information according to the channel dimension to enhance the channel-dimensional information of the input feature map.

[0045] In this embodiment, a remote sensing image target detection network is constructed based on the aforementioned feature extraction network, multiple attention fusion units, multi-scale bidirectional feature pyramid, and channel attention enhancement detection head. The remote sensing image target detection network is then used to directly detect targets in the remote sensing image.

[0046] In this embodiment, classification loss, confidence loss, regression loss, and angle classification loss are used to train the remote sensing image target detection network.

[0047] Specifically, the classification loss is calculated using cross-entropy loss, and the following formula is used: (2) In formula (2), This represents the true classification result. This represents the classification result predicted by the model, where n represents the number of categories.

[0048] Specifically, the confidence loss is also calculated using cross-entropy loss, and the following formula is used: (3) In formula (3), This represents the true confidence level (0 or 1). This represents the confidence level of the model in predicting the current category, where n represents the number of categories.

[0049] In this embodiment, the regression loss calculation method, compared with the traditional intersection-over-union (IoU) calculation method, adds a center distance loss. and aspect ratio loss The center distance loss is calculated by dividing the squared Euclidean distance between the center points of the predicted and ground truth bounding boxes by the area of ​​the ground truth bounding box. This method addresses the issue of small targets in remote sensing imagery being overly sensitive to their own positional deviations when using IoU as the regression loss, thus aiding model convergence. The aspect ratio loss function is more complex. It first calculates the ratio of the squared height and width Euclidean distances to the squared corresponding ground truth values, then sums them, and finally applies area weights. and aspect ratio weight and Adjust its in The weights in the calculation method allow the model to assign different loss weights to the height and width of the detection box, making it more adaptable to situations where the aspect ratio of the target detection box varies greatly in remote sensing scenes.

[0050] Specifically, the regression loss is calculated using the following formula: (4) in, ; ; In formula (4), These are the aspect ratio weights, respectively. The area weight is calculated using the following formula: (5) In formula (5), This is the adjusted Sigmoid function. and These are the areas of the predicted bounding box and the ground truth bounding box, respectively. The calculation method is expressed as follows: (6) In formula (6), It is a control The parameter for the smoothness of the function, when When it is large, The function curve becomes steeper, smaller targets receive smaller weights, and the loss function is less sensitive to small targets than to large targets, but when... When the value is too large, the loss function becomes highly sensitive to both small and large targets; when... When smaller, The function curve is relatively flat, and the area weight is not so sensitive to changes in the size of the target.

[0051] In this embodiment, the angle loss uses classification loss instead of the conventional regression loss. The classification loss calculation method can effectively avoid the boundary problem caused by the discontinuity of the prediction space of angle information. This calculation method first maps the continuous angle space from 0 to π into N discrete intervals (N is usually 180), and constructs an angle classification task to replace the regression task. Then, Gaussian kernel function is used to generate soft labels for adjacent intervals to avoid training jitter caused by hard label boundaries. Finally, the topological continuity of the angle distribution is ensured by the ring adjacency design (the Nth interval is adjacent to the first interval at both ends).

[0052] Specifically: (7) Formula (7) represents the label value of the angle loss, and the corresponding loss value is 1 minus this label value. In formula (7), It's a window function. It is the radius of the window function. This represents the angle of the current bounding box. (When the category (angle in degrees)...) In the true value The radius of the surrounding area is When within the window, the degree at this time The value is obtained through window functions. Calculate the value; otherwise, set it to 0. It also needs to satisfy the window function. The function must exhibit periodicity, symmetry, and monotonicity. Since it represents probability, its maximum value is 1. Suitable functions include rectangular functions, triangular functions, and Gaussian functions. Furthermore, using this window function to calculate the angular classification probability avoids the problem of being unable to distinguish the magnitude of the loss value.

[0053] In the aforementioned remote sensing image target detection method incorporating an attention mechanism, during the feature extraction stage, an attention fusion module adds global self-attention information to the multi-scale feature map to compensate for the lack of geometric information caused by the small target size. During the feature fusion stage, a multi-scale bidirectional feature pyramid structure addresses both the model's inability to distinguish the importance of information from different channels and the problem of small-scale targets being unable to utilize information from large-scale targets due to the model completely separating the detection processes for targets of different scales. In the detection stage, a channel attention enhancement module is used to enhance the input feature map, thereby strengthening key channel information and suppressing noise and background information. Simultaneously, during network training, a multi-scale attention fusion module is used to enhance the input feature map, thereby strengthening key channel information and suppressing noise and background information. The calculation method replaces the IoU calculation method to solve the problem that small targets are too sensitive to their own position; the angle classification loss is used instead of the angle regression loss to avoid the angle boundary problem.

[0054] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0055] In one embodiment, such as Figure 9 As shown, a remote sensing image target detection device with an attention-based fusion mechanism is provided, comprising: an image acquisition module 200, a feature extraction module 210, a self-attention information addition module 220, a multi-scale feature fusion module 230, and a feature enhancement detection module 240, wherein: The image acquisition module 200 is used to acquire remote sensing images and preprocess the remote sensing images; The feature extraction module 210 is used to extract multi-scale features from the preprocessed remote sensing image using a feature extraction network to obtain a multi-scale feature map. The self-attention information addition module 220 is used to add self-attention information to the feature maps at each scale through multiple attention fusion units to obtain the self-attention feature maps at the corresponding scales. The multi-scale feature fusion module 230 is used to perform cross-scale fusion of multi-scale self-attention feature maps using a multi-scale bidirectional feature pyramid network. During the transmission and fusion process of the respective attention feature maps, after channel-level weighted information fusion is performed by the standardized weighted fusion unit, multi-scale information is added by the multi-scale bottleneck unit to obtain the multi-scale fused feature map. The feature enhancement detection module 240 is used to perform target detection based on the multi-scale fused feature map using a channel attention enhancement detection head. During target detection, multiple channel attention enhancement units enhance important information in the fused feature map at each scale before target detection is performed.

[0056] Specific limitations regarding the remote sensing image target detection device based on the fusion attention mechanism can be found in the limitations of the remote sensing image target detection method based on the fusion attention mechanism described above, and will not be repeated here. Each module in the aforementioned remote sensing image target detection device based on the fusion attention mechanism can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0057] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a remote sensing image target detection method based on an attention-based mechanism. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0058] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0059] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Acquire remote sensing images and preprocess the remote sensing images; A feature extraction network is used to extract multi-scale features from the preprocessed remote sensing image to obtain a multi-scale feature map. By adding self-attention information to the feature maps at each scale through multiple attention fusion units, self-attention feature maps at the corresponding scales are obtained. A multi-scale bidirectional feature pyramid network is used to perform cross-scale fusion of multi-scale self-attention feature maps. During the transmission and fusion process of the respective attention feature maps, channel-level weighted information fusion is performed through a standardized weighted fusion unit, and then multi-scale information is added through a multi-scale bottleneck unit to obtain a multi-scale fused feature map. The target detection head is used to perform target detection based on the multi-scale fused feature map. During the target detection process, multiple channel attention enhancement units are used to enhance important information in the fused feature map at each scale before target detection is performed.

[0060] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire remote sensing images and preprocess the remote sensing images; A feature extraction network is used to extract multi-scale features from the preprocessed remote sensing image to obtain a multi-scale feature map. By adding self-attention information to the feature maps at each scale through multiple attention fusion units, self-attention feature maps at the corresponding scales are obtained. A multi-scale bidirectional feature pyramid network is used to perform cross-scale fusion of multi-scale self-attention feature maps. During the transmission and fusion process of the respective attention feature maps, channel-level weighted information fusion is performed through a standardized weighted fusion unit, and then multi-scale information is added through a multi-scale bottleneck unit to obtain a multi-scale fused feature map. The target detection head is used to perform target detection based on the multi-scale fused feature map. During the target detection process, multiple channel attention enhancement units are used to enhance important information in the fused feature map at each scale before target detection is performed.

[0061] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A remote sensing image target detection method incorporating an attention mechanism, characterized in that, The method includes: Acquire remote sensing images and preprocess the remote sensing images; A feature extraction network is used to extract multi-scale features from the preprocessed remote sensing image to obtain a multi-scale feature map. By adding self-attention information to the feature maps at each scale through multiple attention fusion units, self-attention feature maps at the corresponding scales are obtained. A multi-scale bidirectional feature pyramid network is used to perform cross-scale fusion of multi-scale self-attention feature maps. During the transmission and fusion process of the respective attention feature maps, channel-level weighted information fusion is performed through a standardized weighted fusion unit, and then multi-scale information is added through a multi-scale bottleneck unit to obtain a multi-scale fused feature map. The target detection head is used to perform target detection based on the multi-scale fused feature map. During the target detection process, multiple channel attention enhancement units are used to enhance important information in the fused feature map at each scale before target detection is performed.

2. The remote sensing image target detection method based on the fusion attention mechanism according to claim 1, characterized in that, The attention fusion unit includes local branches and global branches; In the local branch, the feature map undergoes preliminary feature transformation through a convolutional layer, then the channel dimensions are shuffled by a channel shuffling block, then the features are refined by a depthwise convolutional layer, and finally the feature dimensions are adjusted by the convolutional layer to obtain local features. In the global branch, the feature map is processed by a convolutional layer to generate a query vector, a key vector, and a value vector. The query vector and the key vector are then processed by Softmax to obtain attention weights, which are then weighted and fused with the value vectors to achieve global context information mining and obtain global features. The self-attention feature map is obtained by concatenating the local and global features.

3. The remote sensing image target detection method based on the fusion attention mechanism according to claim 1, characterized in that, In the multi-scale bidirectional feature pyramid network, the standardized weighted fusion unit performs channel-level weighted information fusion on two adjacent feature maps of different scales. Within the standardized weighted fusion unit: Two feature maps of different scales are concatenated along the channel dimension to obtain a feature map with channel dimension fusion. After standardizing the trainable weight coefficients, the standardized weight coefficients are assigned to each channel of the channel-dimensional fused feature map, and weighted fusion is performed to obtain the standardized weighted fused feature map. The standardized weighted fusion feature map is convolved point by point to obtain the output data of the standardized weighted fusion unit.

4. The remote sensing image target detection method based on the fusion attention mechanism according to claim 1, characterized in that, The multi-scale bottleneck unit includes multi-scale convolutional branches and ordinary convolutional branches; In the multi-scale convolution branch, the input features undergo multiple parallel dilation convolution operations after passing through the convolutional layer. The results of the operations are then added and fused element by element. The fused data is processed using a bottleneck block to obtain the output data of the multi-scale convolution branch. In the ordinary convolutional branch, the input features pass through three convolutional layers in sequence to obtain the ordinary convolutional branch output data; The multi-scale branch output data and the ordinary convolution branch output data are added together and then passed through a convolutional layer to obtain the output data of the multi-scale bottleneck unit.

5. The remote sensing image target detection method based on the fusion attention mechanism according to claim 4, characterized in that, In the bottleneck block, residual connectivity is controlled by the "TRUE / FALSE" parameter; When the parameter is "TRUE", the input features are processed by two concatenated convolutional blocks, while the input features are preserved and fused with the output of the convolutional blocks through residual branches; When the parameter is "FALSE", the residual branch is not enabled, and the input features are processed by passing through two convolutional blocks in sequence.

6. The remote sensing image target detection method based on the fusion attention mechanism according to claim 1, characterized in that, In the channel attention enhancement unit: The spatial dimension of the fused feature map at each scale is compressed by global average pooling. Each compressed channel interacts with multiple neighboring channels to generate a weight vector for each channel. Feature enhancement is achieved by multiplying the weight vector of each channel with the corresponding original information according to the channel dimension.

7. The remote sensing image target detection method based on the fusion attention mechanism according to claim 1, characterized in that, A remote sensing image target detection network is constructed based on the feature extraction network, multiple attention fusion units, multi-scale bidirectional feature pyramids, and channel attention-enhanced detection head. The remote sensing image target detection network is used to detect targets in remote sensing images.

8. The remote sensing image target detection method based on the fusion attention mechanism according to claim 7, characterized in that, The remote sensing image target detection network is trained using classification loss, confidence loss, regression loss, and angle classification loss.

9. A remote sensing image target detection device incorporating an attention mechanism, characterized in that, The device includes: The image acquisition module is used to acquire remote sensing images and preprocess the remote sensing images; The feature extraction module is used to extract multi-scale features from the preprocessed remote sensing image using a feature extraction network, thereby obtaining a multi-scale feature map. The self-attention information addition module is used to add self-attention information to feature maps at each scale through multiple attention fusion units to obtain self-attention feature maps at the corresponding scale. The multi-scale feature fusion module is used to perform cross-scale fusion of multi-scale self-attention feature maps using a multi-scale bidirectional feature pyramid network. During the transmission and fusion process of the respective attention feature maps, after channel-level weighted information fusion is performed by the standardized weighted fusion unit, multi-scale information is added by the multi-scale bottleneck unit to obtain the multi-scale fused feature map. The feature enhancement detection module is used to perform target detection based on the multi-scale fused feature map using the channel attention enhancement detection head. During the target detection process, multiple channel attention enhancement units are used to enhance important information in the fused feature map at each scale before target detection is performed.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images