A remote sensing image registration method based on convolutional attention

Abstract

The present invention relates to the field of image processing technology, and in particular to a remote sensing image registration method based on convolutional attention, comprising: inputting the original remote sensing image pair into the convolution layer of a modality adapter to obtain the original features, using the convolutional attention method to process the original features to obtain the target features; inputting the target features into a feature encoder as modality adaptation features to obtain feature descriptors; inputting the feature descriptors into a feature detector to estimate global feature points using conditional probability; performing preliminary matching of the feature descriptors and the feature points using the FLANN algorithm, and removing external points in combination with the RANSAC algorithm to obtain the registration results. The present invention can provide richer feature representations for remote sensing images in sparse and repeated texture areas, and improve the registration effect.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a remote sensing image registration method based on convolutional attention. Background Art

[0002] Before the emergence of deep learning algorithms, methods based on handcrafted descriptors were widely used in various registration tasks. However, due to the design limitations of existing knowledge, traditional matching algorithms often perform poorly when dealing with more complex matching tasks due to handcrafted descriptors. With the vigorous development of deep learning algorithms, their deep feature extraction and expression capabilities can make up for the defects of shallow feature instability and improve the robustness of registration algorithms. However, for the registration of remote sensing images, due to the presence of a large number of sparse and repeated texture areas, the challenge is to simultaneously solve the dual tasks of feature point detection and description. This requires the model to accurately capture the location of key points while also extracting their high-level semantic information.

[0003] Recent joint detection and description methods extract keypoints from deep but coarse feature maps. The SuperPoint algorithm performs keypoint detection with pixel-level supervision, but this training cost is prohibitive for multimodal remote sensing scenes with limited effective data. Although ASLFeat upsamples and fuses keypoint scores from different resolutions to obtain all keypoints and maintain their spatial accuracy, it only integrates score maps across resolutions, limiting the amount of multi-level information it can capture. As an alternative, the convolutional attention mechanism can enhance the learning of more discriminative feature representations while improving localization accuracy by leveraging its flexible structural features in spatial and channel dimensions. However, their common practice of compressing each dimension to provide richer feature representations to other dimensions inevitably leads to information loss. In addition, utilizing multiple attention modules usually leads to increased computational cost. Therefore, studying how to better integrate features across different dimensions while maintaining low computational overhead is a valuable research topic.

[0004] In a neural network, an attention mechanism is usually an additional neural network that can rigidly select certain parts of the input or assign different weights to different parts of the input. The attention mechanism can filter out important information from a large amount of information. There are many ways to introduce the attention mechanism in a neural network. Taking convolutional neural networks as an example, attention mechanisms can be added in the spatial dimension, in the channel dimension, or in the mixed dimension (mixed spatial and channel dimensions).

[0005] The convolution kernel parameters of the convolutional neural network are fixed, and the weights cannot be adaptively adjusted according to the input, which means that the convolution kernel pays equal attention to features at different positions and cannot distinguish between important and unimportant areas. However, the attention mechanism can select certain parts of the input, or assign different weights to parts of the input, thereby filtering out important information from a large amount of information, improving positioning accuracy while enhancing the learning of more discriminative feature representations. However, the common practice is to compress each dimension to provide richer feature representations for other dimensions, which inevitably leads to information loss. In addition, the use of multiple attention mechanism modules often increases the computational cost. Therefore, how to better integrate features of different dimensions while keeping the computational overhead low is a problem worth studying.

[0006] With the development of convolutional neural networks, extending multiple convolutional layers in convolutional neural networks can significantly enhance the learned feature representations. However, this also leads to the need to stack more deep convolutional layers, consuming a lot of memory and computing resources. As an alternative solution, the attention mechanism, with its flexible structural characteristics, can not only enhance the learning of more discriminative feature representations, but also be easily integrated into the backbone architecture of convolutional neural networks. Therefore, the attention mechanism has gained widespread attention in the field of computer vision research and is often applied to tasks such as image classification and image segmentation.

[0007] As can be seen from the above, attention mechanisms usually include the following three types: channel attention, spatial attention, and a combination of the two. The SE (Squeeze and Excitation) attention module is a channel attention module, so it only considers the attention on the channel dimension and cannot capture the attention on the spatial dimension, that is, it ignores the importance of position information. The CBAM (Convolutional Block Attention Module) attention module is an attention module that combines channel attention and spatial attention. It can pay attention to images from both the spatial dimension and the channel dimension, but it cannot simulate long-term dependencies. The CA (coordinate attention) attention module decomposes channel attention into two one-dimensional feature encoding processes, and aggregates features along two spatial directions respectively. In this way, it can capture long-range dependencies along one spatial direction while retaining the precise position information of the other spatial direction. However, there are still some problems with the attention modules mentioned above: First, the SE attention module, CA attention module, and CBAM attention module all use the scaling of the channel dimension to improve computational efficiency, but this may have side effects for extracting deep feature representations. Second, the CA attention module ignores the importance of interactions between all spatial positions.

[0008] For the problem of side effects in extracting deep feature representations, attention modules using feature grouping have been widely studied. Specifically, the ResNeXt module has proven that the activation of sub-features can be affected by different patterns and noisy backgrounds. The ResNeXt module is a typical hybrid model composed of the basic Inception model and the ResNet model. For the problem that the CA attention module ignores the importance of interactions between all spatial positions, the efficient multi-scale attention EMA module uses three parallel paths to extract attention weight descriptors, one of which provides a cross-spatial information aggregation method of different spatial dimensions to obtain richer feature aggregation. The pyramid spatial attention PSA module fully utilizes the representation capabilities within its channel branch and spatial branch while maintaining the highest internal resolution.

[0009] However, the activation of sub-features (i.e., feature grouping) adopted by ResNeXt lacks communication between groups, resulting in duplication of low-dimensional features to some extent, thus limiting the ability of feature representation; the EMA module also does not implement cross-channel learning. Therefore, both models lack information in the channel dimension, which is crucial for rich descriptor data representation. Therefore, how to make the attention module better aggregate grouping and information between channels to achieve richer feature expression is a technical problem that needs to be solved urgently. Summary of the invention

[0010] In view of this, the present invention aims to provide a remote sensing image registration method based on convolutional attention, which can provide richer feature representation for remote sensing images in sparse and repeated texture areas and improve the registration effect.

[0011] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0012] The invention provides a remote sensing image registration method based on convolutional attention, comprising: inputting the original remote sensing image pair into the convolution layer of the modality adapter to obtain the original features, using the convolutional attention method to process the original features to obtain the target features; inputting the target features as modality adaptation features into the feature encoder to obtain feature descriptors; inputting the feature descriptors into the feature detector to use conditional probability estimation to globally detect feature points; performing preliminary matching of the feature descriptors and feature points using the FLANN algorithm, and removing external points in combination with the RANSAC algorithm to obtain the registration results; the convolutional attention method is applied to the modality adapter and the feature encoder, and the convolutional attention method comprises: inputting the original features into the feature encoder to obtain the feature descriptors; inputting the feature descriptors into the feature detector to obtain the feature points; performing preliminary matching of the feature descriptors and feature points using the FLANN algorithm, and removing external points in combination with the RANSAC algorithm to obtain the registration results; the convolutional attention method is applied to the modality adapter and the feature encoder, and the convolutional attention method comprises: Input the coordinate attention module, the coordinate attention module groups the original features to obtain sub-features, and generates coordinate weights based on the sub-features, and uses the coordinate weights to weight the sub-features to output coordinate features; input the coordinate features and the original features into the cross-channel attention module, the cross-channel attention module generates channel weights based on the coordinate features and the original features; use the channel weights to weight the original features to obtain channel features; input the coordinate features and sub-features into the cross-spatial attention module, the cross-spatial attention module generates spatial weights based on the coordinate features and the sub-features; use the spatial weights to weight the original features to obtain spatial features; fuse the channel features and spatial features to obtain the target features.

[0013] Furthermore, inputting the coordinate features and the original features into the cross-channel attention module includes: inputting the coordinate features and the original features of the first convolution into the cross-channel attention module; inputting the coordinate features and the sub-features into the cross-spatial attention module includes: inputting the coordinate features and the sub-features of the second convolution into the cross-spatial attention module.

[0014] Furthermore, the first convolution is a 3×3 convolution, and the second convolution is a 3×3 convolution.

[0015] Furthermore, generating coordinate weights based on sub-features includes: performing axial compression on the sub-features along the horizontal direction to obtain horizontal compression features, performing axial compression on the sub-features along the vertical direction to obtain vertical compression features, concatenating and convolving the horizontal compression features and the vertical compression features, decomposing the output of the convolution into a first vector and a second vector, performing activation function calculation on the first vector, and performing activation function calculation on the second vector, and multiplying the first vector calculated by the activation function and the second vector calculated by the activation function to obtain the coordinate weight.

[0016] Furthermore, the convolution is a 1×1 convolution.

[0017] Furthermore, the cross-channel attention module generates channel weights based on the coordinate features and the original features, including: performing batch normalization, channel dimension pooling and SoftMax normalization on the coordinate features in sequence to obtain a first transformation feature, performing a dot product operation on the first transformation feature and the original feature of the first convolution to obtain cross-channel adaptive information; performing channel dimension pooling and SoftMax normalization on the original feature of the first convolution in sequence to obtain a second transformation feature, performing a dot product operation on the second transformation feature and the batch-normalized coordinate feature to obtain cross-channel group information; aggregating the cross-channel adaptive information and the cross-channel group information to obtain a first aggregation result, and performing a Sigmoid activation function calculation on the first aggregation result to obtain the channel weight.

[0018] Furthermore, the cross-spatial attention module generates spatial weights based on coordinate features and sub-features, including: performing inter-group normalization, two-dimensional global average pooling, and SoftMax normalization on the coordinate features in sequence to obtain a third transformation feature, performing a dot product operation on the third transformation feature and the sub-feature of the second convolution to obtain a fourth transformation feature, performing a dot product operation on the fourth transformation feature and the coordinate feature that has been normalized between groups to obtain cross-spatial long-range dependency information; aggregating the cross-spatial short-range dependency information and the cross-spatial long-range dependency information to obtain a second aggregation result, and performing a Sigmoid activation function calculation on the second aggregation result to obtain the spatial weight.

[0019] Compared with the prior art, the invention can achieve the following beneficial effects: in the remote sensing image registration method based on convolutional attention, the convolutional attention method adopts a coordinate attention module to capture long-range dependencies with precise position information, adopts a cross-space attention module to aggregate information in different spatial dimensions, and adopts a cross-channel attention module to aggregate information in different channel dimensions and between groups. Therefore, the information between groups and channels can be better aggregated to achieve richer feature expression, and then the convolutional attention method is used for remote sensing image registration, which can provide richer feature representation for remote sensing images in sparse and repeated texture areas and improve the registration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0021] Figure 1 A schematic diagram of the overall architecture of the remote sensing image registration method based on convolutional attention created by the present invention;

[0022] Figure 2An algorithm flow chart of the convolutional attention method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0024] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0025] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0026] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0027] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0028] In order to meet the challenges of effective feature point detection and description in remote sensing scenes with repetitive or sparse texture areas, an embodiment of the present invention provides a remote sensing image registration method based on convolutional attention, which is implemented based on a convolutional attention method with parallel cross-channel attention modules and cross-spatial attention modules. The convolutional attention method uses a parallel network to combine coordinate dimensions, spatial dimensions, and channel dimensions, further enhancing the expressiveness of features, thereby providing richer feature representations for remote sensing images in sparse and repetitive texture areas and improving the registration effect.

[0029] refer to Figure 1 and Figure 2 The remote sensing image registration method based on convolutional attention provided by the present invention includes: S1: inputting the original remote sensing image pair into the convolution layer of the modality adapter to obtain the original features; S2: using the convolutional attention method to process the original features to obtain the target features; the convolutional attention method is applied to the modality adapter and the feature encoder, the convolutional attention method, the convolutional attention method includes: inputting the original features into the coordinate attention module, the coordinate attention module groups the original features to obtain sub-features, and generates coordinate weights based on the sub-features, and weights the sub-features using the coordinate weights to output coordinate features; the coordinate features and the original features are input into the cross-channel attention module, the cross-channel attention module is based on the coordinate features and The original features generate channel weights; the original features are weighted by the channel weights to obtain channel features; the coordinate features and sub-features are input into the cross-spatial attention module, and the cross-spatial attention module generates spatial weights based on the coordinate features and sub-features; the original features are weighted by the spatial weights to obtain spatial features; the channel features and spatial features are fused to obtain the target features; S3: the target features are input into the feature encoder as the modal adaptation features to obtain the feature descriptor; S4: the feature descriptor is input into the feature detector to globally detect feature points using conditional probability estimation; S5: the feature descriptors and feature points are preliminarily matched using the FLANN algorithm, and the outliers are eliminated using the RANSAC algorithm to obtain the final registration result.

[0030] The remote sensing image registration method based on convolutional attention provided by the present invention adopts an improved convolutional attention method to replace the traditional convolution in the modality adapter and feature encoder parts, which can provide richer feature representation for remote sensing images in sparse and repeated texture areas and improve the registration effect.

[0031] refer to Figure 1 , Figure 1The overall architecture of the remote sensing image registration method is presented. It is a convolution-attention-based remote sensing image registration method based on the ReDFeat algorithm. The improved convolution-attention method is used to replace the traditional convolution, and two modal remote sensing images are obtained as input. Specifically, each image is processed by the modality adapter, and then a dense feature descriptor is generated by a weight-shared feature encoder. In addition, a weight-shared feature detector is used to generate feature points. Finally, the FLANN algorithm is used for preliminary matching, and the RANSAC algorithm is combined to remove outliers to obtain the final registration result.

[0032] In some examples, the modality adapter can be composed of three cascaded network modules, each of which includes a convolutional layer, an improved convolutional attention layer (i.e., a system that processes features using the convolutional attention method provided by the present invention), and a skip connection. The output feature map resolution of the three cascaded network modules is maintained at , the number of output channels are 32, 64 and 128 respectively. The convolutional layer can be described as: ;in, represents the first original feature, stands for convolution, BN stands for batch normalization, and ReLU stands for activation function. The feature encoder can be composed of two convolutional layers and an improved convolutional attention layer combined with skip connections. The feature encoder is used to generate dense feature descriptors. , , .

[0033] The input of the feature detector consists of two branches, which is the same as the network structure of ReDFeat. Specifically, one branch is the original detector, which is used to simulate the prior probability , which means that the probability of the i-th point being a feature point is directly calculated based on its descriptor. Indicates that point i is a feature point, and represents the descriptor of the i-th point. The structure of the other branch requires the conditional probability Modeling is performed, and the probability represents the probability of detecting it as a feature point according to the global feature descriptor, where Indicates that point i is detected. Then, Indicates the posterior probability that the detected point is a feature point, which can be expressed by Approximate calculation is performed to finally obtain the feature points.

[0034] Finally, the obtained feature descriptors and feature points are preliminarily matched using the FLANN algorithm, and then RANSAC is used to eliminate external points to obtain the homography matrix and complete the image registration.

[0035] Furthermore, inputting the coordinate features and the original features into the cross-channel attention module is specifically: inputting the coordinate features and the original features of the first convolution into the cross-channel attention module, and inputting the coordinate features and the sub-features into the cross-spatial attention module is specifically: inputting the coordinate features and the sub-features of the second convolution into the cross-spatial attention module.

[0036] It should be noted that the original features of the first convolution refer to the original features processed by the first convolution, and the sub-features of the second convolution refer to the sub-features processed by the second convolution.

[0037] The cross-spatial attention module fuses the coordinate features (global information) with the sub-features (local information) of the second convolution to establish long-distance dependencies and short-distance dependencies in the spatial range respectively. The cross-channel attention module fuses the coordinate features (grouping features) with the original features (full-channel features) of the first convolution to integrate the grouping information while adaptively focusing on important channel information. The output of the entire network is obtained by aggregating the spatial features output by the cross-spatial attention module and the channel features output by the cross-channel attention module. The cross-spatial attention module and the cross-channel attention module are run in parallel, which helps to avoid more sequential processing and greater network depth in the entire network.

[0038] Furthermore, the first convolution is a 3×3 convolution, and the second convolution is a 3×3 convolution. The original features of the first convolution refer to the original features processed by the 3×3 convolution, and the sub-features of the second convolution refer to the sub-features processed by the 3×3 convolution.

[0039] Furthermore, generating coordinate weights based on sub-features includes: performing axial compression on sub-features in the horizontal direction to obtain horizontal compression features, performing axial compression on sub-features in the vertical direction to obtain vertical compression features, concatenating and convolving the horizontal compression features and the vertical compression features, decomposing the output of the convolution into a first vector and a second vector, performing activation function calculation on the first vector, and performing activation function calculation on the second vector, and multiplying the first vector calculated by the activation function and the second vector calculated by the activation function to obtain coordinate weights. Among them, the coordinate attention module performs axial compression on the grouped sub-features in the horizontal and vertical directions so that each precise position is assigned a different feature weight and a larger receptive field is obtained.

[0040] Furthermore, the horizontal compression features and the vertical compression features are concatenated and convolved, and the convolution is a 1×1 convolution.

[0041] Furthermore, the cross-channel attention module generates channel weights based on coordinate features and original features, including: performing batch normalization, channel dimension pooling, and SoftMax normalization on the coordinate features in sequence to obtain a first transformation feature, performing a dot product operation on the first transformation feature and the original feature of the first convolution to obtain cross-channel adaptive information; performing channel dimension pooling and SoftMax normalization on the original feature of the first convolution in sequence to obtain a second transformation feature, performing a dot product operation on the second transformation feature and the batch-normalized coordinate feature to obtain cross-channel group information; aggregating the cross-channel adaptive information and the cross-channel group information to obtain a first aggregation result, and performing a Sigmoid activation function calculation on the first aggregation result to obtain the channel weight. The purpose of the cross-channel attention module is to integrate information between different groups to enrich feature representation.

[0042] Furthermore, the cross-spatial attention module generates spatial weights based on coordinate features and sub-features, including: performing inter-group normalization, two-dimensional global average pooling, and SoftMax normalization on the coordinate features to obtain a third transformation feature, performing dot product operations on the third transformation feature and the sub-features of the second convolution to obtain cross-spatial short-range dependency information; performing two-dimensional global average pooling and SoftMax normalization on the sub-features of the second convolution to obtain a fourth transformation feature, performing dot product operations on the fourth transformation feature and the coordinate features that have been inter-group normalized to obtain cross-spatial long-range dependency information; aggregating cross-spatial short-range dependency information and cross-spatial long-range dependency information to obtain a second aggregation result, and performing Sigmoid activation function calculation on the second aggregation result to obtain spatial weights. The cross-spatial attention module can effectively process short-range dependencies and long-range dependencies in spatial position information.

[0043] The convolutional attention method in the remote sensing image registration method based on convolutional attention provided by the embodiment of the present invention focuses on richer and more comprehensive feature dimensions, including coordinate dimensions, spatial dimensions and channel dimensions, and designs a parallelization structure and a grouping strategy to reduce the amount of calculation, while achieving effective feature transfer and integration. Both the EMA module and the ResNeXt module using feature grouping lack attention learning in the channel dimension. The convolutional attention method provided by the embodiment of the present invention better aggregates grouping and information between channels, achieving richer feature expression. In this way, using the convolutional attention method to replace traditional convolution for remote sensing image registration can provide richer feature representation for remote sensing images in sparse and repeated texture areas, and improve the registration effect.

[0044] The specific implementation example of the convolutional attention method is described below.

[0045] In some embodiments of the present invention, reference Figure 2 , the original features Input coordinate attention module, where , H represents the width of the original feature, W represents the height of the original feature, C represents the number of channels of the original feature, and R represents a real number set. Divide into Group characteristics, The group characteristics are expressed as , the sub-features are expressed as , , i=0, 1, ..., G-1, and divide them into The group features are processed in batch-dimensional parallel computing. Since the channel dimension of the convolution kernel is not affected by the batch dimension, the grouping operation can reduce the channel dimension At the same time, sub-features across channel dimensions can learn different semantics.

[0046] In some examples, It can be 8.

[0047] Continue to refer Figure 2 In some embodiments of the present invention, the coordinate attention module first performs axial compression (X average pooling) on ​​the grouped sub-features in the horizontal direction and in the vertical direction (Y average pooling) to embed the spatial position information into the axial attention map. The pooling kernel for performing axial compression in the horizontal direction is , the pooling kernel that performs axial compression along the vertical direction is , through axial compression to encode each point on the horizontal and vertical coordinates, then concatenate the horizontal compression features and vertical compression features obtained by axial compression, and then use a shared The convolution kernel is used to effectively capture the relationship between axes. The output of the convolution can be expressed as:

[0048] ;

[0049] Among them, Conv1×1 represents Convolution, Concat stands for concatenation, represents the horizontal compression feature, represents the vertical compression feature, represents the feature of the wth column in the sub-feature, w=0, 1, ..., W-1, Represents the feature of the hth row in the sub-feature, h=0, 1, ..., H-1.

[0050] The output of the convolution is decomposed into the first vector and the second vector , , , two nonlinear Sigmoid activation functions are used to process the first vector and the second vector respectively to fit the two-dimensional binomial distribution. In order to realize different cross-axis interaction features between the two parallel paths, the first vector calculated by the activation function and the second vector calculated by the activation function are aggregated by simple multiplication to obtain the coordinate weight, that is, the corresponding two axial attention maps in the two groups are aggregated by multiplication, and the sub-features are weighted by the coordinate weight. Reweight and get coordinate features , , where sigmoid represents the activation function calculation.

[0051] In some embodiments of the present invention, continue to refer to Figure 2 , the long-range input of the cross-spatial attention module is obtained by Using inter-group normalization, the short-range input is obtained by pairing the sub-features After convolution, a cross-spatial weighted fusion strategy is used to process long-range input and short-range input. Specifically, two-dimensional global average pooling (XY average pooling) is used to encode the coordinate features of the normalization between groups. , that is, encoding global spatial information, and then using SoftMax normalization to obtain the third transformation feature, the third transformation feature , the third transformation feature performs matrix dot product operation with the short-range input, that is, , get the short-range dependency information across space , , short-range dependency information across space It can be expressed as:

[0052] ;

[0053] Among them, softmax represents normalization, GN represents inter-group normalization, represents the sub-feature of the second convolution, i.e., the short-range input, represents the third transformation feature. It should be noted that the short-range dependency information across space Compared with cross-spatial long-range dependency information, it can collect spatial information of different scales.

[0054] Continue to refer Figure 2 , for the sub-features of the second convolution Perform two-dimensional global average pooling (XY average pooling) and SoftMax normalization in sequence to obtain the fourth transformation feature , where softmax represents normalization, the fourth transformation feature and the coordinate feature that has been normalized between groups Perform dot product operations to obtain long-range dependency information across space , , long-range dependency information across space The entire precise spatial location information can be retained. Aggregate short-range dependency information across space and long-range dependency information across space Get the second aggregation result , the Sigmoid activation function is calculated on the second aggregation result to obtain the spatial weight , use spatial weights to weight the original features to obtain spatial features , .

[0055] It should be noted that due to the lack of communication between the coordinate attention module and the cross-spatial attention module, low-dimensional features are repeated to some extent, which limits the ability of feature representation. Setting a cross-channel attention module is helpful to solve this problem, and the cross-channel attention module can perform adaptive weight adjustment according to the importance of each channel.

[0056] In some embodiments of the present invention, reference Figure 2 , the channel group input of the cross-channel attention module By using the coordinate feature After batch normalization, , the original input of the channel of the cross-channel attention module By using the original features After convolution, a cross-channel weighted fusion strategy is used to fused the original input and channel group input Processed.

[0057] Specifically, channel dimension pooling (C average pooling) is used to encode the information between groups, that is, channel dimension pooling (C average pooling) is used to input the channel group of the cross-channel attention module After processing, SoftMax normalization is used to transform the shape The first transformation feature , the first transformed feature and the original feature of the first convolution Perform a dot product operation, that is , get cross-channel adaptive information , , , where Conv represents the first convolution, softmax represents normalization, BN represents batch normalization, and c represents xx. It should be noted that cross-channel adaptive information The weight will be adjusted adaptively according to the importance of each channel information.

[0058] Continue to refer Figure 2, the original features of the first convolution Channel dimension pooling (C average pooling) and SoftMax normalization are performed in sequence to obtain the second transformation feature , the second transformation feature Coordinate features with batch normalization Perform dot product operation to obtain cross-channel group information , , where c=i, i=0, 1, ..., C-1, c represents the feature of the i-th channel; cross-channel group information The information between different groups can be integrated to enrich the feature representation.

[0059] Aggregate cross-channel adaptation information and cross-channel group information to obtain a first aggregation result , for the first aggregation result Perform Sigmoid activation function calculation to obtain channel weights , using channel weights Weight the original feature X to obtain the channel feature , , fusion channel features and spatial characteristics To obtain the target feature ( + ).

[0060] The above-mentioned convolutional attention method uses a parallel structure to stack coordinate attention, spatial attention and channel attention, which improves computational efficiency while better integrating features of different dimensions. The cross-channel attention module adopts a cross-channel weighted fusion strategy to better aggregate grouping and information between channels to achieve richer feature expression. Using this convolutional attention method for remote sensing image registration can provide richer feature representation for remote sensing images in sparse and repeated texture areas, which is beneficial to improving the registration effect.

[0061] The reasons why the embodiment of the present invention improves the computational efficiency are: first, the embodiment of the present invention adopts a feature grouping method to reduce the amount of calculation; second, a parallel structure is used to fuse multiple attention features instead of a serial manner, so the computational efficiency is higher.

[0062] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0063] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote sensing image registration method based on convolutional attention, characterized in that: include: Inputting the original remote sensing image pair into the convolution layer of the modality adapter to obtain the original features, and processing the original features using the convolution attention method to obtain the target features; Input the target feature as a modal adaptation feature into a feature encoder to obtain a feature descriptor; input the feature descriptor into a feature detector to use conditional probability estimation to globally detect feature points; perform preliminary matching of the feature descriptor and the feature points using a FLANN algorithm, and remove outliers in combination with a RANSAC algorithm to obtain a registration result; The convolutional attention method is applied to the modality adapter and the feature encoder, and the convolutional attention method includes: inputting the original features into a coordinate attention module, the coordinate attention module groups the original features to obtain sub-features, generates coordinate weights based on the sub-features, and weights the sub-features using the coordinate weights to output coordinate features; Inputting the coordinate feature and the original feature into a cross-channel attention module, wherein the cross-channel attention module generates a channel weight based on the coordinate feature and the original feature; The original features are weighted by using the channel weights to obtain channel features; Inputting the coordinate feature and the sub-feature into a cross-spatial attention module, wherein the cross-spatial attention module generates a spatial weight based on the coordinate feature and the sub-feature; Using the spatial weight to weight the original feature to obtain a spatial feature; The channel feature and the spatial feature are fused to obtain the target feature.

2. The remote sensing image registration method based on convolutional attention according to claim 1, characterized in that: Inputting the coordinate feature and the original feature into the cross-channel attention module includes: inputting the coordinate feature and the original feature of the first convolution into the cross-channel attention module; inputting the coordinate feature and the sub-feature into the cross-spatial attention module includes: inputting the coordinate feature and the sub-feature of the second convolution into the cross-spatial attention module.

3. The remote sensing image registration method based on convolutional attention according to claim 2, characterized in that: The first convolution is a 3×3 convolution, and the second convolution is a 3×3 convolution.

4. The remote sensing image registration method based on convolutional attention according to claim 1, characterized in that: Generating coordinate weights based on the sub-features includes: performing axial compression on the sub-features along the horizontal direction to obtain horizontal compression features, performing axial compression on the sub-features along the vertical direction to obtain vertical compression features, concatenating and convolving the horizontal compression features and the vertical compression features, decomposing the output of the convolution into a first vector and a second vector, performing activation function calculation on the first vector, and performing activation function calculation on the second vector, and multiplying the first vector calculated by the activation function and the second vector calculated by the activation function to obtain the coordinate weights.

5. The remote sensing image registration method based on convolutional attention according to claim 4, characterized in that: The convolution is a 1×1 convolution.

6. The remote sensing image registration method based on convolutional attention according to claim 2, characterized in that: The cross-channel attention module generates channel weights based on the coordinate features and the original features, including: The coordinate features are sequentially subjected to batch normalization, channel dimension pooling, and SoftMax normalization to obtain a first transformed feature, and the first transformed feature is subjected to a dot product operation with the original feature of the first convolution to obtain cross-channel adaptive information; The original features of the first convolution are sequentially subjected to channel dimension pooling and SoftMax normalization to obtain second transformed features, and the second transformed features are subjected to dot product operation with the coordinate features subjected to batch normalization to obtain cross-channel group information; Aggregating the cross-channel adaptive information and the cross-channel group information to obtain a first aggregation result, and performing a Sigmoid activation function calculation on the first aggregation result to obtain the channel weight.

7. The remote sensing image registration method based on convolutional attention according to claim 2, characterized in that: The cross-spatial attention module generates a spatial weight based on the coordinate feature and the sub-feature, comprising: The coordinate features are sequentially subjected to inter-group normalization, two-dimensional global average pooling, and SoftMax normalization to obtain a third transformation feature, and the third transformation feature is subjected to a dot product operation with the sub-feature of the second convolution to obtain cross-space short-range dependency information; Performing two-dimensional global average pooling and SoftMax normalization on the sub-features of the second convolution in sequence to obtain a fourth transformed feature, and performing a dot product operation on the fourth transformed feature and the coordinate feature subjected to the inter-group normalization to obtain cross-space long-range dependency information; The cross-spatial short-range dependency information and the cross-spatial long-range dependency information are aggregated to obtain a second aggregation result, and a Sigmoid activation function is performed on the second aggregation result to obtain the spatial weight.

Images