Feature matching method and device maintaining strong robustness under complex underwater conditions

Abstract

The application provides a feature matching method and device with strong robustness under complex underwater conditions, comprising: constructing a teacher network and a student network with the same structure, both comprising a multi-scale feature extraction module, a low-frequency dominant coarse matching module and a high-frequency dominant fine matching module; the teacher network inputs an image pair spliced in channels by an RGB image and a normal map, and the student network only inputs a pair of RGB images; the student network trained extracts high and low resolution feature maps from the input images, performs coarse matching by using the low-frequency dominant module to obtain an initial matching point pair, performs fine matching by using the high-frequency dominant module on the high resolution feature map based on the initial matching point pair to predict an offset, and obtains a final matching result; and the student network is obtained through multi-level distillation training under the supervision of the teacher. The application enables the network to actively resist degradation from the frequency level, the student network can implicitly learn geometric consistency without a normal map, the inference overhead is low, and global anti-interference and local high-precision requirements are considered.

Description

Technical Field

[0001] This invention relates to the field of underwater detection and identification technology, and in particular to a feature matching method and apparatus that maintains strong robustness under complex underwater conditions. Background Technology

[0002] Feature matching of underwater images is a fundamental and crucial step in advanced visual tasks such as underwater visual localization, 3D reconstruction, target recognition, and tracking. Its goal is to establish a stable and accurate correspondence between two or more images with varying viewpoints and lighting conditions. However, underwater images are severely affected by various physical degradation factors, including selective absorption and scattering of light by water, backscattering noise caused by suspended particles, motion blur due to water flow, and uneven illumination from artificial light sources. These factors result in images exhibiting low contrast, blurred details, and complex noise, severely compromising the discriminative and repeatable nature of local features. Therefore, designing a feature matching algorithm that can maintain high accuracy and robustness even under degradation conditions is particularly important.

[0003] Currently, existing underwater image matching methods rely primarily on data augmentation strategies to improve model performance when faced with complex degradation such as underwater blur, noise, and low contrast. However, the essence of these methods is to teach the model a given degradation pattern, rather than fundamentally learning feature representations that are insensitive to degradation. This passive and indirect approach cannot guarantee stable performance under unseen real-world degradation and lacks inherent robustness to environmental degradation. Most existing image matching methods only use geometric information as a supervisory signal without explicitly incorporating it into the matching network itself, limiting the feature's ability to learn geometric consistency. Integrating geometric information at the feature level through additional network modules increases computational complexity. Furthermore, existing underwater image matching methods use structurally identical modules at different matching stages. Coarse matching requires a stable low-frequency global context to resist interference and establish robust assumptions, while fine matching requires keen high-frequency local details for accurate localization. This homogeneous design fails to achieve an optimal balance between resisting global interference and capturing local details, thus failing to optimize for the stage-specific needs of the matching task. Summary of the Invention

[0004] The purpose of this invention is to provide a feature matching method and apparatus that maintains strong robustness under complex underwater conditions. This addresses the problems of existing underwater image matching methods, which cannot guarantee stable performance under unseen real degradation, lack inherent robustness to environmental degradation, limit the learning ability of features to geometric consistency, increase computational complexity, and fail to optimize for the phased requirements of the matching task.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a feature matching method that maintains strong robustness under complex underwater conditions, the method comprising: Construct teacher and student networks with identical structures. Both teacher and student networks include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input of the teacher network is a pair of images, each image being formed by connecting an RGB image and its corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. The first RGB image to be matched and the second RGB image to be matched are input into the trained student network. The multi-scale feature extraction module extracts the first low-resolution feature map, the first high-resolution feature map, the second low-resolution feature map and the second high-resolution feature map respectively. The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using the teacher network as supervision and using a multi-level distillation loss function. The low-frequency dominant coarse matching module is used to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map to obtain initial matching point pairs. Based on the initial matching point pairs, local windows at corresponding positions are determined on the first high-resolution feature map and the second high-resolution feature map. The high-frequency dominant fine matching module is used to perform fine matching on the features within the local window to predict the offset and obtain the final matching result.

[0006] A second aspect of the present invention provides a feature matching device that maintains strong robustness under complex underwater conditions, the device comprising: The building module is used to construct teacher and student networks with identical structures. Both teacher and student networks include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input of the teacher network is a pair of images, each image being formed by concatenating an RGB image and its corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. The extraction module is used to input the first RGB image to be matched and the second RGB image to be matched into the trained student network, so as to extract the first low-resolution feature map, the first high-resolution feature map, the second low-resolution feature map and the second high-resolution feature map respectively using the multi-scale feature extraction module. The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using the teacher network as supervision and using a multi-level distillation loss function. The coarse matching module is used to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map by utilizing the low-frequency-dominated coarse matching module to obtain initial matching point pairs; The fine matching module is used to determine the local window at the corresponding position on the first high-resolution feature map and the second high-resolution feature map based on the initial matching point pair, and to use the high-frequency dominant fine matching module to perform fine matching on the features within the local window to predict the offset and obtain the final matching result.

[0007] Compared to existing technologies, the feature matching method and apparatus provided by this invention maintains strong robustness under complex underwater conditions. It constructs a teacher network and a student network with identical structures. Both the teacher network and the student network include a multi-scale feature extraction module, a low-frequency-dominated coarse matching module, and a high-frequency-dominated fine matching module. The input to the teacher network is a pair of images, each image consisting of an RGB image and its corresponding normal map concatenated along the channel dimension. The input to the student network is a pair of RGB images. The first RGB image to be matched and the second RGB image to be matched are input into the trained student network to extract a first low-resolution feature map, a first high-resolution feature map, a second low-resolution feature map, and a second high-resolution feature map using the multi-scale feature extraction module. A high-resolution feature map is generated, with the first low-resolution feature map having a lower resolution than the first high-resolution feature map, and the second low-resolution feature map having a lower resolution than the second high-resolution feature map. The trained student network is trained under the supervision of the teacher network and through knowledge distillation using a multi-level distillation loss function. A low-frequency-dominated coarse matching module performs coarse matching on the first and second low-resolution feature maps to obtain initial matching point pairs. Based on these initial matching point pairs, local windows are determined at corresponding positions on the first and second high-resolution feature maps. A high-frequency-dominated fine matching module then performs fine matching on the features within these local windows to predict the offset, resulting in the final matching result. This matching architecture, employing both low-frequency-dominated coarse matching and high-frequency-dominated fine matching modules, actively resists degradation interference such as underwater blurring and noise at the feature frequency level. This enables the network to extract stable information from degraded features without relying on specific synthetic data or enhancement strategies, resulting in more fundamental robustness and stronger generalization ability. The teacher network uses RGB images and normal maps connected in the channel dimension as geometric prior inputs, while the student network uses only RGB images as input. Knowledge distillation training is performed through a multi-level distillation loss function. The student network implicitly learns geometric consistency constraints without requiring normal map input, resulting in low computational overhead during inference. In the coarse matching stage, a low-frequency-dominated coarse matching module extracts stable global structural information from low-resolution feature maps. In the fine matching stage, a high-frequency-dominated fine matching module captures local details from high-resolution feature maps. This allows the algorithm to perform its specific function at each matching stage, accurately responding to the differentiated requirements of matching tasks for global robustness and local high accuracy, and enabling optimization to meet the stage-specific needs of the matching task. Attached Figure Description

[0008] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein: Figure 1A flowchart illustrating a feature matching method that maintains strong robustness under complex underwater conditions is shown schematically. Figure 2 The algorithmic framework of a feature matching method that maintains strong robustness under complex underwater conditions is illustrated schematically. Figure 3 A schematic diagram of a feature matching device that maintains strong robustness under complex underwater conditions is shown. Detailed Implementation

[0009] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0010] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0011] The methods in the embodiments of the present invention will be described in detail below.

[0012] Figure 1 The flowchart of a feature matching method that maintains strong robustness under complex underwater conditions is illustrated schematically. Figure 2 The schematic diagram illustrates the algorithmic framework of a feature matching method that maintains strong robustness under complex underwater conditions. (See attached image.) Figure 1 and Figure 2 As shown, the method may include: S101. Construct teacher and student networks with identical structures.

[0013] Both the teacher network and the student network include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input to the teacher network is a pair of images, each image being formed by connecting an RGB image and its corresponding normal map in the channel dimension. The input to the student network is a pair of RGB images.

[0014] Before constructing teacher and student networks with identical structures, the method also includes: Step A1: Obtain the depth map corresponding to the RGB image.

[0015] Step A2: Based on the depth map, calculate the gradient in the horizontal direction and the gradient in the vertical direction for each pixel to obtain the horizontal gradient component and the vertical gradient component.

[0016] Step A3: Combine the horizontal gradient components, vertical gradient components, and reference vector to form a three-dimensional vector, which is used to represent the depth direction.

[0017] Step A4: Normalize the 3D vectors to obtain normal vectors of unit length, and construct a normal map based on the normal vectors of all pixels.

[0018] Normal information is derived from gradient differences calculated based on depth maps, which are widely available in commonly used natural image datasets. First, the gradient of each pixel in the horizontal and vertical directions is calculated based on the depth map. Then, these two gradient components are combined with a reference vector representing the depth direction to form a three-dimensional vector characterizing the initial orientation of the surface at that point. Finally, this vector is normalized to a unit length to obtain the normal direction information.

[0019] Normal diagram Based on depth map It is derived by calculating the gradient difference at each pixel. At this point, the normal vector of the normal diagram The following formula describes the geometric orientation of a surface: ; in, For pixels The normal vector at that point, For pixels The depth value corresponding to the depth map. The horizontal coordinates of the pixel. The vertical coordinates of the pixel. For depth map The gradient in the horizontal direction, For depth map The gradient in the vertical direction, Let L2 norm be the vector. This is the transpose of the matrix.

[0020] The teacher network and the student network constitute the knowledge distillation framework. The teacher network and the student network have basically the same structure. The difference is that in addition to the RGB optical image, the teacher network also has a corresponding normal map as input. The normal map is connected to the RGB image along the channel dimension.

[0021] S102. Input the first RGB image to be matched and the second RGB image to be matched into the trained student network, so as to extract the first low-resolution feature map, the first high-resolution feature map, the second low-resolution feature map and the second high-resolution feature map respectively using the multi-scale feature extraction module.

[0022] The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using a multi-level distillation loss function with the teacher network as supervision.

[0023] The multi-scale feature extraction module includes a residual network and a feature pyramid network.

[0024] The student network loss includes a matching supervision loss and a multi-level distillation loss function. The teacher network uses a matching supervision loss during training, which is a weighted sum of coarse matching loss and fine matching loss. The multi-level distillation loss function consists of three levels: attention distillation loss, coarse distillation loss, and fine distillation loss.

[0025] Specifically, the training process for a well-trained student network includes: Step B1: Input the paired RGB images and corresponding normal maps from the paired images into the teacher network to generate coarse matching pseudo-labels and fine matching pseudo-labels through the multi-scale feature extraction module, the low-frequency dominant coarse matching module, and the high-frequency dominant fine matching module.

[0026] Step B2: Input paired RGB images into the student network to generate coarse matching prediction results and fine matching prediction results through a multi-scale feature extraction module, a low-frequency-dominated coarse matching module, and a high-frequency-dominated fine matching module. Step B3: Using the teacher network as supervision, the student network loss with a multi-level distillation loss function is used to align the coarse matching prediction results with the coarse matching pseudo-labels, and the fine matching prediction results with the fine matching pseudo-labels. The trained student network is obtained through knowledge distillation training.

[0027] The teacher network receives an RGB image to be matched, along with the corresponding normal map calculated using gradient differencing from the depth map. The normal map serves as geometric guidance information and is used only during the training phase of this branch of the teacher network. The coarse matching phase of the teacher network yields a similarity matrix, which serves as a coarse pseudo-label; the fine matching phase yields the expectation of the heatmap, which serves as a fine pseudo-label. The student network receives only an RGB image to be matched and outputs the matching result. No normal map input is required during training and all deployment inference phases. Geometric consistency constraints are implicitly learned by aligning the pseudo-labels output by the teacher network branch in the coarse and fine matching phases using a multi-level distillation loss function.

[0028] The coarse matching loss is supervised by a negative log-likelihood function using a confidence matrix built with dual softmax operators, while the fine matching loss is supervised by the L2 loss function of local similarity responses. The attention distillation loss calculates the L2 distance between the teacher's and student's attention distributions to align the attention weight maps computed by the teacher and student networks in the low-frequency dominant coarse matching module (i.e., the low-frequency dominant Transformer module). The coarse distillation loss aligns the coarse prediction results and coarse pseudo-labels based on KL divergence, while the fine distillation loss aligns the fine prediction results and fine pseudo-labels using the L2 norm.

[0029] Specifically, the teacher network loss only includes the matching supervision loss, which is a weighted average of the coarse matching loss and the fine matching loss, and its expression is: ; in, For the loss of teachers' network access, To match the supervised loss, These are the weighting coefficients for the coarse matching loss. For coarse matching loss, For fine-grained matching loss.

[0030] The student network loss consists of two parts: the matching supervision loss and the multi-level distillation loss function. In other words, it includes coarse matching loss, fine matching loss, and multi-level distillation loss functions. The expression for the student model loss is: ; in, For student model loss, For coarse matching loss, For fine-matching loss For multi-stage distillation loss function, These are the weighting coefficients of the multi-stage distillation loss function. These are the weighting coefficients for the coarse matching loss. .

[0031] Matching supervised loss The coarse matching loss is supervised by the negative log-likelihood function using a confidence matrix built with double softmax operators, while the fine matching loss is supervised by the L2 loss function of local similarity responses. The expression for the matching supervision loss is: ; in, To match the supervised loss, For coarse matching loss, For fine-matching loss, For the true coarse matching set, For the index of the matching point pair, This serves as the identifier for the matching point pairs in the coarse matching phase. In the confidence matrix established using the double softmax operator for the coarse matching module, the matching pairs The matching confidence probability. For a fine-grained matching set, The model predicts the matching position. For the actual matching location, This is the inverse variance of the locally similar responses.

[0032] The coarse matching module performs coarse matching matrix matching at coarse resolution. With the true coarse matching set Optimization is performed. The fine-grained matching module will optimize based on the inverse variance of local similar responses. For fine matching sets Subpixel differences are penalized, and the penalty strength is proportional to the inverse variance.

[0033] Multistage distillation loss function It is divided into three levels: attention distillation loss, coarse distillation loss, and fine distillation loss. Multi-level distillation loss function. The expression is: ; in, For multi-stage distillation loss function, For the loss during rough distillation, For losses during fine distillation, This is due to the loss of attention distillation.

[0034] Attention distillation loss calculates the L2 distance between the teacher's attention distribution and the student's attention distribution to align the attention weight maps computed by the teacher and student models in the low-frequency-dominated Transformer module. Coarse distillation loss aligns coarse predictions and coarse pseudo-labels based on KL divergence, while fine distillation loss aligns fine predictions and fine pseudo-labels using L2 norm.

[0035] and These are the probability distributions generated by the teacher model and the student model, respectively.

[0036] ; ; ; in, For the loss during rough distillation, For losses during fine distillation, For attention distillation loss, For temperature parameters, For the teacher network in the first The probability distribution values ​​generated at each location For student networks in the first The probability distribution values ​​generated at each location For a fine-grained matching set, For the teacher network in the first The variance of the predictions at each matching point pair. For the teacher network in the first The fine-grained matching results predicted at each matching point pair. For student networks in the first The fine-grained matching results predicted at each matching point pair. This is the attention weight matrix calculated by the teacher network in the low-frequency dominant Transformer module. This is the attention weight matrix corresponding to the student model. It is the square of the Frobenius norm.

[0037] Multi-scale feature extraction uses a convolutional neural network (ResNetFPN) to extract multi-scale feature maps from the input image pair, obtaining a coarse feature map at 1 / 8 scale and a fine feature map at 1 / 2 scale of the original image. The multi-scale feature extraction module is itself a convolutional neural network, with inputs consisting of RGB or (RGB, normal map) connections, and outputs coarse and fine feature maps. The multi-scale feature extraction module uses ResNet+FPN (Feature Pyramid Network) as the backbone network for feature extraction.

[0038] Specifically, the first RGB image to be matched and the second RGB image to be matched are input into the trained student network, so that the multi-scale feature extraction module can extract the first low-resolution feature map, the first high-resolution feature map, the second low-resolution feature map, and the second high-resolution feature map, respectively, including: Step C1: Perform convolution, normalization and non-linear activation processing on the first RGB image to be matched in sequence, and use multiple residual network layers in the residual network to downsample the first result after non-linear activation processing step by step to obtain the first shallow feature map, the first middle feature map and the first deep feature map with progressively decreasing resolution.

[0039] For example, the first RGB image to be matched is taken as input. The input first passes through a 7×7 convolution, followed by batch normalization (BatchNorm) and the ReLU activation function; then it passes through three ResNet layers to progressively extract feature maps at 1 / 2, 1 / 4, and 1 / 8 resolution. The ResNet layers use the residual structure of ResNet-18.

[0040] Step C2: Upsample the first deep feature map through the top-down path of the feature pyramid network, and then perform lateral connection and element-wise fusion with the first middle-layer feature map and the first shallow feature map to obtain the first low-resolution feature map and the first high-resolution feature map.

[0041] For example, through the top-down path of the feature pyramid network, high-level features at 1 / 8 resolution are sequentially upsampled and laterally connected and element-wise fused with low-level features at 1 / 4 and 1 / 2 resolutions. The final output size of this module is the original... Figure 1 / 8 size coarse-level feature map and original Figure 1 Fine-level feature map of size 2.

[0042] Step C3: Perform convolution, normalization and non-linear activation processing on the second RGB image to be matched in sequence, and use multiple residual network layers in the residual network to downsample the second result after non-linear activation processing step by step to obtain the second shallow feature map, the second middle feature map and the second deep feature map with progressively decreasing resolution.

[0043] Step C4: Upsample the second deep feature map through the top-down path of the feature pyramid network, and then perform lateral connection and element-wise fusion with the second middle-layer feature map and the second shallow feature map to obtain the second low-resolution feature map and the second high-resolution feature map.

[0044] The specific operations for matching the second RGB image in steps C3 and C4 are the same as the example operations for matching the first RGB image in steps C1 and C2.

[0045] S103. Using the low-frequency dominant coarse matching module, coarse matching is performed on the first low-resolution feature map and the second low-resolution feature map to obtain the initial matching point pair.

[0046] The coarse matching stage is as follows: First, an inseparable lifting wavelet transform is performed on the coarse feature map. The low-frequency components obtained from the decomposition are used as the key vectors for attention. Four alternating calculations of self-attention and cross-attention are performed to form a low-frequency-dominated Transformer module. The resulting feature representation is then fed into a differentiable matching layer to calculate the matching probability matrix and obtain coarse initial matching pairs. The low-frequency-dominated coarse matching module takes the coarse-level feature map as input and outputs the encoded coarse-level features and coarse matching point pairs. Its overall structure consists of a low-frequency-dominated Transformer encoder module to obtain the encoded coarse-level features, followed by a differentiable matching layer to obtain coarse matching point pairs.

[0047] Specifically, the low-frequency-dominated coarse matching module is used to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map to obtain initial matching point pairs, including: Step D1: Perform wavelet transform on the first low-resolution feature map and the second low-resolution feature map respectively to separate the preset low-frequency sub-band and the preset high-frequency sub-band.

[0048] Perform an inseparable lifting wavelet transform on the coarse feature maps (i.e., the first low-resolution feature map and the second low-resolution feature map) to obtain the preset high-frequency subbands HH, HL, LH and the preset low-frequency subband LL.

[0049] Step D2: Using the preset low-frequency sub-bands of the first low-resolution feature map and the preset low-frequency sub-bands of the second low-resolution feature map as the key vector and value vector of the attention mechanism, respectively, self-attention and cross-attention calculations are performed alternately on the first low-resolution feature map and the second low-resolution feature map to obtain the low-frequency enhanced first low-resolution feature map and the low-frequency enhanced second low-resolution feature map.

[0050] For example, in the low-frequency-dominated Transformer encoder module, the decomposed low-frequency subbands (LL) are flattened into a sequence, and low-frequency key vectors and low-frequency value vectors are generated through a key-value linear transformation layer. The scaled dot product attention between the query vector and the low-frequency key vector is calculated to obtain the low-frequency attention output. The low-frequency attention features are residually concatenated with the input features and then normalized. The normalized features are then input into an MLP feedforward network to obtain feedforward features. The feedforward features are residually concatenated with the normalized features and then normalized again to obtain the final output of the Transformer encoder module. This alternating calculation of self-attention and cross-attention occurs four times throughout the module, resulting in a first low-resolution feature map and a second low-resolution feature map with low-frequency enhancement.

[0051] Step D3: Input the first low-resolution feature map and the second low-resolution feature map enhanced by low frequency into the differentiable matching layer, calculate the matching probability matrix, and select matching pairs with a confidence threshold based on the matching probability matrix to determine the initial matching point pairs.

[0052] The first low-resolution feature map and the second low-resolution feature map enhanced with low frequency are input into the differentiable matching layer to calculate the score matrix. Then, a double Softmax operator is applied in two dimensions to obtain the matching probability matrix. Matches above the confidence threshold are selected as coarse matching pairs, i.e., the initial matching pairs. The MLP feedforward network consists of two layers of linear transformation and ReLU activation function for nonlinear feature mapping.

[0053] S104. Based on the initial matching point pair, determine the corresponding local window on the first high-resolution feature map and the second high-resolution feature map, and use the high-frequency dominant fine matching module to perform fine matching on the features within the local window to predict the offset and obtain the final matching result.

[0054] The fine-grained matching stage involves cropping a local feature window from the fine-grained feature map, centered on the location indicated by the coarse matching pair. An inseparable lifting wavelet transform is applied to this local window, concatenating the three high-frequency components along the channel dimension. A 3×3 convolution is then performed for fusion, followed by alternating self-attention and cross-attention calculations to form a high-frequency-dominated Transformer module. The resulting feature representation is then used by a regression module to predict the offset from the coarse location to the true location, yielding the fine-grained matching result. The high-frequency-dominated fine-grained matching module takes the fine-grained feature map as input and outputs the encoded fine-grained features and the final matching point pair. Its overall structure consists of a high-frequency-dominated Transformer encoder module to obtain the encoded fine-grained features, followed by correlation calculation to obtain the final matching point pair, i.e., the final matching result.

[0055] Specifically, based on the initial matching point pairs, local windows at corresponding locations are determined on the first and second high-resolution feature maps, including: For each pair of initial matching points, a first local window and a second local window are cropped from the first high-resolution feature map and the second high-resolution feature map, respectively, with the position indicated by the initial matching point pair as the center.

[0056] For example, for each coarse-grained matching point pair, its position is determined on the fine-grained feature map, and then a local window of size 5×5 pixels centered on it is cropped and fed into the high-frequency dominant Transformer encoder module.

[0057] Specifically, the high-frequency dominant fine-grained matching module is used to perform fine-grained matching on features within a local window to predict the offset, and the final matching result is obtained, including: Step E1: Perform wavelet transform on the features in the first local window and the features in the second local window respectively to separate the target high-frequency sub-band and the target low-frequency sub-band.

[0058] For example, an indivisible lifting wavelet transform is performed on the fine feature map to obtain the target high-frequency subbands HH, HL, LH and the target low-frequency subband LL.

[0059] Step E2: Concatenate multiple target high-frequency subbands of the first local window along the channel dimension and fuse them through a convolutional layer to obtain the first fused high-frequency feature; concatenate multiple target high-frequency subbands of the second local window along the channel dimension and fuse them through a convolutional layer to obtain the second fused high-frequency feature.

[0060] For example, in a high-frequency dominant Transformer encoder module, the three high-frequency subbands HH, HL, and LH are concatenated along the channel dimension and fused through a 3×3 convolutional layer to obtain fused high-frequency features.

[0061] Step E3: Using the first fused high-frequency feature and the second fused high-frequency feature as the key vector and value vector of the attention mechanism, respectively, self-attention and cross-attention calculations are performed alternately on the features in the first local window and the features in the second local window to obtain the first local feature and the second local feature with high frequency enhancement.

[0062] For example, the fused high-frequency features are flattened into a sequence, and a key-value linear transformation layer is used to generate high-frequency key vectors and high-frequency value vectors. The scaled dot product attention between the query vector and the high-frequency key vector is calculated to obtain the high-frequency attention output. The high-frequency attention features are residually concatenated with the input features and then normalized. The normalized features are then input into an MLP feedforward network to obtain feedforward features. The feedforward features are residually concatenated with the normalized features and then normalized again to obtain the final output of the Transformer encoder module. This alternating calculation of self-attention and cross-attention occurs once throughout the module.

[0063] Step E4: Calculate the correlation between the first local feature and the second local feature of high-frequency enhancement, generate a heatmap based on the correlation, and obtain the offset of the initial matching point pair by calculating the expected probability distribution of the heatmap. The final matching result is obtained based on the initial matching point pair and the offset.

[0064] For example, the correlation of feature vectors corresponding to all pixels within a local window is calculated, a heatmap is generated based on the correlation to represent the matching probability of each pixel, and the final matching point pair, i.e. the final matching result, is obtained by calculating the probability distribution.

[0065] This invention uses dense and continuous normal maps to introduce geometric information, providing stable geometric constraints for feature matching. The normal map and the original image are concatenated at the input of the feature encoder to form a composite input that integrates geometric and appearance information. This method of introducing the joint input is simple and efficient, requiring no additional complex fusion modules and resulting in low computational overhead. Within the matching network, this invention designs a low-frequency-dominated coarse matching module and a high-frequency-dominated fine matching module to address the different needs of coarse and fine matching. A frequency-division processing matching architecture is proposed. The low-frequency-dominated coarse matching module specifically enhances the extraction of global structural information that is insensitive to degradation, while the high-frequency-dominated fine matching module recovers sub-pixel accuracy in usable detail regions, fundamentally enhancing the robustness of feature representation to underwater degradation. This invention uses a multi-level distillation loss function, designing targeted distillation strategies in the attention module, coarse matching stage, and fine matching stage, enabling comprehensive transfer of the geometric perception capabilities learned by the teacher model from normal guidance. This invention trains on general natural images, eliminating the need for specially acquired or synthesized underwater image pairs and corresponding real-world matching labels. This overcomes the limitations of passive data augmentation and demonstrates strong generalization ability. Feature matching in underwater environments faces more severe imaging degradation challenges compared to air environments. This invention introduces normal maps as strong geometric guidance during teacher network training and designs a frequency-division matching strategy. In the coarse matching and fine matching stages, low-frequency and high-frequency dominant Transformer modules are used for feature transformation, respectively, thus adapting to the complex underwater environment.

[0066] Based on the above Figure 1As can be seen from the implementation method, the embodiments of the present invention construct teacher networks and student networks with identical structures. Both teacher networks and student networks include a multi-scale feature extraction module, a low-frequency-dominated coarse matching module, and a high-frequency-dominated fine matching module. The input of the teacher network is a pair of images, each image being formed by concatenating an RGB image and its corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. The first RGB image to be matched and the second RGB image to be matched are input into the trained student network to extract a first low-resolution feature map, a first high-resolution feature map, a second low-resolution feature map, and a second high-resolution feature map, respectively, using the multi-scale feature extraction module. The resolution of the first high-resolution feature map is lower than that of the second low-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained under the supervision of the teacher network and through knowledge distillation using a multi-level distillation loss function. A low-frequency-dominated coarse matching module is used to coarsely match the first and second low-resolution feature maps to obtain initial matching point pairs. Based on the initial matching point pairs, local windows are determined at corresponding positions on the first and second high-resolution feature maps. A high-frequency-dominated fine matching module is then used to finely match the features within the local windows to predict the offset, resulting in the final matching result. Thus, by designing a matching architecture with low-frequency-dominated coarse matching and high-frequency-dominated fine matching modules, the network actively resists degradation interference such as underwater blurring and noise at the feature frequency level. This enables the network to extract stable information from degraded features without relying on specific synthetic data or enhancement strategies, resulting in more fundamental robustness and stronger generalization ability. The teacher network uses RGB images and normal maps connected in the channel dimension as geometric prior inputs, while the student network uses only RGB images as input. Knowledge distillation training is performed through a multi-level distillation loss function. The student network implicitly learns geometric consistency constraints without requiring normal map input, resulting in low computational overhead during inference. In the coarse matching stage, a low-frequency-dominated coarse matching module extracts stable global structural information from low-resolution feature maps. In the fine matching stage, a high-frequency-dominated fine matching module captures local details from high-resolution feature maps. This allows the algorithm to perform its specific function at each matching stage, accurately responding to the differentiated requirements of matching tasks for global robustness and local high accuracy, and enabling optimization to meet the stage-specific needs of the matching task.

[0067] Based on the same inventive concept, as an implementation of the above-mentioned feature matching method that maintains strong robustness under complex underwater conditions, this embodiment of the invention also provides a feature matching device that maintains strong robustness under complex underwater conditions. Figure 3 This is a structural diagram of the feature matching device that maintains strong robustness under complex underwater conditions in an embodiment of the present invention. See also... Figure 3 As shown, the device may include: Module 301 is used to construct teacher networks and student networks with the same structure. Both teacher networks and student networks include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input of the teacher network is a pair of images, each image is formed by connecting an RGB image and the corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. The extraction module 302 is used to input the first RGB image to be matched and the second RGB image to be matched into the trained student network, so as to extract the first low-resolution feature map, the first high-resolution feature map, the second low-resolution feature map and the second high-resolution feature map respectively using the multi-scale feature extraction module. The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using the teacher network as supervision and using the multi-level distillation loss function. The coarse matching module 303 is used to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map by utilizing the low-frequency-dominated coarse matching module to obtain an initial matching point pair; The fine matching module 304 is used to determine the local window at the corresponding position on the first high-resolution feature map and the second high-resolution feature map based on the initial matching point pair, and to use the high-frequency dominant fine matching module to perform fine matching on the features within the local window to predict the offset and obtain the final matching result.

[0068] The device also includes: a normal map construction module, used to obtain the depth map corresponding to the RGB image before constructing teacher and student networks with the same structure; based on the depth map, calculate the gradient in the horizontal direction and the gradient in the vertical direction for each pixel to obtain the horizontal gradient component and the vertical gradient component; combine the horizontal gradient component and the vertical gradient component with the reference vector to form a three-dimensional vector, the reference vector being used to represent the depth direction; normalize the three-dimensional vector to obtain the normal vector of unit length, and construct the normal map based on the normal vectors of all pixels.

[0069] It should be noted that the above description of the feature matching device embodiment for maintaining strong robustness under complex underwater conditions is similar to the description of the feature matching method embodiment for maintaining strong robustness under complex underwater conditions, and has similar beneficial effects. For technical details not disclosed in the embodiments of the feature matching device for maintaining strong robustness under complex underwater conditions of the present invention, please refer to the description of the feature matching method embodiment for maintaining strong robustness under complex underwater conditions of the present invention for understanding.

[0070] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A feature matching method that maintains strong robustness under complex underwater conditions, characterized in that, include: Construct teacher and student networks with identical structures. Both teacher and student networks include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input of the teacher network is a pair of images, each image being formed by connecting an RGB image and its corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. The first RGB image to be matched and the second RGB image to be matched are input into the trained student network, so that the multi-scale feature extraction module can extract a first low-resolution feature map, a first high-resolution feature map, a second low-resolution feature map and a second high-resolution feature map respectively. The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using the teacher network as supervision and using a multi-level distillation loss function. Using the low-frequency dominant coarse matching module, coarse matching is performed on the first low-resolution feature map and the second low-resolution feature map to obtain initial matching point pairs; Based on the initial matching point pair, local windows at corresponding positions are determined on the first high-resolution feature map and the second high-resolution feature map. The high-frequency dominant fine matching module is used to perform fine matching on the features within the local window to predict the offset and obtain the final matching result.

2. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, Before constructing teacher and student networks with identical structures, the method further includes: Obtain the depth map corresponding to the RGB image; Based on the depth map, the gradient in the horizontal direction and the gradient in the vertical direction of each pixel are calculated to obtain the horizontal gradient component and the vertical gradient component. The horizontal gradient component, the vertical gradient component, and the reference vector are combined to form a three-dimensional vector, which represents the depth direction. The three-dimensional vector is normalized to obtain a normal vector of unit length, and the normal map is constructed based on the normal vectors of all pixels.

3. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, The multi-scale feature extraction module includes a residual network and a feature pyramid network. The step of inputting the first RGB image to be matched and the second RGB image to be matched into the trained student network, and using the multi-scale feature extraction module to extract a first low-resolution feature map, a first high-resolution feature map, a second low-resolution feature map, and a second high-resolution feature map, respectively, includes: The first RGB image to be matched is sequentially subjected to convolution, normalization and nonlinear activation processing, and multiple residual network layers in the residual network are used to downsample the first result after nonlinear activation processing step by step to obtain the first shallow feature map, the first middle feature map and the first deep feature map with progressively decreasing resolution. The first deep feature map is upsampled through the top-down path of the feature pyramid network, and the first upsampled result is horizontally connected and fused element by element with the first middle feature map and the first shallow feature map to obtain the first low-resolution feature map and the first high-resolution feature map. The second RGB image to be matched is sequentially subjected to convolution, normalization and nonlinear activation processing, and multiple residual network layers in the residual network are used to downsample the second result after nonlinear activation processing step by step to obtain the second shallow feature map, the second middle feature map and the second deep feature map with progressively decreasing resolution. The second deep feature map is upsampled through the top-down path of the feature pyramid network, and the upsampled second result is laterally connected and element-wise fused with the second middle-layer feature map and the second shallow feature map to obtain the second low-resolution feature map and the second high-resolution feature map.

4. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, The step of using the low-frequency dominant coarse matching module to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map to obtain initial matching point pairs includes: Wavelet transform is performed on the first low-resolution feature map and the second low-resolution feature map respectively to separate the preset low-frequency sub-band and the preset high-frequency sub-band; Using the preset low-frequency sub-bands of the first low-resolution feature map and the preset low-frequency sub-bands of the second low-resolution feature map as the key vector and value vector of the attention mechanism, the first low-resolution feature map and the second low-resolution feature map are alternately subjected to self-attention and cross-attention calculations to obtain the first low-frequency enhanced low-resolution feature map and the second low-frequency enhanced low-resolution feature map. The first low-resolution feature map and the second low-resolution feature map enhanced by low frequency are input into the differentiable matching layer to calculate the matching probability matrix. Based on the matching probability matrix, matching pairs with a confidence threshold are selected and determined as the initial matching point pairs.

5. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, The step of determining a local window at a corresponding position on the first high-resolution feature map and the second high-resolution feature map based on the initial matching point pair includes: For each pair of initial matching points, a first local window and a second local window are cropped on the first high-resolution feature map and the second high-resolution feature map, respectively, with the position indicated by the initial matching point pair as the center.

6. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 5, characterized in that, The step of using the high-frequency dominant fine-matching module to perform fine-matching on features within the local window to predict the offset and obtain the final matching result includes: Wavelet transform is performed on the features within the first local window and the features within the second local window to separate the target high-frequency sub-band and the target low-frequency sub-band. Multiple target high-frequency subbands of the first local window are stitched together along the channel dimension and fused through a convolutional layer to obtain the first fused high-frequency feature; multiple target high-frequency subbands of the second local window are stitched together along the channel dimension and fused through a convolutional layer to obtain the second fused high-frequency feature. Using the first fused high-frequency feature and the second fused high-frequency feature as the key vector and value vector of the attention mechanism, respectively, self-attention and cross-attention calculations are performed alternately on the features in the first local window and the features in the second local window to obtain the high-frequency enhanced first local feature and the high-frequency enhanced second local feature. The correlation between the first local feature and the second local feature of the high-frequency enhancement is calculated, a heatmap is generated based on the correlation, and the offset of the initial matching point pair is obtained by calculating the expected probability distribution of the heatmap. The final matching result is obtained based on the initial matching point pair and the offset.

7. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, The training process of the trained student network includes: The paired RGB images and their corresponding normal maps from the paired images are input into the teacher network to generate coarse matching pseudo-labels and fine matching pseudo-labels through the multi-scale feature extraction module, the low-frequency dominant coarse matching module, and the high-frequency dominant fine matching module. The paired RGB images are input into the student network to generate coarse matching prediction results and fine matching prediction results through the multi-scale feature extraction module, the low-frequency dominant coarse matching module and the high-frequency dominant fine matching module. Using the teacher network as supervision, and employing the student network loss with the multi-level distillation loss function, the coarse matching prediction results are aligned with the coarse matching pseudo-labels, and the fine matching prediction results are aligned with the fine matching pseudo-labels. The trained student network is obtained through knowledge distillation training.

8. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 7, characterized in that, The student network loss includes the matching supervision loss and the multi-level distillation loss function. The teacher network uses the matching supervision loss during training. The matching supervision loss is composed of a weighted sum of coarse matching loss and fine matching loss.

9. The feature matching method for maintaining strong robustness under complex underwater conditions according to claim 1, characterized in that, The multi-level distillation loss function includes attention distillation loss, coarse distillation loss, and fine distillation loss.

10. A feature matching device that maintains strong robustness under complex underwater conditions, characterized in that, include: The module is used to construct teacher networks and student networks with identical structures. Both teacher networks and student networks include a multi-scale feature extraction module, a low-frequency dominant coarse matching module, and a high-frequency dominant fine matching module. The input of the teacher network is a pair of images, each image being formed by connecting an RGB image and its corresponding normal map in the channel dimension. The input of the student network is a pair of RGB images. An extraction module is used to input the first RGB image to be matched and the second RGB image to be matched into the trained student network, so as to extract a first low-resolution feature map, a first high-resolution feature map, a second low-resolution feature map and a second high-resolution feature map respectively using the multi-scale feature extraction module. The resolution of the first low-resolution feature map is lower than that of the first high-resolution feature map, and the resolution of the second low-resolution feature map is lower than that of the second high-resolution feature map. The trained student network is trained by knowledge distillation using the teacher network as supervision and a multi-level distillation loss function. The coarse matching module is used to perform coarse matching on the first low-resolution feature map and the second low-resolution feature map using the low-frequency dominant coarse matching module to obtain an initial matching point pair; The fine matching module is used to determine local windows at corresponding positions on the first high-resolution feature map and the second high-resolution feature map based on the initial matching point pair, and to perform fine matching on the features within the local window using the high-frequency dominant fine matching module to predict the offset and obtain the final matching result.

Images