Feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion

Abstract

The present invention discloses a feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion, comprising the following steps: constructing a feature matching model; the feature matching model comprises a backbone network, a key point detection network and a descriptor generation network; inputting an image to be matched into the backbone network to generate a feature map and a reliability map; inputting the image to be matched into the key point detection network to generate a key point heat map, inputting the key point heat map, the reliability map and the feature map into the descriptor generation network to generate an enhanced descriptor of the image to be matched, and finally performing brute force matching on the enhanced descriptor of the image to be matched; a differential wavelet transform downsampling module in the backbone network of the present invention utilizes discrete wavelet transform to perform frequency domain decomposition on features, separates noise from key features, and combines a differential operation of calculating features and neighborhood average pooling differences to effectively remove redundant information, enhance key features and improve the accuracy of subsequent feature matching.

Description

Technical Field

[0001] The present invention relates to the technical field of feature matching, and in particular to a feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion. Background Art

[0002] Image matching is a crucial task in computer vision, often serving as the first step in many complex visual tasks. It is widely used in areas such as 3D reconstruction, motion estimation, image stitching, and object detection. The goal of image feature matching is to accurately match feature points in different images, typically including corners and edges, and to identify corresponding relationships between images. Image matching can be categorized into the following categories, depending on the imaging conditions: image matching based on different imaging times, image matching based on different viewing angles, and image matching based on templates.

[0003] Feature matching is an image matching method that extracts features (such as points and lines) from an image. By matching these features, it infers the spatial and structural relationships between the images. In many computer vision tasks requiring high precision, feature matching accuracy is crucial and impacts the performance of these tasks.

[0004] The development of deep learning has brought about tremendous breakthroughs in feature matching technology. Deep learning-based feature matching approaches leverage deep neural network architectures to learn the deep features of target objects and optimize them by integrating high-level semantic features. This approach overcomes the limitations of traditional feature extraction methods and offers superior matching accuracy and robustness. However, image matching becomes more difficult due to differences in source time, shooting angle, and acquisition sensor among the images to be matched. In these complex backgrounds and application scenarios, existing technologies are inadequate in distinguishing between noise and key features, resulting in the blurring of key features and reduced matching accuracy. Therefore, reducing mismatches caused by issues like changes in lighting and perspective is an urgent issue in feature matching that needs to be addressed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides a feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion, which aims to solve the problems in the background technology.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solution: a feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion, comprising the following steps: reading an image to be matched; constructing a feature matching model; the feature matching model includes a backbone network, a key point detection network and a descriptor generation network; the backbone network is composed of a feature extraction module and a sixth convolution block; the image to be matched is input into the feature extraction module in the backbone network to generate a feature map, and the feature map is passed through the sixth convolution block to generate a reliability map; the image to be matched is input into the key point detection network to generate a key point heat map, the key point heat map and the reliability map are input into the descriptor generation network to filter out key points, and finally, the feature descriptor of the corresponding image to be matched is extracted from the feature map according to the filtered key points, the feature descriptor of the image to be matched is further processed by an adaptive multi-scale descriptor enhancement module to obtain an enhanced descriptor of the image to be matched, and finally, brute force matching is performed on the enhanced descriptor of the image to be matched; two images to be matched are read, and finally, brute force matching is performed on the enhanced descriptors of the two images to be matched.

[0007] Furthermore, the feature extraction module includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block; the image to be matched passes through the first convolution block and the second convolution block respectively, the outputs of the first convolution block and the second convolution block are added to obtain an added feature, and the added feature passes through the third convolution block, the fourth convolution block, and the fifth convolution block in sequence to obtain a feature map F;

[0008] The first convolution block is composed of a first convolution layer, a first differential wavelet transform downsampling module, a first dynamic multi-scale feature fusion module, a second dynamic multi-scale feature fusion module, and a second differential wavelet transform downsampling module connected in sequence; the image to be matched is sequentially passed through the first convolution layer, the first differential wavelet transform downsampling module, the first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module, and the second differential wavelet transform downsampling module to obtain the output of the first convolution block;

[0009] The second convolution block is composed of a first average pooling layer and a second convolution layer connected in sequence; the image to be matched passes through the first average pooling layer and the second convolution layer in sequence to obtain the output of the second convolution block;

[0010] The third convolution block is composed of a second dynamic multi-scale feature fusion module and a third dynamic multi-scale feature fusion module connected in sequence; the output of the third convolution block is obtained after the added features pass through the second dynamic multi-scale feature fusion module and the third dynamic multi-scale feature fusion module in sequence;

[0011] The fourth convolution block is composed of a third differential wavelet transform downsampling module, a fourth dynamic multi-scale feature fusion module, and a fifth dynamic multi-scale feature fusion module connected in sequence; the output of the third convolution block is sequentially passed through the third differential wavelet transform downsampling module, the fourth dynamic multi-scale feature fusion module, and the fifth dynamic multi-scale feature fusion module to obtain the output of the fourth convolution block;

[0012] The fifth convolutional block is composed of the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer connected in sequence; the output of the fourth convolutional block is passed through the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer in sequence to obtain the output of the fifth convolutional block;

[0013] Among them, the sixth convolution block is composed of the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer connected in sequence; the feature map F passes through the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer in sequence and then passes through the sigmoid function to obtain the reliability map R.

[0014] Furthermore, the first differential wavelet transform downsampling module, the second differential wavelet transform downsampling module, the third differential wavelet transform downsampling module and the fourth differential wavelet transform downsampling module have the same structure; the first differential wavelet transform downsampling module includes a ninth convolution layer, a discrete wavelet transform module, a tenth convolution layer, an edge information enhancement module and an eleventh convolution layer;

[0015] The processing flow of the first differential wavelet transform downsampling module is as follows:

[0016] The output of the first convolutional layer passes through the ninth convolutional layer, and the output of the ninth convolutional layer is input into the discrete wavelet transform module for frequency domain decomposition to obtain low-frequency information and high-frequency information;

[0017] The high-frequency information is further decomposed into horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information;

[0018] The horizontal high-frequency information, vertical high-frequency information and low-frequency information are spliced ​​along the channel dimension to obtain a high-frequency information feature map. The high-frequency information feature map is passed through the tenth convolutional layer and the edge information enhancement module in sequence. The output of the edge information enhancement module is multiplied by the output of the tenth convolutional layer. The multiplied output is then passed through the eleventh convolutional layer to obtain the output of the first differential wavelet transform downsampling module.

[0019] Furthermore, the edge information enhancement module includes a second average pooling layer and a twelfth convolutional layer; the processing flow of the edge information enhancement module is: the output of the tenth convolutional layer passes through the second average pooling layer, and the output of the tenth convolutional layer is subtracted from the output of the second average pooling layer. The subtracted output is first adjusted through the twelfth convolutional layer, and then activated through the Sigmoid function to obtain a weight map, and the weight map is added by 1 to obtain the output of the edge information enhancement module.

[0020] Furthermore, the first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module, the third dynamic multi-scale feature fusion module, the fourth dynamic multi-scale feature fusion module and the fifth dynamic multi-scale feature fusion module have the same structure; the first dynamic multi-scale feature fusion module includes a first depth-separable convolution, a Split function, a second depth-separable convolution, a third depth-separable convolution, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer and a SiLU activation function.

[0021] Furthermore, the processing flow of the first dynamic multi-scale feature fusion module is as follows: the output of the first differential wavelet transform downsampling module passes through the first depth-separable convolution and the fourteenth convolution layer respectively, the output of the first depth-separable convolution is divided into the first group of features, the second group of features and the third group of features in proportion in the channel dimension through the Split function, the second group of features and the third group of features pass through the second depth-separable convolution and the third depth-separable convolution respectively, the output of the second depth-separable convolution and the third depth-separable convolution and the first group of features are spliced ​​in the channel dimension, the spliced ​​result is input into the thirteenth convolution layer, the outputs of the thirteenth convolution layer and the fourteenth convolution layer are multiplied after passing through a SiLU activation function, the multiplied result is passed through the fifteenth convolution layer to obtain the output of the first dynamic multi-scale feature fusion module.

[0022] Furthermore, the key point heat map K and the reliability map R are input into the descriptor generation network to screen out the key points. Finally, the feature descriptors of the corresponding image to be matched are extracted from the feature map F according to the screened key points. The specific process is as follows: the key point heat map K is non-maximum suppressed, and the local response peak points are selected as candidate key points; the response values ​​of the candidate key points are selected from the key point heat map K using nearest neighbor interpolation, and the feature responses of the candidate key points are selected from the reliability map R using bilinear interpolation. The response values ​​of the candidate key points selected from the key point heat map K and the feature responses of the candidate key points selected from the reliability map R are multiplied to obtain a comprehensive score. The comprehensive scores are sorted and screened to obtain key points, and then bilinear interpolation is used to extract the feature descriptors of the corresponding image to be matched from the feature map F according to the screened key points.

[0023] Furthermore, the adaptive multi-scale descriptor enhancement module includes a dynamic hyperbolic tangent function, a first Mamba layer, a second Mamba layer, a third Mamba layer, a linear layer, and a GeLU activation function. The processing flow of the adaptive multi-scale descriptor enhancement module is as follows: processing the feature descriptor of the image to be matched by the dynamic hyperbolic tangent function, adding the output of the dynamic hyperbolic tangent function to the feature descriptor of the image to be matched to obtain a first added descriptor, inputting the first added descriptor into the first Mamba layer, the second Mamba layer, and the third Mamba layer respectively, averaging and fusing the outputs of the first Mamba layer, the second Mamba layer, and the third Mamba layer, adding the average fused output to the first added descriptor to obtain a second added descriptor, inputting the second added descriptor into the linear layer for processing, concatenating the output of the linear layer with the second added descriptor to obtain a third added descriptor, and concatenating the output of the third added descriptor after the GeLU activation function with the third added descriptor to obtain the output of the adaptive multi-scale descriptor enhancement module, i.e., the enhanced descriptor of the image to be matched.

[0024] Furthermore, the image to be matched is input into the key point detection network to generate the key point heat map K. The specific process is as follows: the image to be matched is divided into multiple grid cells, and each grid cell is reshaped into a feature vector , then the eigenvector Through four sixteenth convolution layers, the convolution features are obtained , and then the convolution feature After being processed by the softmax function, it is expanded to the resolution of the image to be matched, and the key point heat map K is obtained.

[0025] Furthermore, the Euclidean distance is used to complete the brute force matching of the enhanced descriptors of the two images to be matched; for the enhanced descriptors of the given two images to be matched, the Euclidean distance between the two is calculated to measure the similarity and perform matching.

[0026] Compared with the existing technology, the present invention has the following beneficial effects:

[0027] (1) The present invention constructs a new dual-branch collaborative architecture. From the perspective of dual-branch collaboration of feature detection and feature description, key point detection and feature description are processed separately. The key point detection network focuses on low-level features, avoids the interference of high-level features, and improves the accuracy of key point detection. At the same time, through the feature extraction backbone network composed of a differential wavelet transform downsampling module and a dynamic multi-scale feature fusion module, discrete wavelet transform is used to separate noise and key features, combined with differential operation, redundant information is preliminarily removed, and key features are enhanced. Then, a multi-branch structure is used to extract multi-scale features, and the contribution of each branch is adjusted in combination with dynamic weighting to further remove redundant information. At the same time, efficient multi-scale feature fusion is achieved to adapt to scale changes under different perspectives. In the descriptor generation network, an adaptive multi-scale descriptor enhancement module is designed, and multiple mechanisms such as dynamic feature adjustment, multi-scale feature fusion, nonlinear activation, and adaptive feature fusion are used to improve the quality and expression ability of the descriptor, thereby improving the accuracy of feature matching.

[0028] (2) The differential wavelet transform downsampling module proposed in the present invention uses discrete wavelet transform to perform frequency domain decomposition of features, separates noise from key features, and combines the differential operation of calculating features and neighborhood average pooling differences to effectively remove redundant information and enhance key features, so that the descriptor can more accurately capture the key features of the image, suppress the contrast distortion caused by illumination changes, improve the stability of feature response in scenes with uneven illumination, and improve the accuracy of subsequent feature matching.

[0029] (3) The dynamic multi-scale feature fusion module proposed in this paper divides the input channel into three parts, and uses depthwise separable convolutions of different sizes to extract multi-scale features, avoiding the high computational cost of full-channel convolution and reducing the amount of computation through grouped convolution. It also combines dynamic weighting to adjust the contribution of each branch, uses activation functions to smoothly amplify key features, and suppresses irrelevant information, thus achieving efficient multi-scale feature fusion, enhancing the scale invariance of the descriptor, and improving the feature matching accuracy under viewing angle changes.

[0030] (4) The adaptive multi-scale descriptor enhancement module proposed in this invention uses a dynamic hyperbolic tangent function to perform nonlinear dynamic scaling on the input descriptor, adaptively adjusts the feature distribution, effectively suppresses the feature ambiguity caused by complex illumination and perspective changes, and improves the robustness of the descriptor; at the same time, it uses mamba layers with different sizes of local convolution to capture feature information of different receptive fields, and integrates the multi-scale information of the descriptor by averaging and fusing the multi-scale information, thereby enhancing the descriptor's ability to express local geometric structure and global semantic information, and achieving high-precision feature matching in complex scenes with complex perspective and illumination changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the feature matching model structure of the present invention.

[0032] Figure 2 This is a structural diagram of the feature extraction module of the present invention.

[0033] Figure 3 This is a structural diagram of the differential wavelet transform downsampling module of the present invention.

[0034] Figure 4 Schematic diagram of the edge information enhancement module structure of the present invention.

[0035] Figure 5 This is a structural diagram of the dynamic multi-scale feature fusion module of the present invention.

[0036] Figure 6 Schematic diagram of the structure of the adaptive multi-scale descriptor enhancement module of the present invention. DETAILED DESCRIPTION

[0037] The present invention provides a technical solution: a feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion, comprising the following steps:

[0038] Read the images to be matched (set to two); build a feature matching model, such as Figure 1 As shown; the feature matching model includes a backbone network, a key point detection network and a descriptor generation network; the backbone network is composed of a feature extraction module and a sixth convolution block; the image to be matched is input into the feature extraction module in the backbone network to generate a feature map F, and the feature map F is passed through the sixth convolution block to generate a reliability map R; the image to be matched is input into the key point detection network to generate a key point heat map K, and the key point heat map K and the reliability map R are input into the descriptor generation network to filter out the key points, and finally, the feature descriptors of the corresponding image to be matched are extracted from the feature map F according to the filtered key points, and the feature descriptors of the image to be matched are further processed by the adaptive multi-scale descriptor enhancement module to obtain the enhanced descriptors of the image to be matched, and finally, the enhanced descriptors of the two images to be matched are brute-force matched; through this process, a set of key points and feature descriptors with high accuracy and high response can be generated, thereby achieving efficient and accurate feature matching.

[0039] Among them, the Euclidean distance is used to complete the brute force matching of the enhanced descriptors of the two images to be matched; for the enhanced descriptors of the given two images to be matched, the Euclidean distance between the two is calculated to measure the similarity for matching.

[0040] The specific process of inputting the image to be matched into the key point detection network and generating the key point heat map K is as follows: the image to be matched is divided into multiple 8×8 grid cells, and each grid cell is reshaped into a 64-dimensional feature vector , used to model the key point distribution within each grid, and then the feature vector Through four 1×1 sixteenth convolution layers, the convolution features are obtained , and then the convolution feature After being processed by the softmax function, it is expanded to the original input image (image to be matched) resolution to obtain the key point heat map K.

[0041] like Figure 2 As shown in the figure, the feature extraction module includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block; the input (image to be matched) passes through the first convolution block and the second convolution block respectively, and the outputs of the first convolution block and the second convolution block are added to obtain the added features, and the added features are sequentially passed through the third convolution block, the fourth convolution block and the fifth convolution block to obtain the feature map F.

[0042] Among them, the first convolution block consists of the first convolution layer (3×3), the first differential wavelet transform downsampling module, the first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module and the second differential wavelet transform downsampling module connected in sequence; the input (image to be matched) passes through the first convolution layer (3×3), the first differential wavelet transform downsampling module, the first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module and the second differential wavelet transform downsampling module in sequence to obtain the output of the first convolution block.

[0043] Among them, the second convolution block consists of the first average pooling layer (4×4) and the second convolution layer (1×1) connected in sequence; the input (image to be matched) passes through the first average pooling layer and the second convolution layer (1×1) in sequence to obtain the output of the second convolution block.

[0044] Among them, the third convolution block consists of the second dynamic multi-scale feature fusion module and the third dynamic multi-scale feature fusion module connected in sequence; the input (additional features) passes through the second dynamic multi-scale feature fusion module and the third dynamic multi-scale feature fusion module in sequence to obtain the output of the third convolution block.

[0045] Among them, the fourth convolution block is composed of the third differential wavelet transform downsampling module, the fourth dynamic multi-scale feature fusion module and the fifth dynamic multi-scale feature fusion module connected in sequence; the input (the output of the third convolution block) passes through the third differential wavelet transform downsampling module, the fourth dynamic multi-scale feature fusion module and the fifth dynamic multi-scale feature fusion module in sequence to obtain the output of the fourth convolution block.

[0046] Among them, the fifth convolution block is composed of the third convolution layer (3×3), the fourth convolution layer (3×3) and the fifth convolution layer (1×1) connected in sequence; the input (the output of the fourth convolution block) passes through the third convolution layer (3×3), the fourth convolution layer (3×3) and the fifth convolution layer (1×1) in sequence to obtain the output of the fifth convolution block.

[0047] Among them, the sixth convolution block is composed of the sixth convolutional layer (1×1), the seventh convolutional layer (1×1) and the eighth convolutional layer (1×1) connected in sequence; the input (feature map F) passes through the sixth convolutional layer (1×1), the seventh convolutional layer (1×1) and the eighth convolutional layer (1×1) in sequence and then passes through the sigmoid function to obtain the reliability map R.

[0048] Among them, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer are sequentially connected to a batch normalization layer (BN) and a ReLU activation function.

[0049] like Figure 3 As shown in FIG, the first differential wavelet transform downsampling module (DWTD), the second differential wavelet transform downsampling module, the third differential wavelet transform downsampling module and the fourth differential wavelet transform downsampling module have the same structure; the first differential wavelet transform downsampling module includes the ninth convolution layer (1×1), the discrete wavelet transform module, the tenth convolution layer (1×1), the edge information enhancement module and the eleventh convolution layer (1×1); the processing flow of the first differential wavelet transform downsampling module is: the input (output of the first convolution layer) ) After the ninth convolutional layer, the output of the ninth convolutional layer is input into the discrete wavelet transform module for frequency domain decomposition to obtain low-frequency information and high-frequency information , which can be expressed as:

[0050] ;

[0051] Where, represents the discrete wavelet forward transform function.

[0052] High-frequency information Further decomposed into horizontal high-frequency information , vertical high-frequency information and diagonal high-frequency information .

[0053] Since the diagonal high-frequency information mainly contains large-scale smooth information and has limited contribution to the extraction of fine-grained edge features, it is directly discarded to reduce redundancy, and only the other three information are retained to focus on details and edge features.

[0054] Horizontal high-frequency information , vertical high-frequency information and low-frequency information Splicing along the channel dimension to obtain a high-frequency information feature map, the high-frequency information feature map is passed through the tenth convolutional layer and the edge information enhancement module in sequence, the output of the edge information enhancement module is multiplied by the output of the tenth convolutional layer, and the multiplied output is passed through the eleventh convolutional layer to obtain the output of the first differential wavelet transform downsampling module.

[0055] The second differential wavelet transform downsampling module, the third differential wavelet transform downsampling module, the fourth differential wavelet transform downsampling module and the first differential wavelet transform downsampling module have the same structure, and the processing flow will not be described in detail here.

[0056] like Figure 4 As shown in Figure 1, the edge information enhancement module includes a second average pooling layer and a twelfth convolutional layer. The processing flow of the edge information enhancement module is as follows: the input (the output of the tenth convolutional layer) passes through the second average pooling layer, and the output of the second average pooling layer is subtracted from the input (the output of the tenth convolutional layer). The subtracted output is first adjusted by the twelfth convolutional layer, and then activated by the Sigmoid function to obtain the weight map. , the weight graph Add 1 (since the output of the sigmoid function is between 0 and 1, adding 1 changes it to 1 to 2, which can prevent errors such as division by zero in subsequent calculations) to obtain the output of the edge information enhancement module ; This design of the edge information enhancement module can maintain the integrity of feature distribution while enhancing edge information, providing a more balanced expression capability for the descriptor.

[0057] The differential wavelet transform downsampling module not only increases the feature expression but also effectively removes noise and redundant information by increasing the frequency domain information through transformation in the feature space domain.

[0058] like Figure 5As shown in the figure, the first dynamic multi-scale feature fusion module (DMFF), the second dynamic multi-scale feature fusion module, the third dynamic multi-scale feature fusion module, the fourth dynamic multi-scale feature fusion module and the fifth dynamic multi-scale feature fusion module have the same structure; the first dynamic multi-scale feature fusion module includes a first depth-wise separable convolution (3×3), a Split function, a second depth-wise separable convolution (5×5), a third depth-wise separable convolution (7×7), a thirteenth convolution layer (1×1), a fourteenth convolution layer (1×1), a fifteenth convolution layer (1×1) and a SiLU activation function; the processing flow of the first dynamic multi-scale feature fusion module is: the input (the output of the first differential wavelet transform downsampling module) is respectively After the first depth-wise separable convolution and the fourteenth convolution layer, the output of the first depth-wise separable convolution is divided into the first group of features, the second group of features and the third group of features in the channel dimension according to (1:1:2) through the Split function. The second group of features and the third group of features are respectively subjected to the second depth-wise separable convolution and the third depth-wise separable convolution. The outputs of the second depth-wise separable convolution and the third depth-wise separable convolution are spliced ​​with the first group of features in the channel dimension, and the spliced ​​result is input into the thirteenth convolution layer. The outputs of the thirteenth convolution layer and the fourteenth convolution layer are multiplied after a SiLU activation function. The multiplied result is passed through the fifteenth convolution layer to obtain the output of the first dynamic multi-scale feature fusion module.

[0059] Among them, the key point heat map K and the reliability map R are input into the descriptor generation network to screen out the key points, and finally the feature descriptors of the corresponding image to be matched are extracted from the feature map F according to the screened key points. The specific process is as follows: the key point heat map K is non-maximum suppressed, and the local response peak points are selected as candidate key points; the response values ​​of the candidate key points are selected from the key point heat map K using nearest neighbor interpolation, and the feature responses of the candidate key points are selected from the reliability map R using bilinear interpolation. The response values ​​of the candidate key points selected from the key point heat map K and the feature responses of the candidate key points selected from the reliability map R are multiplied to obtain a comprehensive score. According to the comprehensive score, a group of high-accuracy and high-response key points are obtained, and then the feature descriptors of the corresponding image to be matched are extracted from the feature map F using bilinear interpolation according to the screened key points.

[0060] like Figure 6As shown in the figure, the adaptive multi-scale descriptor enhancement module (AMDE) includes a dynamic hyperbolic tangent function, a first Mamba layer (convolution kernel size is 2), a second Mamba layer (convolution kernel size is 3), a third Mamba layer (convolution kernel size is 4), a linear layer and a GeLU activation function; the processing flow of the adaptive multi-scale descriptor enhancement module is as follows: the input (feature descriptor of the image to be matched) is processed by a dynamic hyperbolic tangent function (Dy-Tanh), which dynamically adjusts the activation strength and slope according to the input features, thereby introducing nonlinearity and enhancing the feature expression ability of the model, and the output of the dynamic hyperbolic tangent function is compared with the input (feature descriptor of the image to be matched). Add to obtain a first added descriptor, input the first added descriptor into the first Mamba layer, the second Mamba layer and the third Mamba layer respectively, averagely fuse the outputs of the first Mamba layer, the second Mamba layer and the third Mamba layer, add the average fused output to the first added descriptor to obtain a second added descriptor, input the second added descriptor into the linear layer for processing, concatenate the output of the linear layer with the second added descriptor to obtain a third added descriptor, and concatenate the output of the third added descriptor after passing the GeLU activation function with the third added descriptor to obtain the output of the adaptive multi-scale descriptor enhancement module, that is, the enhanced descriptor of the image to be matched.

[0061] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion, characterized in that: The method comprises the following steps: reading an image to be matched; constructing a feature matching model; the feature matching model comprises a backbone network, a key point detection network and a descriptor generation network; the backbone network is composed of a feature extraction module and a sixth convolution block; inputting the image to be matched into the feature extraction module in the backbone network to generate a feature map, passing the feature map through the sixth convolution block to generate a reliability map; inputting the image to be matched into the key point detection network to generate a key point heat map, inputting the key point heat map and the reliability map into the descriptor generation network to screen out key points, finally extracting the feature descriptor of the corresponding image to be matched from the feature map according to the screened key points, processing the feature descriptor of the image to be matched through an adaptive multi-scale descriptor enhancement module to obtain an enhanced descriptor of the image to be matched, and finally performing brute force matching on the enhanced descriptor of the image to be matched; setting two images to be matched to be read, and finally performing brute force matching on the enhanced descriptors of the two images to be matched; The feature extraction module includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block; the image to be matched passes through the first convolution block and the second convolution block respectively, the outputs of the first convolution block and the second convolution block are added to obtain an added feature, and the added feature passes through the third convolution block, the fourth convolution block, and the fifth convolution block in sequence to obtain a feature map F; The first convolution block is composed of a first convolution layer, a first differential wavelet transform downsampling module, a first dynamic multi-scale feature fusion module, a second dynamic multi-scale feature fusion module, and a second differential wavelet transform downsampling module connected in sequence; the image to be matched is sequentially passed through the first convolution layer, the first differential wavelet transform downsampling module, the first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module, and the second differential wavelet transform downsampling module to obtain the output of the first convolution block; The second convolution block is composed of a first average pooling layer and a second convolution layer connected in sequence; the image to be matched passes through the first average pooling layer and the second convolution layer in sequence to obtain the output of the second convolution block; The third convolution block is composed of a second dynamic multi-scale feature fusion module and a third dynamic multi-scale feature fusion module connected in sequence; the output of the third convolution block is obtained after the added features pass through the second dynamic multi-scale feature fusion module and the third dynamic multi-scale feature fusion module in sequence; The fourth convolution block is composed of a third differential wavelet transform downsampling module, a fourth dynamic multi-scale feature fusion module, and a fifth dynamic multi-scale feature fusion module connected in sequence; the output of the third convolution block is sequentially passed through the third differential wavelet transform downsampling module, the fourth dynamic multi-scale feature fusion module, and the fifth dynamic multi-scale feature fusion module to obtain the output of the fourth convolution block; The fifth convolutional block is composed of the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer connected in sequence; the output of the fourth convolutional block is passed through the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer in sequence to obtain the output of the fifth convolutional block; Among them, the sixth convolution block consists of the sixth convolution layer, the seventh convolution layer and the eighth convolution layer connected in sequence; the feature map F passes through the sixth convolution layer, the seventh convolution layer and the eighth convolution layer in sequence and then passes through the sigmoid function to obtain the reliability map; The first differential wavelet transform downsampling module, the second differential wavelet transform downsampling module, the third differential wavelet transform downsampling module and the fourth differential wavelet transform downsampling module have the same structure; the first differential wavelet transform downsampling module includes a ninth convolution layer, a discrete wavelet transform module, a tenth convolution layer, an edge information enhancement module and an eleventh convolution layer; The processing flow of the first differential wavelet transform downsampling module is as follows: The output of the first convolutional layer passes through the ninth convolutional layer, and the output of the ninth convolutional layer is input into the discrete wavelet transform module for frequency domain decomposition to obtain low-frequency information and high-frequency information; The high-frequency information is further decomposed into horizontal high-frequency information, vertical high-frequency information and diagonal high-frequency information; The horizontal high-frequency information, vertical high-frequency information, and low-frequency information are spliced ​​along the channel dimension to obtain a high-frequency information feature map. The high-frequency information feature map is sequentially passed through the tenth convolutional layer and the edge information enhancement module. The output of the edge information enhancement module is multiplied by the output of the tenth convolutional layer. The multiplied output is then passed through the eleventh convolutional layer to obtain the output of the first differential wavelet transform downsampling module. The edge information enhancement module includes a second average pooling layer and a twelfth convolutional layer. The processing flow of the edge information enhancement module is as follows: the output of the tenth convolutional layer passes through the second average pooling layer, the output of the tenth convolutional layer is subtracted from the output of the second average pooling layer, the subtracted output is first adjusted by the twelfth convolutional layer, and then activated by the Sigmoid function to obtain a weight map, and the weight map is added by 1 to obtain the output of the edge information enhancement module. The first dynamic multi-scale feature fusion module, the second dynamic multi-scale feature fusion module, the third dynamic multi-scale feature fusion module, the fourth dynamic multi-scale feature fusion module and the fifth dynamic multi-scale feature fusion module have the same structure; the first dynamic multi-scale feature fusion module includes a first depth-separable convolution, a Split function, a second depth-separable convolution, a third depth-separable convolution, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer and a SiLU activation function; The processing flow of the first dynamic multi-scale feature fusion module is as follows: the output of the first differential wavelet transform downsampling module passes through the first depth-separable convolution and the fourteenth convolution layer respectively, the output of the first depth-separable convolution is divided into the first group of features, the second group of features and the third group of features in proportion in the channel dimension through the Split function, the second group of features and the third group of features pass through the second depth-separable convolution and the third depth-separable convolution respectively, the output of the second depth-separable convolution and the third depth-separable convolution and the first group of features are spliced ​​in the channel dimension, the spliced ​​result is input into the thirteenth convolution layer, the outputs of the thirteenth convolution layer and the fourteenth convolution layer are multiplied after passing a SiLU activation function, and the multiplied result passes through the fifteenth convolution layer to obtain the output of the first dynamic multi-scale feature fusion module; The key point heat map and the reliability map are input into the descriptor generation network to screen out the key points. Finally, the feature descriptors of the corresponding image to be matched are extracted from the feature map F according to the screened key points. The specific process is as follows: the key point heat map is subjected to non-maximum suppression, and the local response peak points are selected as candidate key points; the response values ​​of the candidate key points are selected from the key point heat map using nearest neighbor interpolation, and the feature responses of the candidate key points are selected from the reliability map using bilinear interpolation. The response values ​​of the candidate key points selected from the key point heat map and the feature responses of the candidate key points selected from the reliability map are multiplied to obtain a comprehensive score. The comprehensive scores are sorted and screened to obtain key points. Then, bilinear interpolation is used to extract the feature descriptors of the corresponding image to be matched from the feature map F according to the screened key points. The adaptive multi-scale descriptor enhancement module includes a dynamic hyperbolic tangent function, a first Mamba layer, a second Mamba layer, a third Mamba layer, a linear layer and a GeLU activation function. The processing flow of the adaptive multi-scale descriptor enhancement module is as follows: the feature descriptor of the image to be matched is processed by the dynamic hyperbolic tangent function, the output of the dynamic hyperbolic tangent function is added to the feature descriptor of the image to be matched to obtain a first added descriptor, the first added descriptor is input into the first Mamba layer, the second Mamba layer and the third Mamba layer respectively, the outputs of the first Mamba layer, the second Mamba layer and the third Mamba layer are averaged and fused, the output after the average fusion is added to the first added descriptor to obtain a second added descriptor, the second added descriptor is input into the linear layer for processing, the output of the linear layer is spliced ​​with the second added descriptor to obtain a third added descriptor, the output of the third added descriptor after the GeLU activation function is spliced ​​with the third added descriptor to obtain the output of the adaptive multi-scale descriptor enhancement module, that is, the enhanced descriptor of the image to be matched.

2. The feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion according to claim 1, characterized in that: The specific process of inputting the image to be matched into the key point detection network and generating the key point heat map is as follows: the image to be matched is divided into multiple grid cells, and each grid cell is reshaped into a feature vector , then the eigenvector Through four sixteenth convolution layers, the convolution features are obtained , and then the convolution feature After being processed by the softmax function, it is expanded to the resolution of the image to be matched to obtain the key point heat map.

3. The feature matching method based on differential wavelet transform and dynamic multi-scale feature fusion according to claim 2, characterized in that: The Euclidean distance is used to complete the brute force matching of the enhanced descriptors of the two images to be matched; for the enhanced descriptors of the given two images to be matched, the Euclidean distance between the two is calculated to measure the similarity and perform matching.

Images