Dynamic feature extraction and description method based on difference fusion of multi-layer features
By optimizing keypoint detection through multi-layer feature self-difference fusion and approximate keypoint distance loss function, the problem of low keypoint detection accuracy in existing technologies is solved, achieving higher matching accuracy and improved performance of downstream tasks.
Patent Information
- Application Number
- CN202210849429.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-19
- Publication Date
- 2025-11-04
- Estimated Expiration
- 2042-07-19
AI Technical Summary
Existing joint feature extraction and description methods suffer from low keypoint detection accuracy due to the large receptive field of the keypoint detection score map, which affects matching accuracy and the performance of downstream tasks. Furthermore, they lack an optimized loss function for keypoint detection.
A dynamic feature extraction method based on multi-layer feature self-difference fusion is adopted. The difference score map of the feature map is calculated and fused through the multi-layer feature self-difference detection module. The key point detection and descriptor extraction are optimized by combining the approximate key point distance loss function and the positive matching ratio loss function.
It improves the accuracy of key point detection and matching, reduces the number of false matches, and enhances the performance of downstream tasks.
Smart Images
Figure CN115331021B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of image processing, and more particularly relates to a dynamic feature extraction and description method based on multi-layer feature self-difference fusion. BACKGROUND
[0002] Detecting and describing the pixel-level correspondence between different images is a basic task of many computer vision applications, such as image matching, visual positioning, three-dimensional reconstruction, and image stitching. In these applications, there are usually disturbances such as illumination, noise, blur, rotation, scale, affine, and occlusion, which can make the found pixel-level correspondence less or more errors, affecting the final application effect. Therefore, extracting repeatable key points and distinguishable, robust descriptors is the basis and key of many vision tasks.
[0003] Joint feature extraction and description is the current mainstream image matching method, which uses a network to output key points and descriptors at the same time, simplifying the traditional matching process and obtaining more global information, which helps to improve the robustness of the descriptor and obtain more correspondence.
[0004] At present, most of the joint feature extraction and description matching methods have low key point detection accuracy due to the large receptive field of the key point detection score map, which limits the matching accuracy and the effect of downstream tasks. Moreover, due to the discrete nature and non-differentiability of key point coordinates, existing methods lack a detection loss function that directly optimizes key point detection, further limiting key point detection accuracy and matching accuracy. Therefore, a method that can accurately detect key points and robustly describe is needed to improve the performance of downstream tasks. SUMMARY
[0005] In view of the above defects or improvement needs of the prior art, the present application provides a dynamic feature extraction and description method based on multi-layer feature self-difference fusion, which aims to improve the key point detection accuracy of the current joint feature extraction and description method by using multi-layer feature self-difference fusion and approximate key point distance loss function, and further improve the number and accuracy of matching, and further improve the performance of downstream tasks.
[0006] To achieve the above purpose, in a first aspect, the present application provides a method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion, comprising:
[0007] S1, a feature extraction and description model is built; wherein the feature extraction and description model comprises:
[0008] a feature extraction module for extracting features from input images to obtain feature maps at different levels;
[0009] The multi-layer feature self-difference detection module is configured to calculate a difference between each pixel feature dimension value and an average feature value according to a feature map of a certain layer to obtain a feature self-difference score map of the corresponding layer; and fuse feature self-difference score maps of several layers to obtain a key point detection score map.
[0010] S2, calculating a loss function and training the feature extraction and description model by back propagation of the loss;
[0011] The loss function comprises a key point detection accuracy loss function and a regression accuracy loss function. The key point detection accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of corresponding patches divided by the first key point detection score map and the second key point detection score map, wherein the first key point detection score map is a key point detection score map of the first image obtained by the feature extraction and description model, and the second key point detection score map is a key point detection score map of the second image mapped according to a true value matching map between the first image and the second image under the same scene. The regression accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of each patch divided by the key point detection score map and non-derivable maximum value point coordinates obtained by argmax.
[0012] Further, the loss function further comprises a positive matching proportion loss function and a false matching proportion loss function. The positive matching proportion loss function is used to increase the number of correct matches by making the proportion between positive samples and the most difficult negative samples as small as possible when training the feature extraction and description model. The false matching proportion loss function is used to filter out points difficult to be correct matches by making the distance between the nearest neighbor and the second nearest neighbor of the points difficult to be correct matches as large as possible when training the feature extraction and description model.
[0013] Further, F and F' are the descriptor feature maps of the first image I and the second image I', the K largest pixels in the first key point detection score map S are selected as key points, and the descriptor set D = {d1, d2, d3, …, dK} of these key points is selected in the descriptor feature map F. Meanwhile, the corresponding descriptor D' = {d'1, d'2, d'3, …, d'K} is found in the descriptor feature map F' according to the true value matching map M. The Euclidean distance between the descriptor set D and D' is calculated to obtain a distance matrix J with a size of KxK. The elements on the diagonal line of the distance matrix J represent the descriptor distance of the corresponding point pair. K K
[0014] The positive matching proportion loss function is represented as:
[0015]
[0016] where E(·) is the Euclidean distance function between two descriptors, c p is a constant coefficient to ignore the better samples that have been distinguished, d k and d′ k are the descriptors of the corresponding points, d′ n is the descriptor in D′ that has the smallest Euclidean distance to d k except the matched pair, d n is the descriptor in D that has the smallest Euclidean distance to d′ k except the matched pair.
[0017] The mismatch ratio loss function is expressed as:
[0018]
[0019] where Q is the element on the diagonal of distance matrix J, J q,min1 and J q,min2 are the minimum and second minimum values of the row where the point is difficult to be the correct match, c n is a constant coefficient.
[0020] Further, let S and S′ be the key point detection score maps of the first image I and the second image I′, and let S′ M be the detection score map of the second image I′ mapped according to the ground truth matching map M, the key point detection score maps S and S′ M are divided into N patches on average.
[0021] The key point detection accuracy loss function is expressed as:
[0022]
[0023] where x n and y n are the approximate derivable maximum point coordinates of the nth patch of the key point detection score map S obtained through softargmax regression, x n′ and y n′ are the approximate derivable maximum point coordinates of the nth patch of the key point detection score map S′ M obtained through softargmax regression.
[0024] The regression accuracy loss function is expressed as:
[0025]
[0026] where, and is the non-differentiable maximum point coordinate of the nth patch of the key point detection score map S obtained by argmax, and is the non-differentiable maximum point coordinate of the nth patch of the key point detection score map S' obtained by argmax. M is the non-differentiable maximum point coordinate of the nth patch of the key point detection score map S obtained by argmax.
[0027] Further, the feature extraction module comprises a plurality of standard convolution layers and dynamic attention convolution layers in cascade.
[0028] In a second aspect, the present application provides a dynamic feature extraction and description method based on multi-layer feature self-difference fusion, comprising:
[0029] The image to be processed is input into the feature extraction and description model constructed by the construction method of the dynamic feature extraction and description model based on multi-layer feature self-difference fusion of the first aspect, to obtain the description sub-feature map and the key point detection score map corresponding to the image to be processed.
[0030] In a third aspect, the present application provides an image matching method, comprising:
[0031] The two images to be matched are input into the feature extraction and description model constructed by the construction method of the dynamic feature extraction and description model based on multi-layer feature self-difference fusion of the first aspect, to obtain the respective description sub-feature map and key point detection score map;
[0032] The two key point detection score maps are subjected to non-maximum suppression and the respective maximum K key points are selected and According to the key points, the corresponding description sub-features in the description sub-feature map are selected and
[0033] The two description sub-features are matched and The nearest neighbor and the second nearest neighbor ratio matching are applied to obtain the matching relationship.
[0034] In a fourth aspect, the present application provides a computer readable storage medium comprising a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located is controlled to execute the construction method of the dynamic feature extraction and description model based on multi-layer feature self-difference fusion of the first aspect.
[0035] In a fifth aspect, the present application provides a computer readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls a device in which the storage medium is located to perform the multi-layer feature self-difference fusion based dynamic feature extraction and description method according to the second aspect and / or the image matching method according to the third aspect.
[0036] Overall, compared with the prior art, the above method conceived by the present application can achieve the following beneficial effects:
[0037] 1、The feature self-difference calculation module calculates the corresponding feature self-difference maps for the outputs of different network layers, and then the multi-layer fusion module fuses them to obtain a key point detection score map with multi-layer information, which can effectively improve the key point detection accuracy, and further improve the matching quantity and matching accuracy; at the same time, the extracted key points can be more distributed in the texture region, thereby reducing the number of false matches caused by the distribution of key points in non-texture regions. Further, a key point detection accuracy loss function and a regression accuracy loss function are proposed to optimize the key point detection capability of the network in the training stage, and the problem of non-differentiable key point coordinates caused by discrete key point coordinates is solved by using softargmax to approximate the local region maximum value point coordinates, which can effectively improve the key point detection accuracy of the network.
[0038] 2、The positive matching ratio loss function and the false matching ratio loss function are used to optimize the distance between positive examples and difficult negative examples, improve the number of correct matches, reduce the number of false matches, and further improve the matching accuracy.
[0039] 3、The feature extraction module based on dynamic attention convolution is used to extract dense features from the image, which can select the most suitable network parameters for different samples, and further improve the key point detection accuracy and the discriminability of the descriptor. BRIEF DESCRIPTION OF DRAWINGS
[0040] Figure 1 is a flowchart of the construction method of the multi-layer feature self-difference fusion based dynamic feature extraction and description model provided by the embodiment of the present application;
[0041] Figure 2 is a schematic diagram of the overall network model provided by the embodiment of the present application;
[0042] Figure 3 is a schematic diagram of the approximate key point distance loss function calculation flow provided by the embodiment of the present application;
[0043] Figure 4 is a schematic diagram of the descriptor distance matrix provided by the embodiment of the present application;
[0044] Figure 5 is a flowchart of the image matching provided by the embodiment of the present application;
[0045] Figure 6 is a key point extraction and matching effect diagram provided by the embodiment of the present application. DETAILED DESCRIPTION
[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.
[0047] Embodiment 1
[0048] Referring to Figure 1 , in combination Figures 2 to 4 , the present application provides a construction method of a dynamic feature extraction and description model based on multi-layer feature self-difference fusion, comprising:
[0049] S1, building a feature extraction and description model; wherein the feature extraction and description model comprises:
[0050] a feature extraction module for extracting features of an input image to obtain feature maps of different levels;
[0051] a multi-layer feature self-difference detection module for calculating the difference between each pixel feature value and the average feature value in the feature value dimension according to a feature map of a certain level to obtain a feature self-difference score map of the corresponding level; and fusing the feature self-difference score maps of several levels to obtain a key point detection score map;
[0052] S2, calculating a loss function and training the feature extraction and description model by back-propagating the loss;
[0053] The loss function comprises a key point detection accuracy loss function and a regression accuracy loss function. The key point detection accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of corresponding patches divided by a first key point detection score map and a second key point detection score map, wherein the first key point detection score map is a key point detection score map of a first image obtained by feature extraction and description of the first image, and the second key point detection score map is a key point detection score map of a second image mapped according to a true value matching map between the first image and the second image in the same scene. The regression accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of each patch divided by a key point detection score map and non-derivable maximum value point coordinates obtained by argmax.
[0054] In this embodiment, as shown in the figure, Figure 2 The network model comprises a feature extraction module based on dynamic attention convolution, a multi-layer feature self-difference detection module, an approximate key point distance detection loss function and a positive matching and false matching proportion description loss function. The proposed algorithm is described in detail below.
[0055] (1) Feature extraction module based on dynamic attention convolution
[0056] As shown in the figure, Figure 2 The feature extraction module based on dynamic attention convolution is used for image dense feature extraction, which comprises four standard convolution layers and five dynamic attention convolution layers. Each convolution layer has a 3x3 convolution kernel and a step length of 1, and uses dilated convolution to increase the receptive field. The dynamic attention convolution layer is used to adaptively change the network parameters for different samples. Multiple sets of convolution kernel parameters are set for each layer, and the weights of different convolution kernel parameters are obtained by an attention mechanism. Finally, the different convolution kernel parameters are weighted and aggregated to realize the dynamic change of the parameters, thereby improving the feature extraction capability of the network. The final feature extraction module outputs a 128-dimensional feature map representing the descriptor of each pixel.
[0057] The dynamic attention convolution dynamically changes the network convolution parameters through the attention mechanism for different input pictures during testing, so as to adapt to different sample features and enhance the feature extraction capability of the network. The calculation formula of the input feature x and the output feature y is as follows:
[0058]
[0059]
[0060]
[0061] wherein g(·) represents an activation function, Wk denotes the convolution weight matrix corresponding to the kth set of convolution parameters, b k denotes the bias vector corresponding to the kth set of convolution parameters, K denotes the number of sets of convolution parameters, c k denotes the attention weight of the kth set of convolution parameters. In this patent, K = 4, that is, each dynamic attention convolution layer has four different sets of convolution parameters.
[0062] As Figure 2 shown, the attention mechanism of the dynamic attention convolution first down-samples the feature map and expands the feature channel, fuses the spatial information into the feature dimension, secondly, uses global average pooling to compress the spatial dimension, then, uses a fully connected layer and sigmoid activation function to regress the attention weight c k of the K convolution kernel parameters, and finally, weights all the convolution kernel parameters to generate the final convolution kernel weight for convolution.
[0063] (2) Multi-layer feature self-difference detection module
[0064] The multi-layer feature self-difference detection module includes a feature self-difference calculation module and a multi-layer fusion part, which is used to obtain a detection score map with multi-layer information and capable of realizing accurate key point detection.
[0065] (2-1) Feature self-difference calculation module
[0066] As Figure 2 shown, the feature self-difference calculation module calculates the self-difference of the feature dimension of each pixel of the feature map to represent the saliency, so that the more information around the key point, the larger the score, and these key points are more likely to be detected at the same position in other images. Given the feature map f∈R H×W×C , the feature self-difference score of each position (i,j) can be calculated as:
[0067]
[0068]
[0069] where C is the number of feature channels, and the corresponding feature self-difference score map S l can be calculated according to the output feature map of the lth convolution layer in this way.
[0070] (2-2) Multi-layer fusion
[0071] The multi-layer fusion is used to fuse the difference score maps of low, medium, and high-level features to obtain a keypoint detection score map, thereby fully utilizing the low-level structural information and higher-level semantic information learned by the network to improve keypoint detection accuracy. In this embodiment, the feature maps of layers 2, 4, and 6 are fused to generate the keypoint detection score map, and the specific fusion method is shown in the following formula:
[0072]
[0073] (3) Approximate key point distance detection loss function
[0074] The approximate keypoint distance detection loss function includes a keypoint detection accuracy loss function and a regression accuracy loss function, which are used to optimize the keypoint detection capability of the network during the training phase. By using softargmax to approximate the coordinates of the local maximum point, the problem of non-differentiability caused by the discrete keypoint coordinates is solved, which can effectively improve the keypoint detection accuracy of the network.
[0075] The softargmax method is a technique that combines a differentiable softmax with a non-differentiable argmax, and can approximate the index of the maximum value of an array through a probability distribution. Given a two-dimensional array H∈R H×W The coordinates of the maximum point can be approximated as:
[0076]
[0077]
[0078]
[0079] For two images I and I′ of the same scene, and the truth-matching graph M∈R between them. H×W×2 Specifically, it can be done through M i,j = (i′,j′) finds the pixel (i′,j′) in the second image corresponding to the pixel (i′,j′) in the first image. Let S and S′ be the keypoint detection scores calculated by the multi-layer feature self-difference detection module for images I and I′, and let S′ M For the detection score map of image I′ mapped from M, then the key point p in image S... i,j and in Figure S′ M The corresponding point p′ in i,j They are identical in coordinates. Since the softargmax function can only regress the coordinates of the maximum points in a single region and cannot obtain a large number of key points for loss calculation, this patent uses score maps S and S′... M The average number of patches is α, so N = HW / α can be regressed from the score plot S. 2A key point.
[0080] The key point detection accuracy loss function, that is, the mean square error loss of constructing the approximate maximum point coordinates of the corresponding patch, is used to improve the key point detection accuracy, and the calculation formula is as shown in the following formula:
[0081]
[0082] [x n ,y n ]=Φ(P n ),[x n′ ,y n′ ]=Φ(P n′ )
[0083] Wherein, P n and P n′ are the corresponding patches in the score map S and S' M , and Φ(·) is the softargmax function described above.
[0084] The regression accuracy loss function is used to ensure the accuracy of the softargmax function in approximating the region maximum point coordinates, and the distance between the approximate derivable maximum point coordinates obtained by the softargmax regression and the non-derivable maximum point coordinates obtained by the argmax is optimized to change the data distribution, and the calculation formula is as shown in the following formula:
[0085]
[0086]
[0087] Wherein, x n and y n are the approximate derivable coordinates calculated by the softargmax in P n , and are the accurate non-derivable coordinates obtained by the argmax in P n .
[0088] In addition, for the 192x192 size training image used in the patent, if α is set too large, the number of key points used to calculate the loss is too small, and if α is set too small, the information of the regression key point coordinates is insufficient, therefore, the patent uses a two-scale method to calculate the loss, specifically, the key point detection score map S and S' M are divided into blocks with sizes of 4 and 8, respectively, and the key point detection accuracy loss function and the regression accuracy loss function are calculated, and the final approximate key point distance detection loss function can be calculated as:
[0089] L AKD =L ra(α＝4)+L ra(α＝8) +β(L kda(α＝4) +L kda(α＝8) )
[0090] Wherein, β is the weighting coefficient for balancing the keypoint detection accuracy loss function and the regression accuracy loss function, which is set to 0.25 in this patent.
[0091] (4) Loss function describing the ratio of positive to false matches
[0092] The positive and negative matching ratio description loss function is used to optimize the network's ability to describe features during the training phase, thereby improving matching performance. It includes a positive matching ratio loss function and a negative matching ratio loss function, and is designed based on the nearest neighbor to second nearest neighbor ratio matching method.
[0093] Let F and F′ be the descriptor feature maps of images I and I′, and let F′ M This is the descriptor feature map of image I′ mapped from M. First, the K largest pixels in the detection score map S are selected as keypoints, and descriptors D = {d1, d2, d3, ..., d4} for these keypoints are selected in the descriptor feature map F. K Meanwhile, according to M, the corresponding descriptor D′={d′1,d′2,d′3,…,d′ is found in F′. K By calculating the Euclidean distance between the descriptive subsets D and D′, a distance matrix J of size K×K can be obtained, as shown in the figure. Figure 4 As shown, the elements on the diagonal represent the descriptor distance between corresponding point pairs.
[0094] (4-1) Positive Matching Proportional Loss Function
[0095] The positive matching ratio loss function is used to optimize the network's descriptive ability during training by minimizing the ratio between positive samples and the most difficult negative samples to increase the number of correct matches. The calculation formula is shown below:
[0096]
[0097] Where E(·) is the Euclidean distance function that measures the distance between two descriptors, c p This is a coefficient used to ignore samples that have already been well-differentiated; in this patent, it is set to 0.4, d. k and d′ k It is the descriptor of the corresponding point, d′ n In D′, besides matching externally with d, k The descriptor with the smallest Euclidean distance represents the most difficult counterexample; similarly, d n In D, besides matching externally with d′, k The descriptor with the smallest Euclidean distance.
[0098] (4-2) Mis-matching ratio loss function
[0099] The mis-matching ratio loss function is used to train the network by making the distance between the nearest neighbor and the second nearest neighbor of the points that are difficult to be correctly matched as large as possible to filter out by the ratio matching, become not matching instead of mis-matching, reduce the number of mis-matching, and further improve the matching accuracy. The calculation formula is as follows:
[0100]
[0101] Wherein, Q is the number of elements on the diagonal line which is not the minimum value of the corresponding row, J q,min1 and J q,min2 are the minimum value and the second minimum value of the row where these points that are prone to cause mis-matching are located, c n is a constant coefficient, which is set to 1 in this patent.
[0102] Embodiment 2
[0103] A dynamic feature extraction and description method based on multi-layer feature self-difference fusion, comprising:
[0104] The image to be processed is input into the feature extraction and description model constructed by the construction method of the dynamic feature extraction and description model based on multi-layer feature self-difference fusion of embodiment 1, to obtain the description sub-feature map and key point detection score map corresponding to the image to be processed.
[0105] Embodiment 3
[0106] An image matching method, as shown in Figure 5 For two images I1 and I2 to be matched, first input into the network model proposed in this patent to obtain the corresponding key point detection score maps S1 and S2 and description sub-feature maps F1 and F2, then perform non-maximum suppression on the two key point detection score maps and select the corresponding maximum K key points and According to these key points, the corresponding description sub-features are selected in the description sub-feature maps and Finally, the nearest neighbor and the second nearest neighbor ratio matching can be applied to the two description sub-features to obtain the matching relationship.
[0107] In summary, the dynamic feature extraction and description method based on the difference fusion of multi-layer features proposed in the application adopts a feature extraction module based on dynamic attention convolution to perform dense feature extraction on the image, uses dynamic convolution based on attention to enable the network to adaptively convolve parameters for different samples, which can improve the feature extraction capability of the network; a multi-layer feature self-difference detection module is used to generate a key point detection score map, the key point feature self-variance is used to express the saliency, and multi-layer fusion is used to increase the low-layer structure information, so that the key point distribution pays more attention to the texture area, which can improve the key point detection accuracy while reducing the false matching caused by the key point distribution in the non-texture area; an approximate key point distance loss function is used, and the softargmax approximation regression small patch extreme point coordinate method is used to solve the problem of lack of effective detection loss function caused by the non-differentiable key point coordinates, which can effectively improve the key point detection accuracy of the network, and further improve the matching accuracy; a positive matching and false matching ratio description loss function is used, based on the nearest neighbor and second nearest neighbor ratio matching method, the distance between the positive sample pair and the negative sample pair is optimized, the number of correct matching is increased, and the samples that are difficult to be correct matching are made to be unmatched as much as possible instead of false matching, which effectively improves the description performance of the network, and further improves the matching accuracy.
[0108] Those skilled in the art will easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion, characterized in that, The method comprises: S1, constructing a feature extraction and description model; wherein the feature extraction and description model comprises: a feature extraction module, configured to perform feature extraction on an input image to obtain feature maps at different levels; a multi-layer feature self-difference detection module, configured to calculate, according to a feature map at a certain level, a difference between each feature value in a feature dimension of each pixel and an average feature value, to obtain a feature self-difference score map at the corresponding level; and fuse feature self-difference score maps at several levels to obtain a key point detection score map; S2, calculating a loss function and training the feature extraction and description model by back propagation of the loss; The loss function comprises a key point detection accuracy loss function and a regression accuracy loss function, the key point detection accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of corresponding patches divided by a first key point detection score map and a second key point detection score map, wherein the first key point detection score map is a key point detection score map of a first image obtained by the feature extraction and description model, and the second key point detection score map is a key point detection score map of a second image mapped according to a true value matching map between the first image and the second image under the same scene; and the regression accuracy loss function represents an average distance between approximate derivable maximum value point coordinates obtained by softargmax regression of each patch divided by the key point detection score map and non-derivable maximum value point coordinates obtained by argmax. 2.The method of claim 1, wherein, The loss function further comprises a positive matching proportion loss function and a false matching proportion loss function, the positive matching proportion loss function is used to increase the number of correct matches by making the proportion between positive samples and the most difficult negative samples as small as possible when training the feature extraction and description model; and the false matching proportion loss function is used to filter out points difficult to be correct matches by making the distances between the nearest neighbor and the second nearest neighbor of the points difficult to be correct matches as large as possible when training the feature extraction and description model.
3. The method according to claim 2, wherein the positive matching proportion loss function is represented as: Let F and F' be the descriptor feature maps of the first image I and the second image I', select the K largest pixels in the first keypoint detection score map S as the key points, and select the descriptors of these key points in the descriptor feature map F, D = {d1, d2, d3, …, dK};At the same time, find the corresponding descriptors D' = {d'1, d'2, d'3, …, d'K} in the descriptor feature map F' according to the ground truth matching map M;Calculate the Euclidean distance between the descriptor set D and D' to obtain a distance matrix J with a size of K × K, the elements on the diagonal of the distance matrix J represent the descriptor distance of the corresponding point pair. K K};Calculate the Euclidean distance between the descriptor set D and D' to obtain a distance matrix J with a size of K × K, the elements on the diagonal of the distance matrix J represent the descriptor distance of the corresponding point pair. The false matching proportion loss function is represented as: where E(·) is the Euclidean distance function between two descriptors, c p is a constant coefficient to ignore the better samples that have been distinguished, d k and d′ k are the descriptors of the corresponding points, d′ n is the descriptor in D' with the smallest Euclidean distance to d k except the matching pair, d n is the descriptor in D with the smallest Euclidean distance to d′ k except the matching pair.
4. The method according to claim 1, wherein the key point detection accuracy loss function is represented as: where Q is the diagonal element of distance matrix J, J q,min1 and J q,min2 are the minimum and second minimum of the row in which the point is difficult to be correctly matched, c n is a constant coefficient. The regression accuracy loss function is represented as: Let S and S' be the keypoint detection score maps of the first image I and the second image I', let S' be the detection score map of the second image I' mapped from the ground truth matching map M, and let S and S' be the keypoint detection score maps of the first image I and the second image I' mapped from the ground truth matching map M. M M average score of the N patches. The feature extraction module comprises a plurality of standard convolution layers and dynamic attention convolution layers in cascade. Where, x n and y n The coordinates of the approximate differentiable maxima of the nth patch in the keypoint detection score map S, obtained through softargmax regression, are x. n′ and y n′ The key point detection score map S′ M The coordinates of the approximate differentiable maximum point of the nth patch are obtained by softargmax regression; The method comprises: wherein, and are the non-differentiable maximum point coordinates of the keypoint detection score map S obtained by argmax for the n-th patch, and are the non-differentiable maximum point coordinates of the keypoint detection score map S' obtained by argmax for the n-th patch. M the non-differentiable maximum point coordinates of the keypoint detection score map S obtained by argmax for the n-th patch.
5. The method of claim 1, wherein the method further comprises: inputting an image to be processed into a feature extraction and description model constructed by the method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion according to any one of claims 1 to 5, to obtain a descriptor feature map and a key point detection score map corresponding to the image to be processed.
6. A dynamic feature extraction and description method based on multi-layer feature self-difference fusion, characterized in that, The method comprises: 7. An image matching method characterized by, input two images to be matched into a feature extraction and description model constructed by the method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion according to any one of claims 1 to 5, to obtain respective corresponding descriptor feature maps and key point detection score maps; Non-maximum suppression is performed on the two keypoint detection score maps and the corresponding maximum K key points are selected and The corresponding descriptors are selected in the descriptor feature map according to the key points and two descriptors and The nearest neighbor and the second nearest neighbor ratio matching are applied to obtain the matching relationship.
8. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion according to any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion according to any one of claims 1 to 5. The computer readable storage medium comprises a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method for constructing a dynamic feature extraction and description model based on multi-layer feature self-difference fusion according to any one of claims 1 to 5.