A method and system for hetero image matching

Abstract

The application discloses a kind of heterogenous image matching method, system, belongs to visual navigation technical field.Firstly, reference image and multiple heterogenous measured images are acquired;Then, through the auto-encoder structure containing residual network, the feature extraction of heterogenous measured image is completed;Add the fusion strategy containing attention mechanism, determine the proportion of heterogenous measured image fusion information;Using dense connection and decoding network to retain intermediate layer features, image reconstruction is completed, and measured fusion image is obtained;Finally, the reference image and measured fusion image are matched by improved SIFT algorithm, and the final matching position is obtained.The application effectively reduces the information difference between heterogenous measured images by fusing the features of heterogenous measured images, improves the accuracy, robustness and real-time performance of heterogenous measured image matching.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of visual navigation, and particularly relates to a heterogeneous image matching method and system. BACKGROUND

[0002] A navigation system is an indispensable component of a modern aircraft, which can provide necessary data parameters for aircraft control and mission planning. At present, the relatively mature navigation methods include INS (Inertial Navigation System), GNSS (Global Navigation Satellite System) and visual navigation. Among them, the INS has the shortcomings of being unable to work for a long time and accumulating errors; the GNSS has the shortcomings of being susceptible to interference and not having autonomous navigation capability; the visual navigation, as an autonomous navigation technology, has the advantages of all-weather independent work, low cost and high precision, but has the problem of difficulty in image acquisition in specific environments.

[0003] With the development of imaging technology, it is possible to ensure stable image acquisition in various environments by carrying multiple sensors; meanwhile, the heterogeneous image information obtained by different imaging devices is more abundant, which is helpful to improve the adaptability and navigation precision of the visual navigation system to various complex navigation environments. However, since the imaging methods of the heterogeneous images are different, the image features are different, and it is difficult to directly use the traditional image matching algorithm to realize the matching of the heterogeneous images. How to obtain more effective information from the heterogeneous images, discard redundant information and construct common features becomes a technical difficulty of the heterogeneous image matching.

[0004] A large number of matching algorithms based on deep learning mainly use the learning ability of neural networks to construct feature points, style transfer, generative adversarial networks and other methods to complete image matching. Compared with the traditional matching method, this kind of method can construct more common features on the heterogeneous images or can directly output the image matching result through the learning ability of the network, but is limited by the data set in terms of matching precision and application scope.

[0005] Image fusion aims to extract and integrate the information of multiple images to obtain a fused image with more rich features. The fusion of multiple target real-time images can obtain more effective information from the heterogeneous images, discard redundant information, improve the utilization rate of image information, enhance the feature extraction in the navigation stage and improve the matching precision. At the same time, since the constructed image fusion network has a certain universality, it can achieve excellent fusion effect on various heterogeneous images, can construct common features with different data sets of heterogeneous images and realize image matching.

[0006] Existing fusion algorithms are roughly divided into two categories: traditional fusion algorithms and deep learning-based fusion algorithms. The traditional algorithm focuses on the fusion method based on scale transformation and sparse representation, but there are problems of large amount of calculation data, complex manual operation and low fusion precision. The deep learning-based fusion algorithm still has problems such as low utilization rate of intermediate layer features and shallow layer features, and complex network parameters. SUMMARY

[0007] The present application aims at the deficiencies of the prior art and provides a heterogeneous image matching method and system, which can effectively obtain more effective information from heterogeneous images, discard redundant information and construct common features.

[0008] In order to realize the above functions, the present application adopts the following technical solutions:

[0009] A heterogeneous image matching method, comprising the following steps:

[0010] S1, obtaining a reference image A0 from an airborne map database, and obtaining multiple heterogeneous measured images B1……Bn from a vehicle-mounted multi-type sensor. n .

[0011] S2, since the fused image needs to retain more effective features in the heterogeneous measured image and discard redundant features, the measured image needs to be first feature-extracted to obtain a feature set. A self-made data set and a combined Loss function are used for data iteration to obtain a self-encoder structure containing a residual network, the heterogeneous measured image is feature-extracted to obtain each level feature of the heterogeneous measured image.

[0012] S3, each level feature is enhanced by using a spatial attention mechanism and a channel attention mechanism, each level is weighted, the proportion of the heterogeneous measured image fusion information is determined, and each level double-attention-mechanism-enhanced fusion feature is obtained.

[0013] S4, each level fusion feature is constructed into the same size same level fusion feature through a decoding network, and is restored into a measured fusion image through dense connection and transposed convolution.

[0014] S5, using an improved SIFT algorithm of RANSAC algorithm, the measured fusion image and the reference image are image-matched to obtain the final matching position.

[0015] Further, in the step S2, the specific steps for obtaining each level feature of the heterogeneous measured image are:

[0016] S201, inputting a self-made data set image into a self-encoder structure for feature extraction.

[0017] S202, restoring the extracted features by using a decoding network to obtain a reconstructed image.

[0018] S203, judging the similarity of the reconstructed image and the self-made dataset image by using a similarity judgment function, thereby improving the feature extraction capability of the autoencoder structure. The similarity judgment function is composed of function Loss1 and function Loss2, and the specific formula is:

[0019] Loss1=MSE(in_img, out_img)

[0020] Loss2=1-msssim(in_img, out_img)

[0021]

[0022] Wherein, in_img represents the input self-made dataset image; out_img represents the reconstructed image; MSE function is a mean square error calculation function; msssim is a multi-scale structural similarity calculation function; M represents different scales, and m is the number of different scales; μ p , μ g represents the mean of the image; σ p , σ g represents the standard deviation of the image; σ pg represents the covariance between the two images; β m , γ m represents the importance between the two items; c1, c2 are non-zero constants to prevent the denominator from being 0.

[0023] Loss1 can represent the similarity by calculating the MSE (Mean Squared Error) of the input self-made dataset image and the reconstructed image, which well reflects the pixel gray change and overall intensity distribution of the image. The smaller the value of Loss1 is, the higher the similarity of the two images is.

[0024] Loss2 is biased towards the structure and edge features of the image, and the similarity of the image is judged by calculating the brightness, contrast and structural difference. The extraction capability of the network for edges and details and the anti-interference capability for pixel noise are enhanced. The smaller the value of Loss2 is, the higher the similarity of the two images is.

[0025] S204, adjust the proportion of Loss1 and Loss2 in Loss by combining the coefficient α, so that there is a relatively balanced constraint effect on the intensity change and structural change of the image, and the specific formula is:

[0026] Loss=Loss1+αLoss2.

[0027] The training process is iterated through data, so that the corresponding loss function reaches a minimum value, which proves that the input image and the reconstructed image have high similarity, and at this time the self-encoder structure with excellent feature extraction performance can be obtained.

[0028] S205, inputting the heterogeneous measured image into the self-encoder structure for feature extraction, the self-encoder structure being composed of 3, 4, 6, 3 residual modules to form a network backbone structure, and the residual modules being connected to output the intermediate layer features at the connection positions, so as to obtain the hierarchical features of the heterogeneous measured image.

[0029] Further, the specific steps of obtaining the hierarchical dual-attention mechanism enhanced fusion features in step S3 are as follows:

[0030] S301, obtaining the weights of the hierarchical features by using global average pooling to obtain a feature weight map of H*W.

[0031] S302, multiplying the feature weight map and the corresponding features to obtain the H*W*C dimensional image features after weight distribution.

[0032] S303, combining the features of multiple heterogeneous measured images to obtain the H*W*C dimensional spatial attention enhanced hierarchical features, and the calculation formula is as follows:

[0033] f spatial =spa_w1*feature1+spa_w2*feature2

[0034] Wherein, spa_w1 represents the spatial attention weight of the input image 1, spa_w2 represents the spatial attention weight of the input image 2, feature1 represents the image feature of the input image 1, and feature2 represents the image feature of the input image 2.

[0035] S304, using average pooling to obtain information data S by calculating the information in the channel dimension of the features, and the calculation formula is as follows:

[0036]

[0037] Wherein, H*W represents the total number of pixels of the image, represents each pixel point on the matrix of the image.

[0038] S305, obtaining a weight vector of C dimension according to the information data S, multiplying the weight vector and the corresponding features to obtain the H*W*C dimensional channel attention enhanced hierarchical features after weight distribution, and the calculation formula is as follows:

[0039] f channel= cha_w1*feature1 + cha_w2*feature2

[0040] Wherein, cha_w1 represents the spatial attention weight of input image 1, cha_w2 represents the spatial attention weight of input image 2, feature1 represents the image feature of input image 1, and feature2 represents the image feature of input image 2.

[0041] S306, the spatial attention enhanced each level feature and the channel attention enhanced each level feature are fused into each level fusion feature by weighted average, and the weight ratio is 1:1, and the calculation formula is as follows:

[0042] f fus = 0.5*f spatial + 0.5*f channel

[0043] Wherein, f spatial represents the image feature enhanced by the spatial attention module, and f channel represents the image feature enhanced by the channel attention module.

[0044] Further, in the step S4, the step of obtaining the fusion image is: the each level fusion feature is ensured to become the same level fusion feature with consistent size by upsampling, the intermediate layer feature is weighted to the final fusion result by dense connection, and then the fusion feature is restored to the normal size fusion image by deconvolution.

[0045] Further, in the step S5, the specific content of using the improved SIFT algorithm for image matching is: the reference image and the measured fusion image are feature detected, 128-dimensional feature description vectors are constructed, a feature point set is formed, the Euclidean distance is used to judge the similarity degree of the corresponding feature points, and when the nearest feature point and the second nearest feature point are within the specified distance, it is determined as a correct matching point; after obtaining the correct matching point, the random sample consensus method is used to remove the mis-matching points, and the final matching result is obtained.

[0046] Further, the application also provides a heterogeneous image matching system, comprising:

[0047] The image feature extraction module is used for data iteration by using self-made data set and combined Loss function, obtaining a self-encoder structure containing a residual network, extracting features of the heterogeneous measured image, and obtaining each level feature of the heterogeneous measured image.

[0048] The enhanced fusion feature extraction module is configured to enhance features of each level by using a spatial attention mechanism and a channel attention mechanism, to perform weighting of each level, to determine a proportion of the heterogeneous measured image fusion information, and to obtain the dual-attention-mechanism-enhanced fusion features of each level.

[0049] The measured image fusion module is configured to construct the fusion features of each level into the same-level fusion features with the same size by using a decoding network, and to restore the measured fusion image by using dense connection and transposed convolution.

[0050] The image matching module is configured to perform image matching on the measured fusion image and the reference image by using an improved SIFT algorithm based on the RANSAC algorithm, and to obtain the final matching position.

[0051] Further, in the image feature extraction module, the specific steps for obtaining the features of each level of the heterogeneous measured image are as follows:

[0052] Step 1: input the self-made dataset image into the autoencoder structure for feature extraction.

[0053] Step 2: restore the extracted features by using a decoding network to obtain a reconstructed image.

[0054] Step 3: determine the similarity between the reconstructed image and the self-made dataset image by using a similarity judgment function, and the similarity judgment function is composed of a function Loss1 and a function Loss2, and the specific formula is as follows:

[0055] Loss1 = MSE(in_img, out_img)

[0056] Loss2 = 1 - msssim(in_img, out_img)

[0057]

[0058] in_img represents the input self-made dataset image; out_img represents the reconstructed image; MSE function is a mean square error calculation function; msssim is a multi-scale structural similarity calculation function; M represents different scales, and m represents the number of different scales; μ p , μ g represents the mean of the image; σ p , σ g represents the standard deviation of the image; σ pg represents the covariance between the two images; β m , γ m represents the importance between the two items; c1 and c2 are non-zero constants to prevent the denominator from being zero.

[0059] Step 4, adjust the proportion of Loss1 and Loss2 in Loss by combining the coefficient a, so that there is a more balanced constraint effect on the intensity and structure changes of the image, and the specific formula is:

[0060] Loss = Loss1 + aLoss2

[0061] Through data iteration, the corresponding Loss function is minimized to obtain a self-encoder structure with feature extraction capability.

[0062] Step 5, input the heterogeneous measured image into the self-encoder structure for feature extraction, and the self-encoder structure is composed of 3, 4, 6, and 3 residual modules to form the network backbone structure. The intermediate layer features at the connection of the residual modules are reserved and output to obtain the hierarchical features of the heterogeneous measured image.

[0063] Further, in the enhanced fusion feature extraction module, the specific steps for obtaining the hierarchical double attention mechanism enhanced fusion features are as follows:

[0064] Step 1, use global average pooling to obtain the weight of each hierarchical feature to obtain a feature weight map of H*W.

[0065] Step 2, multiply the feature weight map and the corresponding feature to obtain the H*W*C dimensional image feature after weight distribution.

[0066] Step 3, combine the features of multiple heterogeneous measured images to obtain H*W*C dimensional spatial attention enhanced hierarchical features, and the calculation formula is as follows:

[0067] f spatial = spa_w1*feature1 + spa_w2*feature2

[0068] Where spa_w1 represents the spatial attention weight of input image 1, spa_w2 represents the spatial attention weight of input image 2, feature1 represents the image feature of input image 1, and feature2 represents the image feature of input image 2.

[0069] Step 4, use average pooling to calculate the information of the features in the channel dimension to obtain information data S, and the calculation formula is as follows:

[0070]

[0071] Where H*W represents the total number of pixels of the image, represents each pixel point on the matrix of the image.

[0072] Step 5, obtain the weight vector of C dimension according to the information amount data S, multiply the weight vector and the corresponding feature, obtain the H*W*C channel attention enhanced hierarchical feature after weight distribution, and the calculation formula is as follows:

[0073] f channel = cha_w1*feature1 + cha_w2*feature2

[0074] Wherein, cha_w1 represents the spatial attention weight of input image 1, cha_w2 represents the spatial attention weight of input image 2, feature1 represents the image feature of input image 1, and feature2 represents the image feature of input image 2.

[0075] Step 6, the spatial attention enhanced hierarchical feature and the channel attention enhanced hierarchical feature are fused into hierarchical fusion features by weighted average, and the weight ratio is 1:1, and the calculation formula is as follows:

[0076] f fus = 0.5*f spatial + 0.5*f channel

[0077] Wherein, f spatial represents the image feature enhanced by the spatial attention module, and f channel represents the image feature enhanced by the channel attention module.

[0078] Further, in the measured image fusion module, the specific content of obtaining the fusion image is that the hierarchical fusion features are ensured to be the same hierarchical fusion features with consistent size through upsampling, the intermediate layer features are weighted to the final fusion result through dense connection, and the fusion features are restored to the normal size of the fusion image through deconvolution.

[0079] Further, in the image matching module, the specific content of image matching is that the reference image and the measured fusion image are subjected to feature detection, 128-dimensional feature description vectors are constructed, a feature point set is formed, the Euclidean distance is used to judge the similarity of the corresponding feature points, and when the nearest feature point and the second nearest feature point are within the specified distance, it is determined as a correct matching point; after obtaining the correct matching point, the random sample consensus method is used to remove the mis-matching points, and the final matching result is obtained, the final matching result is subjected to navigation parameter solving, and finally the navigation information is obtained.

[0080] The above technical scheme is adopted in the application, compared with the prior art, and the significant technical effects are as follows:

[0081] In the coding structure, the residual network is used to construct the autoencoder structure, which enhances the deep learning ability of the network and can obtain deeper features; in the decoding process, the corresponding intermediate layer features are applied to the image recovery in the form of dense connection, more shallow features are retained; the deep features and shallow features of the image are retained in the subsequent fused image. The attention mechanism is introduced in the fusion strategy to optimize the information proportion, ensure the balanced distribution of the information of the heterogeneous images in the fused image, and better construct the common features in the matching process. By introducing the fused image, the feature difference between the heterogeneous images is greatly reduced, more effective information is retained, and a large amount of redundant information is discarded, so that the image matching has the advantages of large number of features, high matching precision and strong robustness. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 It is the overall step flowchart of the present application.

[0083] Figure 2 It is the flowchart of the present application for training the fused network with the autoencoder structure.

[0084] Figure 3 It is the schematic diagram of the heterogeneous image fusion network structure of the present application.

[0085] Figure 4 It is the schematic diagram of the fusion strategy of the present application with the attention mechanism.

[0086] Figure 5 It is the schematic diagram of the coding and decoding network structure of the present application.

[0087] Figure 6 It is the flowchart of the improved SIFT algorithm of the present application. DETAILED DESCRIPTION

[0088] The specific implementation technical solutions of the present application will be further described below in combination with the drawings:

[0089] The present application introduces the fused network with the autoencoder structure based on the residual network, obtains the fused image containing more effective information and discarding more redundant information, so that the proportion of the heterogeneous information fusion is more balanced; in addition, the imaging difference between the heterogeneous images is reduced by introducing the fused image in the image matching stage, more common features are constructed, and the accuracy, robustness and real-time performance of the image matching are improved.

[0090] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific implementation manners.

[0091] The present application provides a kind of heterogeneous image matching method, as shown in Figure 1 , comprising the following steps: ​

[0092] S1, obtain a reference image A0 from an airborne map database, and obtain two heterogeneous measured images B1...Bn from a multi-type sensor carried by an aircraft n .

[0093] S2, perform data iteration using a self-made data set and a combined Loss function to obtain an autoencoder structure containing a residual network, as shown in Figure 2 The specific steps are as follows:

[0094] S201, input the self-made data set image into the autoencoder structure for feature extraction.

[0095] S202, restore the extracted features using a decoding network to obtain a reconstructed image.

[0096] S203, judge the similarity between the reconstructed image and the self-made data set image using a similarity judgment function, which is composed of a function Loss1 and a function Loss2, and the specific formula is:

[0097] Loss1 = MSE(in_img, out_img)

[0098] Loss2 = 1 - msssim(in_img, out_img)

[0099]

[0100] Where in_img represents the input self-made data set image; out_img represents the reconstructed image; MSE function is a mean square error calculation function; msssim is a multi-scale structural similarity calculation function; M represents different scales, and m is the number of different scales; μ p , μ g represents the mean of the image; σ p , σ g represents the standard deviation of the image; σ pg represents the covariance between the two images; β m , γ m represents the importance between the two items; c1, c2 are non-zero constants to prevent the denominator from being zero.

[0101] The smaller the values of Loss1 and Loss2 are, the higher the similarity between the two images is.

[0102] S204, adjust the proportion of Loss1 and Loss2 in Loss by combining the coefficient α, so that there is a relatively balanced constraint effect on the intensity change and structural change of the image, and the specific formula is:

[0103] Loss = Loss1 + αLoss2.

[0104] During the training process, data iteration is used to minimize the corresponding Loss function. When α = 100, a good extraction result can be obtained, which proves that the input image and the reconstructed image have a high similarity. At this time, an autoencoder structure with excellent feature extraction performance can be obtained.

[0105] S205, such as Figure 3 As shown, the heterogeneous measured images B1...B n Input the autoencoder structure for feature extraction. The autoencoder structure consists of 3, 4, 6, and 3 residual modules to form a network backbone structure. The autoencoder extracts features from the heterogeneous measured image, and retains the intermediate layer features at the connection of the residual module and outputs them to obtain the features F of each level of the heterogeneous measured image. 11 ...F 1n , F 21 ...F 2n ...F n1 ...F nn .

[0106] S3, using the spatial attention mechanism and channel attention mechanism to enhance the features of each level, perform weighting on each level, determine the proportion of heterogeneous measured image fusion information, and obtain the enhanced fusion features of the dual attention mechanism at each level, such as Figure 4 As shown, the specific steps are:

[0107] S301. Use global average pooling to obtain the weights of features at each level and obtain a feature weight map H*W.

[0108] S302: Multiply the feature weight map and the corresponding feature to obtain the H*W*C-dimensional image feature after weight distribution.

[0109] S303: Combine the features of multiple heterogeneous measured images to obtain the H*W*C-dimensional spatial attention enhancement features at each level. The calculation formula is as follows:

[0110] f spatial =spa_w1*feature1+spa_w2*feature2

[0111] Among them, spa_w1 represents the spatial attention weight of input image 1, spa_w2 represents the spatial attention weight of input image 2, feature1 represents the image feature of input image 1, and feature2 represents the image feature of input image 2.

[0112] S304: Utilize the information of the average pooling statistical feature in the channel dimension to obtain information volume data S. The calculation formula is as follows:

[0113]

[0114] wherein H*W represents the total number of pixels of the image, represents each pixel point on the matrix of the image.

[0115] S305, obtain the weight vector of C dimension according to the information amount data S, multiply the weight vector and the corresponding feature, obtain the H*W*C dimension channel attention enhanced hierarchical feature after weight distribution, and the calculation formula is as follows:

[0116] f channel = cha_w1*feature1 + cha_w2*feature2

[0117] wherein cha_w1 represents the spatial attention weight of the input image 1, cha_w2 represents the spatial attention weight of the input image 2, feature1 represents the image feature of the input image 1, and feature2 represents the image feature of the input image 2.

[0118] S306, the spatial attention enhanced hierarchical feature and the channel attention enhanced hierarchical feature are fused into the hierarchical fusion feature through weighted average, and the weight ratio is 1:1, and the calculation formula is as follows:

[0119] f fus = 0.5*f spatial + 0.5*f channel

[0120] wherein f spatial represents the image feature enhanced through the spatial attention module, and f channel represents the image feature enhanced through the channel attention module.

[0121] S4, as shown in Figure 5 , the hierarchical fusion feature is constructed into the same hierarchical fusion feature with the same size through the decoding network, and is restored into the measured fusion image F1 through the dense connection and the transposed convolution, and the specific content is:

[0122] (1) Dense connection: the first layer intermediate feature obtained is weighted and transposed convolved with the up-sampled second layer intermediate feature result to obtain the fusion feature 1_1; the second layer intermediate feature obtained is weighted and transposed convolved with the up-sampled third layer intermediate feature result to obtain the fusion feature 2_1; the fusion feature 1_1, the up-sampled fusion feature 2_1 result and the first layer intermediate feature are weighted and transposed convolved. In this way, it is ensured that each time the feature is weighted, the hierarchical features of the previous layers are included, so that the details of the shallow features are better preserved.

[0123] (2) Transposed convolution: due to different degrees of down-sampling, the sizes of the corresponding intermediate layer features are different, and the transposed convolution is needed to convert the corresponding features of each layer to the features of the same layer.

[0124] S5, using the SIFT algorithm improved by the RANSAC algorithm, image matching is performed on the measured fusion image and the reference image to obtain the final matching position, and the specific content is as follows:

[0125] As shown in Figure 6 , the measured fusion image F1 and the reference image A0 are respectively constructed into a Gaussian pyramid and a Gaussian difference pyramid to realize scale space construction, and points whose positions do not change in the continuous space are found as key points. By fitting the Gaussian difference pyramid space, the accurate positions and scales of the extreme points are obtained, and the extreme points at the edge positions are removed to improve the stability of the key points. After the gradient of the key points is calculated, the gradient histogram is used to represent the direction information of the feature points, and finally a 128-dimensional description vector is obtained to construct a SIFT feature point set. The feature point sets M1 and M2 of the measured fusion image F1 and the reference image A0 are matched by setting the Euclidean distance threshold. When the Euclidean distance between the nearest feature point and the second nearest feature point is within the set threshold range, it is determined as a correct matching point; when the Euclidean distance between the nearest feature point and the second nearest feature point is outside the set threshold range, it is determined as an incorrect matching point; and the corresponding correct matching points are constructed into a coarse matching feature point set M.

[0126] The coarse matching feature point set M is further screened by the random sample consensus, and the point set determined as the correct matching point is constructed into a final matching point set R. The navigation parameters are solved through the final matching point set R to obtain the final matching position.

[0127] The embodiment of the present application also provides a heterogeneous image matching system, which comprises an image feature extraction module, an enhanced fusion feature extraction module, a measured image fusion module, an image matching module and a computer program capable of running on a processor. It should be noted that each module in the above system corresponds to the specific steps of the method provided by the embodiment of the present application, has the corresponding function modules and beneficial effects of the method. Technical details not described in detail in the embodiment can be referred to the method provided by the embodiment of the present application.

[0128] The above embodiments only illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical solution falls within the protection scope of the present application.

Claims

1. A method of heteroimage matching, characterized by, The application comprises the following steps: S1, obtaining a reference image and a plurality of heterogeneous measured images; S2, using a self-made data set and a combined loss function for data iteration to obtain a self-encoder structure containing a residual network, extracting features of the heterogeneous measured images, and obtaining hierarchical features of the heterogeneous measured images; S3, using a spatial attention mechanism and a channel attention mechanism to enhance the hierarchical features, performing hierarchical weighting, determining the proportion of the heterogeneous measured image fusion information, and obtaining hierarchical double-attention mechanism enhanced fusion features; Specifically, S301, obtaining the weight of the hierarchical features by using a global average pooling, and obtaining a feature weight map of H*W; S302, multiplying the feature weight map and the corresponding feature to obtain an H*W*C-dimensional image feature after weight distribution; S303, combining the features of the plurality of heterogeneous measured images to obtain H*W*C-dimensional spatial attention enhanced hierarchical features, and the calculation formula is as follows: f spatial = spa_w1 * feature1 + spa_w2 * feature2 Wherein, spa_w1 represents the spatial attention weight of the input image 1, spa_w2 represents the spatial attention weight of the input image 2, feature1 represents the image feature of the input image 1, and feature2 represents the image feature of the input image 2; S304, using the average pooling to calculate the information of the features in the channel dimension to obtain information data S, and the calculation formula is as follows: where H*W represents the total number of pixels of the image, represents each pixel point on the matrix of the image; S305, obtaining a weight vector of C dimension according to the information data S, multiplying the weight vector and the corresponding feature channel, and obtaining an H*W*C-dimensional channel attention enhanced hierarchical feature after weight distribution, and the calculation formula is as follows: f channel = cha_w1 * feature1 + cha_w2 * feature2 Wherein, cha_w1 represents the spatial attention weight of the input image 1, cha_w2 represents the spatial attention weight of the input image 2, feature1 represents the image feature of the input image 1, and feature2 represents the image feature of the input image 2; S306, fusing the spatial attention enhanced hierarchical features and the channel attention enhanced hierarchical features into hierarchical fusion features through weighted average fusion, and the weight ratio is 1:1, and the calculation formula is as follows: f fus = 0.5 * f spatial + 0.5 * f channel wherein f spatial represents the image feature enhanced by the spatial attention module, f channel represents the image feature enhanced by the channel attention module; S4, using a decoding network to construct the hierarchical fusion features into same-level fusion features with the same size, and restoring them into measured fusion images through dense connection and transposed convolution; S5, using an improved SIFT algorithm based on RANSAC algorithm to perform image matching on the measured fusion image and the reference image to obtain the final matching position.

2. The heterogenous image matching method of claim 1, wherein, In the step S2, the specific steps for obtaining the hierarchical features of the heterogeneous measured images are as follows: S201, inputting the self-made data set image into the self-encoder structure for feature extraction; S202, restoring the extracted features by using a decoding network to obtain a reconstructed image; S203, using a similarity judgment function to judge the similarity of the reconstructed image and the self-made data set image, and the similarity judgment function is composed of a function Loss1 and a function Loss2, and the specific formula is as follows: Loss1=MSE(in_img,out_img) Loss2=1-msssim(in_img,out_img) Wherein, in_img represents the input self-made data set image; out_img represents the reconstructed image; the MSE function is the mean square error calculation function; msssim is the multi-scale structural similarity calculation function; M represents different scales, and m is the number of different scales; μ p , μ g represents the mean of the image; σ p , σ g represents the standard deviation of the image; σ pg represents the covariance between the two images; β m , γ m represents the importance between the two items; c1, c2 are non-zero constants to prevent the denominator from being 0; S204, adjust the proportion of Loss1 and Loss2 in Loss by combining the coefficient a, and the specific formula is: Loss = Loss1 + aLoss2; Through data iteration, the corresponding Loss function is minimized to obtain a self-encoder structure with feature extraction capability; S205, the self-encoder structure is composed of 3, 4, 6, 3 residual modules to form a network backbone structure, the heterogeneous measured images are input into the self-encoder structure for feature extraction, and the intermediate layer features at the connection of the residual modules are reserved and output to obtain the hierarchical features of the heterogeneous measured images.

3. The heterogenous image matching method of claim 1, wherein, In step S4, the specific content of obtaining the fused image is: the hierarchical fusion features are ensured to become the same size of the same level fusion features through upsampling, the intermediate layer features are weighted to the final fusion result through dense connection, and then the fusion features are restored to the normal size of the fusion image through deconvolution.

4. The method of claim 1, wherein, In step S5, the specific content of using the improved SIFT algorithm for image matching is: the reference image and the measured fusion image are feature detected, 128-dimensional feature description vectors are constructed, a feature point set is formed, the Euclidean distance is used to judge the similarity of the corresponding feature points in the feature point set, and if the nearest feature point and the second nearest feature point are within a specified distance, it is determined as a correct matching point; After obtaining the correct matching points, the false matching points are removed by the random sample consensus method to obtain the final matching result.

5. A system for matching images from different sources, characterized in that Comprise: An image feature extraction module is configured to use a self-made data set and a combined Loss function to perform data iteration, obtain a self-encoder structure containing a residual network, perform feature extraction on heterogeneous measured images, and obtain hierarchical features of the heterogeneous measured images; An enhanced fusion feature extraction module is configured to use a spatial attention mechanism and a channel attention mechanism to enhance the hierarchical features, perform hierarchical weighting, determine the proportion of fusion information of the heterogeneous measured images, and obtain enhanced fusion features of the hierarchical dual attention mechanism; Specifically: Step 1, use global average pooling to obtain the weight of each level feature to obtain a H*W feature weight map; Step 2, multiply the feature weight map and the corresponding feature to obtain a H*W*C image feature after weight allocation; Step 3, combine the features of multiple heterogeneous measured images to obtain H*W*C spatial attention enhanced hierarchical features, and the calculation formula is as follows: f spatial = spa_w1 * feature1 + spa_w2 * feature2 Wherein, spa_w1 represents the spatial attention weight of input image 1, spa_w2 represents the spatial attention weight of input image 2, feature1 represents the image feature of input image 1, and feature2 represents the image feature of input image 2; Step 4, use average pooling to calculate the information of the features in the channel dimension to obtain information data S, and the calculation formula is as follows: where H*W represents the total number of pixels of the image, represents each pixel point on the matrix of the image; Step 5, obtain a C-dimensional weight vector according to the information data S, multiply the weight vector and the corresponding feature to obtain a H*W*C channel attention enhanced hierarchical feature after weight allocation, and the calculation formula is as follows: f channel = cha_w1 * feature1 + cha_w2 * feature2 Wherein, cha_w1 represents the spatial attention weight of the input image 1, cha_w2 represents the spatial attention weight of the input image 2, feature1 represents the image feature of the input image 1, and feature2 represents the image feature of the input image 2; Step 6, the spatial attention enhanced hierarchical features and the channel attention enhanced hierarchical features are fused into hierarchical fusion features by weighted average with a weight ratio of 1:1, and the calculation formula is as follows: f fus = 0.5 * f spatial + 0.5 * f channel wherein f spatial represents the image feature enhanced by the spatial attention module, f channel represents the image feature enhanced by the channel attention module; The measured image fusion module is configured to use a decoding network to construct the hierarchical fusion features into same-level fusion features with the same size, and restore them into a measured fusion image through dense connection and transposed convolution. The image matching module is configured to use an improved SIFT algorithm based on RANSAC algorithm to perform image matching on the measured fusion image and the reference image to obtain the final matching position.

6. The heterogenous image matching system of claim 5, wherein, In the image feature extraction module, the specific steps for obtaining the hierarchical features of the heterogeneous measured image are as follows: Step 1, input the self-made dataset image into the autoencoder structure for feature extraction; Step 2, use a decoding network to restore the extracted features to obtain a reconstructed image; Step 3, use a similarity judgment function to judge the similarity of the reconstructed image and the self-made dataset image, and the similarity judgment function is composed of function Loss1 and function Loss2, and the specific formula is as follows: Loss1 = MSE(in_img, out_img) Loss2 = 1-msssim(in_img, out_img) Wherein, in_img represents the input self-made data set image; out_img represents the reconstructed image; the MSE function is a mean square error calculation function; msssim is a multi-scale structural similarity calculation function; M represents different scales, and m is the number of different scales; μ p , μ g represents the mean of the image; σ p , σ g represents the standard deviation of the image; σ pg represents the covariance between the two images; β m , γ m represents the importance between the two terms; c1, c2 are non-zero constants to prevent the denominator from being 0; Step 4, adjust the proportion of Loss1 and Loss2 in Loss by combining the coefficient α, and the specific formula is as follows: Loss = Loss1 + αLoss2 Through data iteration, the corresponding Loss function reaches the minimum value, and the autoencoder structure with feature extraction capability is obtained; Step 5, the autoencoder structure is composed of 3, 4, 6, and 3 residual modules to form a network backbone structure, the heterogeneous measured image is input into the autoencoder structure for feature extraction, and the intermediate layer features at the connection of the residual modules are reserved and output to obtain the hierarchical features of the heterogeneous measured image.

7. The heterogenous image matching system of claim 5, wherein, In the measured image fusion module, the specific content for obtaining the fusion image is as follows: the hierarchical fusion features are converted into same-level fusion features with consistent size by upsampling, the intermediate layer features are weighted to the final fusion result by dense connection, and the fusion features are restored to a normal size fusion image by deconvolution.

8. The heterogenous image matching system of claim 5, wherein, In the image matching module, the specific content of image matching is as follows: the reference image and the measured fusion image are subjected to feature detection to construct 128-dimensional feature description vectors and form a feature point set, and the Euclidean distance is used to judge the similarity of the corresponding feature points in the feature point set; when the nearest feature point and the second nearest feature point are within a specified distance, it is determined as a correct matching point; After obtaining the correct matching points, the random sample consensus method is used to remove the false matching points to obtain the final matching result.

Images