Multi-modal image matching method and system for three-dimensional phase orientation invariant feature transformation

Abstract

The invention discloses a multi-modal image matching method and system for three-dimensional phase orientation invariant feature transformation, and belongs to the field of image processing. The method comprises the following steps of: firstly, constructing a multi-direction difference graph through a difference filter, then calculating three-dimensional phase tensor characteristics by utilizing a three-dimensional phase consistency model, and further calculating a phase difference graph; thirdly, calculating the maximum direction value of the phase difference diagram to obtain a significant difference phase feature; and finally, obtaining feature points in the differential phase torque space. Then, optimal feature indexing is carried out under multi-scale and multi-direction three-dimensional phase features, the positions of feature points are positioned, a histogram of oriented gradients of local areas of the feature points is calculated and counted, and then a three-dimensional phase orientation feature descriptor is constructed; finally, Euclidean distance is adopted as a matching measure, homonymy points are obtained by describing the ratio of the nearest neighbor distance to the next nearest neighbor distance through three-dimensional phase orientation features, mismatching points are removed through a random sampling consistency algorithm, and multi-modal image matching is completed.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and particularly relates to a multi-modal image matching method and system based on three-dimensional phase orientation invariant feature transformation. Background Art

[0002] Remote sensing image matching plays a crucial role in the fields of computer vision and photogrammetry. As the basis for key technologies such as image fusion, change detection, positioning and navigation, and three-dimensional reconstruction, it provides strong support for various remote sensing applications. With the increasing demand for remote sensing applications, a single data source can no longer meet the processing requirements of complex geographical scenes, and integrating images from different data sources has gradually become a new research hotspot. In recent years, imaging technologies have developed rapidly, and new sensors such as visible light cameras, infrared cameras, synthetic aperture radar (SAR), etc. have emerged continuously, making the acquisition of multi-modal remote sensing images more diverse and covering various surface information and environmental changes. However, with the increase in data modalities, the huge differences between different modalities further increase the matching difficulty. Therefore, how to effectively eliminate the differences between modalities and improve the accuracy and robustness of image alignment has become the key to multi-modal image collaborative processing.

[0003] The core task of multi-modal image matching is to overcome the matching problems caused by differences in sensor types, shooting angles, lighting conditions, etc. between different modalities to achieve high-precision image alignment. The specific challenges faced by multi-modal image matching include: 1. Geometric differences, imaging angles, sensor characteristics, and ground object changes may lead to differences in the geometric shapes and spatial positions of images; 2. Spectral differences, there are significant differences in spectral responses between different modalities of remote sensing images such as optical and radar, and the same ground object may exhibit different characteristics in different modalities; 3. Noise and non-linear radiation distortion will lead to the loss of image details and color distortion, increasing the interference in the feature extraction process. In response to these difficulties, experts and scholars have proposed a series of solutions. For example, the scale-invariant feature transform (SIFT) algorithm proposed by Lowe et al. effectively alleviates the geometric difference problem by extracting scale and rotation invariant features. However, such methods still have deficiencies in dealing with radiation differences, lighting differences, and non-linear radiation distortion. Therefore, researchers have begun to explore the phase features in the image frequency domain, and have successfully improved geometric and radiation differences through methods such as matching optimized by Log-Gabor filtering (LGHD), extended phase correlation algorithm (LGEPC), and histogram of oriented phase consistency (HOPC). However, in the face of complex situations such as missing texture information, noise interference, or non-linear radiation distortion, these methods may still lead to matching failures. Deep learning methods, with their ability to learn high-level features, can effectively handle non-linear matching problems, but due to the current lack of multi-modal data samples, their matching generalization is limited to a certain extent. Summary of the Invention

[0004] To solve the problems existing in the background technology, the present invention proposes a multi-modal image matching method based on three-dimensional phase orientation invariant feature transformation, aiming to effectively cope with geometric and radiometric differences. This method combines a three-dimensional phase difference map and a three-dimensional phase coherence orientation histogram. First, the present invention innovatively proposes a differential phase feature extraction technique. By using a differential filter to construct differential maps in multiple directions and combining with a three-dimensional phase coherence model to calculate three-dimensional phase features. The three-dimensional phase difference map can effectively weaken the spectral differences between multi-modal images, thus significantly improving the ability to extract feature points. Second, the present invention designs a three-dimensional phase coherence orientation histogram. By constructing an optimal index map of three-dimensional phase coherence with multiple scales and multiple directions, the matching stability is further enhanced. In particular, when dealing with situations such as depth maps or electronic maps where the modal differences are too large, the noise interference is severe, and the texture information is missing, the three-dimensional phase coherence orientation histogram can effectively capture the local and global geometric characteristics of the image, ensure the invariance of features, and improve the matching accuracy.

[0005] Through the innovative method of the present invention, the multi-modal image matching method based on three-dimensional phase orientation invariant feature transformation effectively overcomes the geometric and radiometric difference problems in multi-modal image matching, and solves the challenges brought by the lack of texture details, noise interference, and non-linear radiometric distortion, providing strong technical support for the precise alignment, collaborative processing, and optimization of multi-modal images.

[0006] The present invention proposes a multi-modal image matching method based on three-dimensional phase orientation invariant feature transformation to solve the matching problem of multi-modal images.

[0007] The technical solution adopted by the present invention is as follows: The multi-modal image matching method based on three-dimensional phase orientation invariant feature transformation includes the following steps:

[0008] Step 1, initialize the calculation parameters for multi-modal image matching, and perform non-linear diffusion on the multi-modal images;

[0009] Step 2, calculate a multi-channel differential gradient feature map using the non-linear diffusion result;

[0010] Step 3, perform image domain calculations on the multi-channel differential gradient feature map, and use the three-dimensional phase coherence model to estimate the image phase features, thereby generating three-dimensional phase features;

[0011] Step 4, perform Gaussian difference filtering on the three-dimensional phase features, and statistically calculate the maximum eigenvalue in each channel to generate a significant differential phase feature map, and extract feature points from the significant differential phase feature map;

[0012] Step 5: Decompose the three-dimensional phase amplitude feature in the three-dimensional phase feature, and count the feature intensity of each channel, record the optimal feature index, and obtain the optimal feature index map;

[0013] Step 6: Statistically analyze the histogram of oriented gradients feature in the neighborhood of feature points in the optimal feature index map, and then construct the three-dimensional phase orientation feature descriptor vector feature;

[0014] Step 7: Perform similarity measurement on the three-dimensional phase orientation feature descriptor vector for the nearest neighbor and the second nearest neighbor distances, establish the initial correspondence through a two-way matching strategy, and then identify the correct homologous point pairs through the gross error rejection method to complete the multi-modal image matching.

[0015] Furthermore, in Step 1, initialize the number of layers in the image scale space, the number of feature points, the neighborhood window of the feature point descriptor, and the gross error rejection threshold parameter.

[0016] Furthermore, in Step 2, the multi-channel differential gradient feature map is defined as:

[0017] G r,c,o =|cos o*Gx r,c,o +sin o*Gy r,c,o |

[0018] where G r,c,o represents the multi-channel differential gradient feature map; r is the number of rows of the image, c is the number of columns of the image, o is the direction, No is the number of directions, Gx r,c,o is the gradient feature in the x direction, and Gy r,c,o is the gradient feature in the y direction.

[0019] Furthermore, in Step 3, the mathematical expression of the three-dimensional phase consistency model is defined as:

[0020]

[0021] In the formula, PC ρ,Φ (l) represents the gradient phase feature result under the three-dimensional phase consistency model, ω Φ (l) is the weight factor for frequency expansion, l represents polar coordinates, A ρ,Φ (l) is the filtering convolution response amplitude of the three-dimensional Log-Gabor filter at position l under the scale ρ and the three-dimensional spatial direction Φ, A ρ,Φ (l)=|EO ρ,Φ |, EO ρ,Φ is the complex response of the three-dimensional Log-Gabor filter under the scale ρ and the three-dimensional spatial direction Φ, ΔΨ ρ,Φ (l) represents the phase deviation at l, scale ρ and direction Φ, which is used to measure the significance of the local phase, T is a noise threshold, and ε is a very small factor to prevent division by zero;

[0022]

[0023] Among them, and are the Fourier transform and the inverse Fourier transform respectively, and filter ρ,o is a frequency domain filter in the scale ρ and the three-dimensional spatial direction o, and is used to extract the frequency components of the image at a specific scale and direction.

[0024] Furthermore, the specific implementation method in step 4 is as follows.

[0025] First, perform the summation of three-dimensional phase orientation features, and the specific mathematical expression is as follows:

[0026]

[0027] Among them, is a three-dimensional tensor, representing the phase eigenvalue at the position (x, y), the direction j, and the channel k. j is the direction index, representing the directionality of the feature, k is the channel index, usually representing different feature channels or filter responses, and K represents the number of channels; then use a three-dimensional Gaussian filter to smooth the three-dimensional phase features to generate a smoothed feature map PC3D_m;

[0028] Generate Dog_m1 by using the difference before and after Gaussian smoothing to capture the significant changes of the features;

[0029] Dog_m1(x, y, j) = PC3D_m1(x, y, j) - PC_m1(x, y, j)

[0030] Select the maximum value of each pixel point in all directions in the phase difference map as the final feature response PC_m1, and perform normalization processing on it to obtain the locally most significant direction feature;

[0031] PC_m1(x, y) = maxDog_m1(x, y, j)

[0032]

[0033] Among them, S Dog represents the significant difference phase feature map, min(·) and max(·) represent the minimum value and maximum value calculation functions respectively; perform feature point extraction in the significant difference phase feature map, including FSAT and Harris detection.

[0034] Furthermore, the specific implementation of step 5 includes:

[0035] First, for the calculation of the three-dimensional phase feature convolution sequence, the two-dimensional maximum direction phase feature maps at each scale ρ are amplitude-added. The specific formula is as follows:

[0036]

[0037] In the formula, CS(x, y, j) represents the sequence accumulation value of the phase feature tensor at the position of pixel (x, y) and on the j-th layer channel; EO i,j (x, y, z) represents the value of the i-th input feature map on the j-th layer, with z channels, and ns is the number of feature maps calculated for each j channel; max(·) represents taking the maximum value;

[0038] After amplitude-adding the two-dimensional maximum direction phase feature results at each scale ρ, the maximum value of the amplitude in different directions is statistically calculated to complete the optimal feature index BoIM of the three-dimensional phase feature. The specific mathematical expression is as follows:

[0039] MI CS (x, y) = argmax j CS(x, y, j)

[0040] MI CS (x, y) represents the optimal feature index result of the three-dimensional phase feature.

[0041] Furthermore, the specific implementation method of step 6 is as follows;

[0042] In the optimal feature index BoIM, the method of the histogram of oriented gradients descriptor is used to calculate the feature vector; for each feature point, a region of size K×K is determined and divided into a grid of K×K. In each grid, the pixel histogram of the BoIM map is calculated, and it is assigned to N directions according to the direction of the pixel value; each small region generates an N-dimensional feature vector, representing the histogram information of the N directions of this region. Therefore, the descriptor vector of each feature point is K×K×N-dimensional and contains significant information related to the feature point.

[0043] Furthermore, the Euclidean distance is used for similarity measurement in step 7, and the two-way matching strategy is used to ensure the one-to-one correspondence of the matching points, and the random sample consensus algorithm is used to eliminate the wrong matches to achieve stable matching of multi-modal images.

[0044] Furthermore, it also includes step 8, using the correct homologous points to evaluate the matching effect of the multi-modal images. For each pair of images, the root mean square error RMSE of the homologous points and the number of matching homologous point pairs are used for quantitative inspection, where the unit of RMSE is pixel.

[0045] The present invention also provides a multimodal image matching system for three-dimensional phase orientation invariant feature transformation, comprising:

[0046] A processor and a memory, where the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multimodal image matching method for three-dimensional phase orientation invariant feature transformation as described in the above technical solution.

[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0048] The present invention proposes a multimodal image matching method for three-dimensional phase orientation invariant feature transformation. By introducing an innovative differential phase feature extraction technology and using multi-directional differential filters and three-dimensional phase feature calculation, the spectral differences between multimodal images are effectively reduced, and the ability to extract feature points is significantly improved. In addition, the designed three-dimensional phase consistency orientation histogram enhances the feature expression ability while maintaining geometric consistency, and is particularly suitable for multimodal images with weak texture and missing details. The results show that the method proposed by the present invention can better achieve stable matching of multimodal images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 : Flowchart of the method of the present invention;

[0050] Figure 2 : Schematic diagram of feature point extraction of the saliency differential phase feature map;

[0051] Figure 3 : Schematic diagram of optimal feature index map calculation;

[0052] Figure 4 : Multimodal image dataset, where (a) is a multi-temporal image, (b) is an infrared image and an optical image, (c) is an electronic navigation map and an optical image, and (d) is a SAR image and an optical image;

[0053] Figure 5 : Multimodal image matching results, where (a) is the matching result of the multi-temporal image, (b) is the matching result of the infrared image and the optical image, (c) is the matching result of the electronic navigation map and the optical image, and (d) is the matching result of the SAR image and the optical image. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0055] Please refer to Figure 1Flowchart. A multimodal matching method for three-dimensional phase orientation invariant feature transformation provided by the present invention includes the following steps:

[0056] Step 1: Initialize the calculation parameters for multimodal image matching, and perform non-linear diffusion on the multimodal image;

[0057] Preferably, in Step 1, parameters such as the number of layers in the image scale space, the number of feature points, the neighborhood window of the feature point descriptor, and the gross error rejection threshold are initialized. According to a large amount of experimental experience, the number of layers in the scale space is 4 layers, the number of feature points is set to 3500, the neighborhood window of the feature point descriptor is set to 96 pixels, and the gross error rejection threshold is set to 3 pixels.

[0058] Step 2: Use the non-linear diffusion result to perform Gaussian filtering to create an image scale space, calculate the multi-channel image gradient map, and perform differential feature calculation to obtain a multi-channel differential gradient feature map.

[0059] Preferably, in Step 2, a set of gradually smoothed image gradient maps are generated from the non-linear diffusion result image. The Gaussian filter and its derivative help the image suppress details and extract features through smoothing operations and multi-scale representations, generating smoothed images of different scales. The window size W of the Gaussian filter is determined by the standard deviation σ and is defined as:

[0060] W = 2 * |2 * σ| + 1 (1)

[0061] Based on the image after Gaussian filtering, multi-channel Sobel gradient feature calculation is performed. Define the gradient filtering template H in the x direction as:

[0062]

[0063] Set the filtering template in the y direction as the transpose of the x direction. After using the Sobel filtering template to calculate the differential gradient maps of the image in the x direction and the y direction respectively, with the x direction as the reference (usually defined as the horizontal direction, i.e., the 0-degree direction), the gradient direction o is divided into several discrete directions. The number of directions No determines the division granularity of the directions, and 360 degrees is evenly divided into No directions:

[0064]

[0065] Then the multi-channel gradient feature is defined as:

[0066] G r,c,o = |cos o * Gx r,c,o + sin o * Gy r,c,o | (4)

[0067] where G r,c,orepresents a multi-channel gradient feature map; where r is the number of rows of the image, c is the number of columns of the image, o is the direction, No is the number of directions, Gx r,c,o is the gradient feature in the x direction, Gy r,c,o is the gradient feature in the y direction.

[0068] Step 3: Perform image domain calculation on the multi-channel differential gradient feature map using Fourier transform, and estimate the image phase feature using a three-dimensional phase consistency model, thereby generating a three-dimensional phase feature.

[0069] Preferably, the three-dimensional phase consistency model first completes multi-scale filtering, combines the amplitude and phase information of the filtering response, calculates the three-dimensional phase feature of the image, and enhances the feature saliency through noise suppression and directional weight adjustment. To better calculate the three-dimensional phase feature, it is assumed that given the row number r and column number c of a known image point and determining a plane direction o, the corresponding gradient value G can be found in three-dimensional space r,c,o , and it can also be determined by the scale ρ in polar coordinates and the three-dimensional space direction Φ (Φ consists of the elevation angle θ and azimuth angle δ on a sphere with a unit radius).

[0070] Among them, the key to calculating the three-dimensional phase feature is to introduce three-dimensional Log-Gabor filtering, which is similar to using the phase consistency model for feature detection in two-dimensional Log-Gabor filtering. The present invention extends from the two-dimensional phase consistency model to three-dimensional and constructs a three-dimensional phase consistency model. In order to better load the multi-channel image gradient map as input into the three-dimensional phase consistency model, the multi-directional phase consistency feature map of the phase consistency model and the convolution response value of the three-dimensional Log-Gabor filtering are calculated respectively. The mathematical expression of the three-dimensional phase consistency model is defined as:

[0071]

[0072] In the formula, PC ρ,Φ (l) represents the gradient phase feature result under the three-dimensional phase consistency model, ω Φ (l) is the weight factor for frequency expansion, l represents polar coordinates, A ρ,Φ (l) is the filtering convolution response amplitude A of the three-dimensional Log-Gabor filter at position l under the scale ρ and three-dimensional space direction Φ ρ,Φ (l) = |EO ρ,Φ |, EO ρ,Φ is the complex response of the three-dimensional Log-Gabor filter under the scale ρ and direction Φ, and its amplitude A ρ,Φ (l) = ∣EO ρ,Φ ∣ represents the local signal intensity; ΔΨ ρ,Φ$(l)$ represents the phase deviation at scale $\rho$ and direction $\varPhi$ for measuring the significance of local phase. $T$ is a noise threshold, and $\varepsilon$ is a very small factor to prevent division by zero.

[0073]

[0074] Among them, and are the Fourier transform and the inverse Fourier transform respectively. filter ρ,o is a frequency-domain filter at scale $\rho$ and direction $o$ (such as a Log-Gabor filter) for extracting the frequency components of an image at a specific scale and direction.

[0075] Step 4: Perform Gaussian difference filtering using the three-dimensional phase features, and statistically calculate the maximum eigenvalue in each channel to generate a significant difference phase feature map. Extract feature points using the FAST feature detector in the significant difference phase feature map.

[0076] Preferably, after processing the multi-channel gradient features using the three-dimensional phase consistency model, in order to extract more significant features and describe them, first obtain the three-dimensional phase tensor features, and then perform three-dimensional Gaussian convolution on them. Then, subtract the feature tensor after Gaussian convolution from the feature tensor before convolution to obtain a three-dimensional Gaussian difference phase map. According to the number of division directions $N_o$, the dimension of the three-dimensional Gaussian difference phase map is $r*c*N_o$. Finally, take the maximum value of the three-dimensional Gaussian difference phase map along the $N_o$ direction to obtain the final significant difference phase feature map.

[0077] First, perform the summation of three-dimensional phase orientation features. The specific mathematical expression is as follows:

[0078]

[0079] Among them, is a three-dimensional tensor representing the phase eigenvalue at position $(x,y)$, direction $j$, and channel $k$. $j$ is the direction index representing the directionality of the feature (for example, $j$ can be from 1 to $N_o$, where $N_o$ is the number of directions), $k$ is the channel index, usually representing different feature channels or filter responses, and $K$ represents the number of channels (in the embodiment of the present invention, the range of $k$ is from 1 to 6). To reduce noise interference, use a three-dimensional Gaussian filter to smooth the three-dimensional phase features to generate a smoothed feature map $PC3D\_m$.

[0080] Generate $Dog\_m1$ by taking the difference between before and after Gaussian smoothing to capture the significant change of the feature.

[0081] $Dog\_m1(x,y,j)=PC3D\_m1(x,y,j)-PC\_m1(x,y,j)$ (8)

[0082] In the phase difference map, select the maximum value of each pixel point in all directions as the final feature response PC_m1, and perform normalization processing on it to obtain the locally most significant directional feature.

[0083]

[0084] Among them, S Dog represents the significant difference phase feature map, and min(·) and max(·) respectively represent the minimum value and maximum value calculation functions. Feature point extraction is performed on the significant difference phase feature map. In the present invention, the FSAT detector is used for feature point extraction, as Figure 2 shown.

[0085] Step 5: Decompose the three-dimensional phase amplitude feature in the three-dimensional phase feature, and count the feature intensity of each channel, record the optimal feature index, and obtain the optimal feature index map.

[0086] Preferably, when calculating the three-dimensional phase consistency model, a set of three-dimensional orthogonal bases and can be obtained for each position l(ρ, Φ) in the image, and the spectral feature EQ(e ρ,Φ , q ρ,Φ ) is obtained by combining the multi-directional differential gradient map. At scale ρ and three-dimensional space direction Φ, a three-dimensional phase feature tensor with dimensions of r*c*No is obtained, and then the maximum value along the o direction is statistically calculated, that is, the two-dimensional maximum direction phase feature is obtained.

[0087] First, for the calculation of the three-dimensional phase feature convolution sequence, the two-dimensional maximum direction phase feature maps at each scale ρ are amplitude-added, and the specific formula is as follows:

[0088]

[0089] In the formula, CS(x, y, j) represents the sequence accumulation value of the phase feature tensor at the position of pixel (x, y) and the j-th layer channel; EO i,j (x, y, z) represents the value of the i-th input feature map at the j-th layer, with z channels; EO i,j (x, y, z) is the complex response value evolved from EO ρ,Φ , with the z-channel dimension added, representing multi-channel feature information. Its role in the formula is to generate the sequence accumulation value of the phase feature tensor by taking the maximum value of the z channels and accumulating. ns is the number of feature maps calculated for each j channel; max(·) represents taking the maximum value.

[0090] After adding the amplitudes of the two-dimensional maximum direction phase feature results at each scale ρ and then statistically calculating the maximum amplitude in different directions, the optimal feature index BoIM (Best-optimal Index Map) of the three-dimensional phase feature is completed. The specific mathematical expression is as follows:

[0091] MI CS (x,y) = argmax j CS(x,y,j) (12)

[0092] MI CS (x,y) represents the optimal feature index result of the three-dimensional phase feature. The specific process is as Figure 3 shown.

[0093] Step 6: Statistically calculate the histogram of oriented gradients of the feature point neighborhood in the optimal feature index map, and then construct a three-dimensional phase orientation feature descriptor vector;

[0094] Preferably, in BoIM, the method of the histogram of oriented gradients descriptor is used to calculate the feature vector. For each feature point, a region of size K×K is determined and divided into a grid of K×K (in the present invention, the grid is set to 8×8 according to existing experience). In each grid, the pixel histogram of the BoIM map is calculated, and it is assigned to 6 directions according to the direction of the pixel value (from the 0-degree direction to the 360-degree direction. If 6 directions are set, 1 corresponds to 60 degrees, 2 corresponds to 120 degrees, and so on, 6 corresponds to 360 degrees). Each small region generates a 6-dimensional feature vector, representing the histogram information of the 6 directions of this region. Therefore, the descriptor vector of each feature point is 384-dimensional (8×8×6), containing significant information related to the feature point.

[0095] Step 7: Use the Euclidean distance as the matching measure to perform similarity measurement on the nearest neighbor and the next nearest neighbor distances of the three-dimensional phase orientation feature descriptor vector, and establish an initial corresponding relationship through a two-way matching strategy. Then, obtain the correct homologous point pairs by rejecting gross errors to complete the multi-modal image matching. The result is as Figure 5 shown.

[0096] Step 8: Use the correct homologous points to evaluate the matching effect of the multi-modal images. The present invention uses 4 groups of multi-modal images to test the performance of the algorithm. The dataset is shown in Figure 4For each pair of images, the root-mean-square error (RMSE) of homologous points and the number of matched homologous point pairs are used for quantitative inspection, where the unit of RMSE is pixels. The multi-modal image matching method proposed in the present invention is named the DoIFT algorithm, and it is compared with several optimal image matching methods (LGHD, PSO-SIFT, and RIFT). The comparison results are shown in Table 1.

[0097] Table 1 Comparison of several image matching methods

[0098]

[0099] As can be seen from Table 1, in multi-modal remote sensing image data, the DoIFT algorithm can obtain more homologous point pairs compared with the LGHD, PSO-SIFT, and RIFT algorithms. The DoIFT algorithm proposed in the present invention can achieve a relatively optimal result. Among them, the RMSE of the DoIFT algorithm is better than that of the LGHD, PSO-SIFT, and RIFT methods. The RMSE values of the DoIFT algorithm proposed in the present invention are all less than 2 pixels, which further proves that the DoIFT algorithm not only greatly increases the number of matched homologous points but also maintains good matching accuracy. At the same time, through a large number of experiments, it is found that when the extraction difficulty of multi-modal images is relatively large, the size of the descriptor sub-domain window can be increased; conversely, when the multi-modal images are rich in texture, the parameter values can be appropriately reduced. Among them, the number of layers parameter in the image scale space is set between 3 and 6.

[0100] On the other hand, the embodiment of the present invention also provides a multi-modal image matching system for three-dimensional phase orientation invariant feature transform, including:

[0101] A processor and a memory, where the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multi-modal image matching method for three-dimensional phase orientation invariant feature transform as described in the above technical solution.

[0102] It should be understood that the parts not elaborated in detail in this specification all belong to the prior art.

[0103] It should be understood that the above description of the preferred embodiment is relatively detailed, and it should not be considered as a limitation to the protection scope of the present invention patent. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or deformations without departing from the protection scope defined by the claims of the present invention, and all fall within the protection scope of the present invention. The scope of protection requested by the present invention shall be subject to the appended claims.

Claims

1. A multimodal image matching method based on three-dimensional phase orientation invariant feature transformation, characterized in that: The following steps are involved: Step 1, initializing the calculation parameters of multimodal image matching and performing nonlinear diffusion on the multimodal image; Step 2, using the nonlinear diffusion result to calculate the multi-channel differential gradient feature map; Step 3, performing image domain calculation on the multi-channel differential gradient feature map, and estimating the image phase feature using a three-dimensional phase consistency model, thereby generating a three-dimensional phase feature; Step 4, using the three-dimensional phase feature to perform Gaussian difference filtering, and counting the maximum eigenvalue in each channel to generate a significant difference phase feature map, and extract feature points from the significant difference phase feature map; Step 5, decomposing the three-dimensional phase amplitude feature in the three-dimensional phase feature, and counting the feature intensity of each channel, recording the optimal feature index, and obtaining the optimal feature index map; Step 6, counting the directional gradient histogram features of the neighborhood of the feature point in the optimal feature index map, and then constructing a three-dimensional phase directional feature descriptor vector feature; Step 7, perform similarity measurement of the nearest neighbor and next nearest neighbor distances on the three-dimensional phase-oriented feature descriptor vector, establish the initial correspondence through a two-way matching strategy, and then identify the correct same-name point pairs through a gross error elimination method to complete multimodal image matching.

2. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: In step 1, the number of layers in the image scale space, the number of feature points, the feature point description sub-neighborhood window, and the gross error elimination threshold parameters are initialized.

3. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The multi-channel differential gradient feature map in step 2 is defined as: G r,c,o =|coso*Gx r,c,o +sino*Gy r,c,o | Among them G r,c,o Represents a multi-channel differential gradient feature map; r is the number of image rows, c is the number of image columns, o is the direction, No is the number of directions, Gx r,c,o is the gradient feature in the x direction, Gy r,c,o is the gradient feature in the y direction.

4. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The mathematical expression of the three-dimensional phase consistency model in step 3 is defined as: In the formula, PC ρ,Φ (l) represents the gradient phase characteristic result under the three-dimensional phase consistency model, ω Φ (l) is the weight factor of frequency expansion, l represents polar coordinates, A ρ,Φ (l) is the filter convolution response amplitude of the 3D Log-Gabor filter at position l under scale ρ and 3D spatial direction Φ, A ρ,Φ (l)=|EO ρ,Φ |,EO ρ,Φ is the complex response of the 3D Log-Gabor filter at scale ρ and 3D spatial direction Φ, ΔΨ ρ,Φ (l) represents the phase deviation under l, scale ρ and direction Φ, which is used to measure the significance of the local phase. T is a noise threshold and ε is a minimum factor to prevent division by zero. in, and They are Fourier transform and inverse Fourier transform, filter ρ,o It is a frequency domain filter at scale ρ and three-dimensional spatial direction o, which is used to extract the frequency components of the image at a specific scale and direction.

5. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The specific implementation method in step 4 is as follows. First, the three-dimensional phase directional characteristics are summed. The specific mathematical expression is as follows: in, It is a three-dimensional tensor, which represents the phase feature value at position (x, y), direction j and channel k. j is the direction index, which indicates the directionality of the feature. k is the channel index, which usually indicates different feature channels or filter responses. K indicates the number of channels. Then, a three-dimensional Gaussian filter is used to smooth the three-dimensional phase feature to generate a smoothed feature map PC3D_m. Dog_m1 is generated by using the difference before and after Gaussian smoothing to capture the significant changes of features; Dog_m1(x,y,j)=PC3D_m1(x,y,j)-PC_m1(x,y,j) The maximum value of each pixel in all directions in the phase difference image is selected as the final feature response PC_m1, and normalized to obtain the most significant local directional feature; PC_m1(x,y)=maxDog_m1(x,y,j) Among them, S Dog Represents the significant difference phase feature map, min(·) and max(·) represent the minimum and maximum value calculation functions respectively; feature point extraction is performed in the significant difference phase feature map, including FSAT and Harris detection.

6. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The specific implementation of step 5 includes: First, to calculate the three-dimensional phase feature convolution sequence, the amplitude of the two-dimensional maximum directional phase feature map at each scale ρ is added. The specific formula is as follows: Where CS(x,y,j) represents the sequence accumulation value of the phase feature tensor at the position of pixel (x,y) and the j-th layer channel; EO i,j (x, y, z) represents the value of the i-th input feature map at the j-th layer, with z channels, ns is the number of feature maps calculated for each j channel; max(·) means taking the maximum value; After completing the amplitude addition of the two-dimensional maximum directional phase feature results at each scale ρ, the maximum amplitude under different directions is counted to complete the optimal feature index BoIM of the three-dimensional phase feature. The specific mathematical expression is as follows: MI CS (x,y)=argmax j CS(x,y,j) MI CS (x,y) represents the optimal feature index result of the three-dimensional phase feature.

7. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The specific implementation of step 6 is as follows; In the optimal feature index BoIM, the method of directional gradient histogram descriptor is used to calculate the feature vector; for each feature point, a region of size K×K is determined and divided into K×K grids. In each grid, the pixel histogram of the BoIM image is calculated and distributed to N directions according to the direction of the pixel value; each small area generates an N-dimensional feature vector, which represents the histogram information of the region in N directions. Therefore, the descriptor vector of each feature point is K×K×N dimensional, which contains the significant information related to the feature point.

8. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: In step 7, Euclidean distance is used for similarity measurement, and a two-way matching strategy is used to ensure one-to-one correspondence of matching points. A random sampling consistency algorithm is used to eliminate false matches and achieve stable matching of multimodal images.

9. The multimodal image matching method of three-dimensional phase orientation invariant feature transformation according to claim 1, characterized in that: The method further includes step 8, using correct same-name points to evaluate the matching effect of the multimodal images. For each image pair, a quantitative test is performed using the root mean square error RMSE of the same-name points and the number of same-name point pair matches, wherein the unit of RMSE is pixel.

10. A multimodal image matching system based on three-dimensional phase orientation invariant feature transformation, characterized in that: include: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multimodal image matching method of a three-dimensional phase orientation invariant feature transformation as described in any one of claims 1 to 9.

Images