Multi-source remote sensing image registration method based on phase consistency information and multi-moment feature map

Abstract

The invention discloses a multi-source remote sensing image registration method based on phase consistency information and a multi-moment characteristic pattern. The method comprises the following steps: calculating according to a multi-angle bandwidth filter bank and the phase consistency information to obtain a plurality of groups of multi-moment characteristic patterns of an image; performing feature point detection on the multiple groups of multi-moment feature maps to obtain feature point detection results; obtaining a feature vector according to a feature point detection result; performing feature point rough matching according to the feature vector and the feature point detection result to obtain a rough matching point pair set; performing feature point fine matching on the rough matching point pair set by adopting cascade sample consistent estimation to obtain a fine matching point pair set; and transforming the to-be-registered image according to the fine matching point pair set to complete registration of the reference image and the to-be-registered image. According to the method, the angle difference, the noise interference difference and the scale difference between the reference image and the to-be-registered image can be suppressed or eliminated respectively, image registration can be completed in various scenes, the time consumed in the registration process is short, the calculated amount is small, and the result is stable and accurate.

Description

Technical Field

[0001] The present invention belongs to the field of image registration, and in particular relates to a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps. Background Art

[0002] With the continuous advancement of image sensor technology, remote sensing images are evolving from simple single-platform, single-modality, single-temporal, and low-resolution formats to complex data systems characterized by multi-platform collaboration, multi-modal fusion, multi-temporal coverage, and high resolution. Efficient and comprehensive processing of these multi-source remote sensing images has become a hot research topic. Multi-source remote sensing images contain diverse information features of targets, each with its own strengths and weaknesses under different imaging conditions. Taking optical and SAR images as examples, optical images offer excellent readability, but their results are susceptible to weather and environmental factors, limiting the scope of imaging scenarios. In contrast, SAR images offer high resolution, wide coverage, and all-day, all-weather capabilities, but their results suffer from poor interpretability. Joint analysis and processing of these two types of images can complement their different characteristics and increase information redundancy within the images. Therefore, achieving good MRSIR (Multisource Remote Sensing Images Registration) results is an essential processing step for subsequent applications, such as image fusion, change detection, resource monitoring, disaster warning, target recognition, 3D reconstruction, and navigation. MRSIR can perform pixel-level alignment of two or more remote sensing images acquired from different platforms, imaging environments, viewing angles, and imaging times. In the literature, MRSIR algorithms can be categorized into region-based and feature-based registration methods.

[0003] Region-based registration methods first rely on a coarse registration of prior geographic information. This registration result often has large error offsets, but it can provide a limited search range for subsequent registration. On this basis, region-based registration methods extract information from the image to be registered to generate a template. Within a limited search region of the reference image, similarity metrics such as mutual information, autocorrelation information, normalized mutual correlation information, absolute error and information, and sum of squares are calculated to find the best match. Region-based registration methods can, to a certain extent, achieve lower matching errors, i.e., more accurate registration results. However, their high time consumption and reliance on coarse registration preprocessing (coarse registration eliminates scale and orientation differences between images) limit their application.

[0004] Feature-based registration methods can be divided into four steps: feature detection, feature description, feature matching, outlier removal, and image transformation. Based on the image information used in registration, these methods can be broadly categorized into intensity-based methods, gradient-based methods (GBM), phase congruency (PC)-based methods, nonlinear diffusion filter-based methods, and other methods. Feature-based registration methods have a complete processing pipeline from feature extraction, feature description, feature matching, and image transformation, enabling them to independently and flexibly handle most registration tasks. Compared to region-based registration methods, they are computationally less complex and time-consuming. However, some algorithms designed to eliminate scale differences between images require the establishment of a scale space, which in turn increases the computational effort in the registration process. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps.

[0006] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps, the method comprising:

[0008] A plurality of first multi-moment feature maps of the reference image and a plurality of second multi-moment feature maps of the image to be registered are calculated based on a multi-angular bandwidth filter bank and phase consistency information; wherein the multi-angular bandwidth filter bank is obtained by changing the angular filter bandwidth of a 2D Log-Gabor filter;

[0009] Performing feature point detection on the multiple groups of first multi-moment feature maps and the multiple groups of second multi-moment feature maps respectively to obtain first feature point detection results and second feature point detection results;

[0010] Obtaining a first feature vector according to the first feature point detection result;

[0011] Performing rough matching of feature points based on the first feature vector and the second feature point detection result to obtain a rough matching point pair set;

[0012] Performing fine feature point matching on the coarse matching point pair set using cascade sample consistency estimation to obtain a fine matching point pair set;

[0013] The image to be registered is transformed according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

[0014] Optionally, before obtaining the plurality of first multi-moment feature maps of the reference image and the plurality of second multi-moment feature maps of the image to be registered by calculating the plurality of first multi-moment feature maps of the reference image and the plurality of second multi-moment feature maps of the image to be registered according to the multi-angle bandwidth filter bank, the method further comprises:

[0015] constructing a first anisotropic scale space according to a first gradient of the reference image;

[0016] constructing a second anisotropic scale space according to a second gradient of the image to be registered;

[0017] The multiple groups of first multi-moment feature maps are calculated in the first anisotropic scale space, and the multiple groups of second multi-moment feature maps are calculated in the second anisotropic scale space.

[0018] Optionally, the multi-angle bandwidth filter bank and phase consistency information include:

[0019] Changing the angular filter bandwidth of the 2D Log-Gabor filter to obtain a multi-angular bandwidth filter bank;

[0020] Calculating multiple groups of first phase-consistent information of the reference image and multiple groups of second phase-consistent information of the image to be registered according to the multi-angle bandwidth filter group;

[0021] Calculate multiple groups of first multi-moment feature maps based on multiple groups of first phase consistency information of the reference image;

[0022] A plurality of groups of second multi-moment feature maps are calculated based on the plurality of groups of second phase consistency information of the images to be registered.

[0023] Optionally, the multi-angle bandwidth filter bank is expressed as follows:

[0024]

[0025] Among them, LG2(ρ,θ,σ θ,i ) represents the multi-angle bandwidth filter bank, exp() represents the exponential function, ρ is the frequency in the logarithmic coordinate system, ρ0 is the center frequency in the logarithmic coordinate system, σ ρ is the filter bandwidth in the logarithmic coordinate system, θ is the angle, θ0 is the filter angle, σ θ,i is the bandwidth of the multi-angle filter, σ θ,i =π / N o ·d i , d i Represents the spacing of the multi-angle bandwidth filter bank in the angle dimension, N o Indicates the number of directions.

[0026] Optionally, performing feature point detection on the multiple groups of first multi-moment feature maps and the multiple groups of second multi-moment feature maps respectively to obtain first feature point detection results and second feature point detection results includes:

[0027] Detect feature points of the maximum multi-moment feature map and the minimum multi-moment feature map in each group of first multi-moment feature maps in blocks, and obtain a first intermediate result and a second intermediate result of each group of first multi-moment feature maps;

[0028] Combining the first intermediate result and the second intermediate result to obtain a first detection result under the current angular bandwidth;

[0029] Analyze the first detection results under different angular bandwidths, and take the feature points whose occurrence times are greater than or equal to a preset threshold as the first feature point detection results;

[0030] Detect feature points in blocks from the maximum multi-moment feature map and the minimum multi-moment feature map in each group of second multi-moment feature maps, and obtain a third intermediate result and a fourth intermediate result for each group of second multi-moment feature maps;

[0031] Combining the third intermediate result and the fourth intermediate result to obtain a second detection result under the current angular bandwidth;

[0032] The second detection results under different angular bandwidths are analyzed, and feature points whose occurrence times are greater than or equal to a preset threshold are taken as the second feature point detection results.

[0033] Optionally, performing rough matching of feature points according to the first feature vector and the second feature point detection result to obtain a rough matching point pair set includes:

[0034] Select any feature point from the second feature point detection result, and calculate the closest Euclidean distance and the second closest Euclidean distance between the feature point and the first feature vector of each feature point in the first feature point detection result one by one;

[0035] Obtaining a ratio threshold according to the nearest Euclidean distance and the second nearest Euclidean distance;

[0036] The coarse matching point pair set is obtained according to the ratio threshold.

[0037] Optionally, transforming the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered includes:

[0038] Obtaining transformation parameters between the reference image and the image to be registered according to the set of fine matching point pairs and the image transformation model;

[0039] The image to be registered is transformed according to the transformation parameters to complete the registration of the reference image and the image to be registered.

[0040] In a second aspect, the present invention provides a multi-source remote sensing image registration device based on phase consistency information and multi-moment feature maps, the device comprising:

[0041] a multi-moment feature map acquisition module, configured to calculate a plurality of first multi-moment feature maps of the reference image and a plurality of second multi-moment feature maps of the image to be registered based on a multi-angular bandwidth filter bank, wherein the multi-angular bandwidth filter bank is obtained by varying the angular filter bandwidth of a 2D Log-Gabor filter;

[0042] A detection result acquisition module, configured to perform feature point detection on each of the plurality of first multi-moment feature maps and the plurality of second multi-moment feature maps to obtain a first feature point detection result and a second feature point detection result;

[0043] A feature vector acquisition module, configured to obtain a first feature vector according to the first feature point detection result;

[0044] a coarse matching module, configured to perform coarse matching of feature points based on the first feature vector and the second feature point detection result to obtain a coarse matching point pair set;

[0045] A fine matching module, configured to perform fine feature point matching on the coarse matching point pair set using cascade sample consistency estimation to obtain a fine matching point pair set;

[0046] A registration module is used to transform the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

[0047] The technical solutions provided by the embodiments of the present invention may have the following beneficial effects:

[0048] In the above technical scheme, the present invention uses a multi-angle bandwidth filter group to extract phase-consistent information of the image, effectively suppresses the noise in the image through incoherent accumulation, extracts feature points based on the phase-consistent multi-moment feature map, and performs coarse matching and fine matching based on the feature point detection results, and then transforms the image to be registered according to the matching results, thereby realizing the registration of the image to be registered. The present invention can suppress or eliminate the angle difference, noise interference difference, and scale difference between the reference image and the image to be registered, and can complete image registration in a variety of scenarios. The registration process is time-consuming, computationally intensive, and the results are stable and accurate.

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention provides a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps;

[0051] Figure 2 1 is a schematic diagram of phase consistency information direction calculation provided by an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of a feature point detection process provided by an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of a feature vector calculation process provided by an embodiment of the present invention;

[0054] Figure 5a is a schematic diagram of an optical image feature point detection result provided by an embodiment of the present invention;

[0055] Figure 5b 1 is a schematic diagram of a SAR image feature point detection result provided by an embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of a feature matching result provided by an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of an image registration result provided by an embodiment of the present invention;

[0058] Figure 8 This is a block diagram of a multi-source remote sensing image registration device based on phase consistency information and multi-moment feature maps provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to facilitate the understanding of the present invention, the related art and the inventive concept of the present invention are briefly described first.

[0060] The existing technology mainly includes the following methods:

[0061] A scale-invariant feature transformation method based on gradient information description, proposed in 2004, is suitable for registration between homologous images. The general process is as follows: First, the image is filtered using Gaussian kernels of different scales and sampled at a specific scale to obtain a Gaussian pyramid. Adjacent layers in the Gaussian pyramid are then subtracted to obtain a difference pyramid. Feature detection is performed within the difference pyramid. For each pixel, the maximum value of its three-dimensional neighborhood is selected, and the precise coordinates are obtained through interpolation. The main direction of each feature point is obtained by counting the gradient directions of its neighboring pixels. For each feature point, its square neighborhood is divided into 4x4 equal-sized subregions. Within each subregion, a histogram is calculated, indexed by the gradient direction bit and weighted by the gradient modulus. The descriptor vector for the feature point is then integrated and normalized. Finally, feature matching is performed using the nearest neighbor Euclidean distance ratio method. Through a series of processing, the scale-invariant feature transformation method ensures that the feature points and their descriptors are invariant to rotation, scale, illumination, viewpoint, and noise interference. However, it is only applicable to homologous image registration and cannot handle the problem of image grayscale differences caused by radiation differences in heterogeneous images.

[0062] The heterogeneous image registration method proposed in 2018 has a registration process framework that is basically the same as the scale-invariant feature transformation method, but there are the following differences: 1. For the calculation of gradients for heterogeneous images, this method adopts different calculation methods for different images to adapt to their respective radiation characteristics and noise; 2. Instead of the Gaussian difference pyramid, this method establishes a Harris scale space and uses corner point response values for feature detection; 3. Unlike the square neighborhood used in feature description in the scale-invariant feature transformation method, this algorithm uses gradient positioning and directional histogram descriptors to describe the circular neighborhood of feature points.

[0063] A multi-source remote sensing image registration method based on a radiometrically insensitive feature transformation was proposed in 2019. First, phase congruency analysis is used to extract feature points that are insensitive to radiometric changes, avoiding reliance on traditional gradient or intensity information. Second, a hybrid descriptor is constructed based on the maximum index map of the phase congruency amplitude, combining rotational invariance and dimensionality reduction to generate compact features. A bidirectional matching strategy is then used for initial matching, and an algorithm is used to remove mismatched points. Finally, the least squares method is used to optimize the geometric transformation parameters to achieve high-precision spatial alignment of multi-source images. This method significantly improves the robustness of registration of heterogeneous remote sensing images under complex radiometric variations through its combination of radiometrically invariant features and an efficient matching mechanism.

[0064] Several image registration methods mentioned above, due to differences in the radiation characteristics and noise interference of multi-source images, lead to differences in image grayscale and gradient calculation results. This brings difficulties to feature extraction and description: it affects the independence and robustness of feature descriptors. The scale-invariant feature transformation method cannot cope with the impact of scale differences in multi-source images and is sensitive to angular differences between images, which limits the application scenarios of the registration method. Therefore, this paper proposes a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps to solve this technical problem.

[0065] Figure 1 The embodiment of the present invention provides a multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps, such as Figure 1 As shown, the method includes the following steps:

[0066] S101. Calculate multiple groups of first multi-moment feature maps of the reference image and multiple groups of second multi-moment feature maps of the image to be registered based on a multi-angular bandwidth filter group and phase consistency information; wherein the multi-angular bandwidth filter group is obtained by changing the angular filter bandwidth of the 2D Log-Gabor filter.

[0067] It can be understood that the reference image is an optical image, and the image to be registered is a SAR image. Similarly, the reference image can also be a SAR image, in which case the image to be registered is an optical image. Changing the angular filter bandwidth of the 2D Log-Gabor filter can yield the angular filter bandwidth. Phase congruence information is obtained by convolving the input image with the 2D Log-Gabor filter. The multi-moment feature map includes a maximum moment feature map and a minimum moment feature map.

[0068] Alternatively, the multi-angle bandwidth filter bank is expressed as follows:

[0069]

[0070] Among them, LG2(ρ,θ,σ θ,i ) represents a multi-angle bandwidth filter bank, exp() represents an exponential function, ρ is the frequency in the logarithmic coordinate system, ρ0 is the center frequency in the logarithmic coordinate system, σ ρ is the filter bandwidth in the logarithmic coordinate system, θ is the angle, θ0 is the filter angle, σ θ,i is the bandwidth of the multi-angle filter, σ θ,i =π / N o ·d i , d i Represents the spacing of the multi-angle bandwidth filter bank in the angle dimension, N o Indicates the number of directions.

[0071] Optionally, before S101, the method may further include:

[0072] constructing a first anisotropic scale space according to a first gradient of the reference image;

[0073] constructing a second anisotropic scale space according to a second gradient of the image to be registered;

[0074] A plurality of groups of first multi-moment feature maps are calculated in a first anisotropic scale space, and a plurality of groups of second multi-moment feature maps are calculated in a second anisotropic scale space.

[0075] In one embodiment, taking an optical image as the reference image and a SAR image as the image to be registered, the first gradient includes a first gradient magnitude and a first gradient direction, and the second gradient includes a second gradient magnitude and a second gradient direction. The Sobel operator is used to calculate the first gradient of the optical image: the Sobel operator convolves the template in the horizontal and vertical directions with the optical image to obtain the gradient values of the target pixel in both directions. For ease of calculation, the Sobel operator is simplified to a rectangular window. It can be expressed as:

[0076]

[0077] in, and They represent the simplified templates of the Sobel operator in two directions, and the window size is the current layer scale β of the optical image. j , represents the intensity image of the optical image after Gaussian smoothing, × is the matrix convolution operation, and are the gradient values of the target pixel in the horizontal and vertical directions, respectively. h, v represent the horizontal and vertical directions, respectively, and j represents the number of scale layers in the scale space.

[0078] The gradient magnitude and direction of the target pixel can be calculated using the following formula:

[0079]

[0080] in, represents the first gradient magnitude of the optical image, Represents the first gradient direction of the optical image.

[0081] The Adaptive ROEWA operator is used to calculate the gradient of the SAR image: Taking the horizontal gradient calculation as an example, when calculating the gradient of the target pixel, the pixels in the column where the target pixel is located are included in the calculation of the ROEWA operator, as shown in the following formula:

[0082]

[0083] in, Represents the gradient value of the target pixel in the horizontal and vertical directions, α i is the scale of the layer where the target pixel SAR image is currently located, Represents the adaptive constant values in the horizontal and vertical directions respectively. M, N represent the number of rows and columns of the calculation window respectively. The window size is related to α i I(x,y) represents the image intensity at coordinate (x,y) in the SAR image. Represents the local exponentially weighted average of pixel intensities, where loc=r,d,l,r represents the four directions of up, down, left, and right.

[0084] In order to eliminate the interference of speckle noise caused by special imaging mechanism in SAR imaging process on subsequent image processing, Take the logarithm respectively to get the gradient value of the target pixel in the horizontal and vertical directions, and then calculate the gradient amplitude and direction, as shown in the following formula:

[0085]

[0086] in, and It represents the gradient value of the target pixel in two directions after taking the logarithm. represents the second gradient amplitude of the SAR image, Indicates the second gradient direction of the SAR image.

[0087] In addition, to ensure that the scale space of the optical image and the SAR image is consistent, when calculating the gradient, α i and β i Need to meet:

[0088] α i+1 / α i =kβ i+1 / β i =k,α1=β1;

[0089] Among them, k represents the ratio between two adjacent layers in the scale space.

[0090] The optical image and SAR image are filtered using the nonlinear diffusion equation, as follows:

[0091]

[0092] Where L represents the input image (optical image or SAR image), t represents the current scale, c(x, y, t) is the diffusion function, Δ, The diffusion function c(x, y, t) is a function of the image gradient modulus. Different calculation methods can be used depending on the focus of the image content, as shown below:

[0093]

[0094] in, Represents the image gradient, and K is a constant.

[0095] For the nonlinear diffusion equation, the additive splitting operator is used to solve it, which is expressed as follows:

[0096]

[0097] Where τ is the time step, which controls the amount of time each iteration "advances"; I is the identity matrix; L i Represents the state of the image at the i-th iteration; A l is a state that depends on the current image state L i The matrix encodes the image conductivity in the lth direction and is used to control the diffusion rate and direction in this direction; m is the number of dimensions, usually the dimension of the image.

[0098] The optical image and SAR image are processed separately using the nonlinear diffusion equation, and the first anisotropic scale space ASS of the optical image is finally obtained. O and the second anisotropic scale space ASS of SAR images S .

[0099] Optionally, S101 may include: obtaining multiple groups of first phase-consistent information of the reference image and multiple groups of second phase-consistent information of the image to be registered by calculating according to a multi-angle bandwidth filter bank;

[0100] Calculate multiple sets of first multi-moment feature maps based on multiple sets of first phase consistency information of the reference image;

[0101] A plurality of groups of second multi-moment feature maps are calculated based on the plurality of groups of second phase consistency information of the images to be registered.

[0102] It is understandable that the process of inputting the reference image and the image to be registered is the same in this step, so it is summarized with the input image. The input image is convolved with the 2D log-Gabor filter to obtain phase consistency information. The frequency domain expression of the filter is as follows:

[0103]

[0104] Among them, ρ is the frequency in the logarithmic coordinate system, ρ0 is the center frequency in the logarithmic coordinate system and σ ρis the filter bandwidth in the logarithmic coordinate system. θ is the angle, θ0 is the filter angle, σ θ is the angular filter bandwidth. The real and imaginary parts of the filtering results correspond to the filtering results of the even symmetric filter and the odd symmetric filter respectively. s and e s The local energy E is related to the frequency component amplitude and The calculation is as follows:

[0105]

[0106] Taking into account factors such as noise, frequency expansion weighting, and sensitivity to phase deviation, the two-dimensional phase consistency information based on different frequency bandwidths and different filter angles is calculated in a logarithmic coordinate system:

[0107]

[0108] Where (x, y) represents the two-dimensional coordinates in the input image; s = 1, 2, ..., N s ,o=1,2,...,N o are scale and direction index respectively, N s is the number of scales, N o is the number of directions; W o Represents the weight function in each direction, which is used to correct the effect of the filter on frequency expansion; Indicates the amplitude value of each frequency component; ΔΦ so is the deviation angle of local energy. Compared with the cosine function, this method is more sensitive to phase deviation; T o Represents the noise threshold, which is used to filter out the effects of noise fluctuations in the signal or image. ε ensures that there will be no zero value in the denominator during division and is generally set to a small value of 0.01. Indicates that the contained quantity is equal to itself if its value is positive, and zero otherwise.

[0109] Figure 2 Schematic diagram of phase-consistent information direction calculation provided by an embodiment of the present invention, such as Figure 2 As shown, the 2D Log-Gabor odd-symmetric filter is rotated at several angles to filter the image, and the derivatives in different directions are calculated. These derivative values are then projected onto the x-axis and y-axis, corresponding to 0° and 90° respectively. Finally, the inverse tangent value is calculated from the projection values in the two directions to obtain the direction of the pixel point. The calculation of the above process is as follows:

[0110]

[0111] O pc =arctan(O y ,Ox )

[0112] Among them, O x ,O y are the projections of the derivatives in different directions on the x-axis and y-axis in the coordinate system with the pixel point in the upper left corner of the image as the origin; θ o The angle representing the filter direction; o so is the filter result of 2D Log-Gabor odd symmetric filter on the image, O pc is the direction of phase-consistent information, and arctan is the inverse tangent operator.

[0113] The calculation formula for phase consistency information reconstructed according to the filter angle dimension is obtained:

[0114]

[0115] Among them, θ o is the filter angle, PC(θ o ,x,y) means the point (x,y) is in θ o Phase consistency information in the direction. The calculation formulas of the main axis Φ, the maximum moment characteristic map M and the minimum moment characteristic map m are as follows:

[0116]

[0117] Where arctan is the inverse tangent operator, and a, b, and c are intermediate values in the calculation process. The calculation is as follows:

[0118]

[0119] S102 , performing feature point detection on the multiple groups of first multi-moment feature maps and the multiple groups of second multi-moment feature maps respectively to obtain first feature point detection results and second feature point detection results.

[0120] Optionally, S102 may include:

[0121] Detect feature points of the maximum multi-moment feature map and the minimum multi-moment feature map in each group of first multi-moment feature maps in blocks, and obtain a first intermediate result and a second intermediate result of each group of first multi-moment feature maps;

[0122] Combining the first intermediate result and the second intermediate result to obtain a first detection result under the current angular bandwidth;

[0123] Analyze the first detection results under different angles and bandwidths, and take the feature points whose number of occurrences is greater than or equal to a preset threshold as the first feature point detection results;

[0124] Detect feature points in blocks from the maximum multi-moment feature map and the minimum multi-moment feature map in each group of second multi-moment feature maps, and obtain a third intermediate result and a fourth intermediate result for each group of second multi-moment feature maps;

[0125] Combining the third intermediate result and the fourth intermediate result to obtain a second detection result under the current angular bandwidth;

[0126] The second detection results under different angular bandwidths are analyzed, and feature points whose occurrence times are greater than or equal to a preset threshold are taken as second feature point detection results.

[0127] It is understandable that Figure 3 is a schematic diagram of a feature point detection process provided by an embodiment of the present invention, such as Figure 3 As shown in the figure, based on multiple sets of first multi-moment feature maps and multiple sets of second multi-moment feature maps, FAST features are detected in blocks in each maximum and minimum moment feature map. The FAST features detected in the blocks of the maximum and minimum moment feature maps are merged to obtain the detection results under the current angle bandwidth. The results of different bandwidths are voted. That is, a feature point that appears multiple times in the moment feature maps of different bandwidths is considered to be a stable and reliable feature. All voting results are collected as the feature point detection result.

[0128] S103 : Obtain a first feature vector according to the first feature point detection result.

[0129] Figure 4 This is a flow chart of a feature vector calculation process provided by an embodiment of the present invention. Figure 4 As shown, first, the feature point obtained in step 3 is taken as the center of the circle, and the length (12σ i ) is the radius to establish a logarithmic polar coordinate system, σ i is the current image layer scale, and this area is divided into 17 sub-areas: along the radial direction according to 3σ i , 8σ iThe circular area is divided into three parts, using a [1,8,8] angular segmentation strategy from the inside out. This results in a feature vector calculation neighborhood divided into 17 subregions. Before the calculation begins, to ensure the invariance of each feature point with respect to direction, all points within the feature point neighborhood are first rotated to the corresponding coordinate system, using the feature point direction obtained in step 2 as the positive horizontal axis of the rectangular coordinate system. To avoid gradient flipping, the final calculation result is limited to [0,180°). Gradient histogram statistics are then performed on the points within the neighborhood. Specifically, the feature point neighborhood is constructed as a polar coordinate grid structure, divided into 17 subregions: the center is a circular area, the middle ring is evenly divided into 8 sectors according to the angular direction, and the outer ring is further divided into 8 sectors. In other words, the entire neighborhood contains a total of 17 subregions. The directional range [0, 180) is evenly divided into eight directional bins. Using the gradient direction as the index, the maximum and minimum moment values of each pixel are weighted and accumulated into the corresponding directional bins, thereby constructing a maximum-moment-based directional histogram and a minimum-moment-based directional histogram, respectively. Each subregion independently generates an 8-dimensional directional histogram, resulting in a descriptor with 17 × 8 = 136 dimensions. Finally, the maximum-moment-based directional histogram and the minimum-moment-based directional histogram are concatenated to form a 272-dimensional one-dimensional multi-moment feature description vector. To reduce the sensitivity of feature points to light intensity, the feature vectors are normalized after calculation. High values due to high intensity are clipped, and then normalized again to ensure additive normalization of the feature vectors.

[0130] S104 , performing rough matching of feature points based on the first feature vector and the second feature point detection result to obtain a rough matching point pair set.

[0131] Optionally, S104 may include:

[0132] Select any feature point from the second feature point detection result, and calculate the closest Euclidean distance and the second closest Euclidean distance between the feature point and the first feature vector of each feature point in the first feature point detection result one by one;

[0133] The ratio threshold is obtained according to the nearest Euclidean distance and the second nearest Euclidean distance;

[0134] A set of coarse matching point pairs is obtained according to the ratio threshold.

[0135] It can be understood that the feature points in the two images are roughly matched first. Let the feature point set of the image to be registered be O, and select a feature point P from it. O , calculate the Euclidean distance between the feature vectors of the feature point set S of the reference image point by point, if it satisfies:

[0136]

[0137] Among them, ED1 and ED2 represent points P respectively. O The closest and second closest Euclidean distances between the feature vectors of each point in the point set S are given by Thres, which is the ratio threshold. The feature point pairs that meet the above formula are considered to be a set of coarse matching point pairs and recorded in the coarse matching point pair set CM.

[0138] S105 , performing fine feature point matching on the coarse matching point pair set using cascade sample consistency estimation to obtain a fine matching point pair set.

[0139] It can be understood that by matching the coarse matching point pair set CM again and using Cascade Sample Consensus (CSC), the erroneous point pairs can be further eliminated.

[0140] (1) First, scale constraints are applied to the point pair set CM:

[0141] a) Calculate any two point pairs [P i ,Q i ],[P j ,Q j ], where P i represents the reference image feature points, Q i Represents the feature points of the registered image. That is:

[0142]

[0143] b) Traverse all possible point pairs and make D ij Perform histogram statistics. Remove the point corresponding to the minimum value in the histogram and calculate the RMSE at this time;

[0144] c) Repeat the above two steps until the RMSE changes little or there are only three point pairs in the point set.

[0145] (2) After the scale constraint processing, the scale-constrained matching point pair set SC-CM is obtained, and the first-level matching is performed using Fast Sample Consensus (FSC). The FSC matching method is as follows:

[0146] After completing the scale constraint processing, the initial scale-constrained matching point pair set SC-CM is obtained. Next, the Fast Sample Consensus (FSC) algorithm is used to perform the first stage of coarse matching screening. The FSC processing flow is as follows:

[0147] i) Randomly select three matching point pairs from the matching point pair set SC-CM as samples, and calculate a candidate image transformation parameter θ based on these three point pairs;

[0148] ii) Bring all the remaining point pairs in the set CM into the transformation model, and according to the preset error threshold, count the number of matching point pairs num whose error is less than the threshold under this transformation inliers , which is regarded as the number of internal points under the current model;

[0149] iii) Repeat steps i) and ii) to iteratively update the maximum number of inner points num max , until the maximum number of iterations is reached or the number of internal points meets the early termination condition;

[0150] Finally, the set of matching point pairs with the largest number of inliers is selected as the precise matching point pair set, and the corresponding transformation parameter θ max It is used as the initial estimation result for subsequent registration processing.

[0151] (3) Select the nearest neighbor of FM and take the feature point P of the reference image in FM i By mapping the transformation parameter θ to the registration image, find the nearest feature point Q in its neighborhood i If it exists, put the point pair into the high confidence matching point pair set C sample Match point pairs with low confidence set C total Otherwise, find the distance P in the global i The nearest feature point Q j , put the point pair into C total middle.

[0152] (4) C sample with C total Input it into FSC again for the second level matching to obtain the final fine matching point pair set FM final .

[0153] S106 : transforming the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

[0154] Optionally, S106 may include:

[0155] According to the fine matching point pair set and the image transformation model, the transformation parameters between the reference image and the image to be registered are obtained;

[0156] The image to be registered is transformed according to the transformation parameters to complete the registration of the reference image and the image to be registered.

[0157] It can be understood that according to the fine matching point set FM final, combined with different image transformation models, such as similarity transformation, affine transformation, projective transformation, etc., to estimate the transformation parameter θ between the reference image and the image to be registered final The image to be registered is transformed into the coordinate system of the reference image using similarity, projection, and affine transformations based on the transformation parameters, completing the registration between the two images. Generally, to assess and test the accuracy of the registration, the two registered images are plotted together using a checkerboard grid, and the registration effect is evaluated by comparing edges and regions.

[0158] To verify the performance of the registration algorithm proposed in this paper, optical-SAR image pairs from different scenarios were selected for registration experiments. The comparison methods included the Optical-SAR Scale-Invariant Feature Transform (OS-SIFT), the Scale-Rotation-Invariant Feature Transform (SRIF), the Rotation-Invariant Feature Transform (RIFT), the Improved KAZE (I-KAZE), and the Adjacent Self-Similarity Feature (ASS). The image pairs used in this experiment came from three different datasets, and their details are shown in Table 1.

[0159] Table 1

[0160]

[0161] Figure 5a is a schematic diagram of an optical image feature point detection result provided by an embodiment of the present invention, Figure 5b 3 is a schematic diagram of a SAR image feature point detection result provided by an embodiment of the present invention. The present invention adopts a block extraction method, so the number of features of the two images is relatively close. Figure 6 This is a schematic diagram of a feature matching result provided by an embodiment of the present invention. The descriptor used in the present invention has better independence and robustness, which results in a higher number of matches and matching accuracy during feature matching. The results of the six image registration methods are compared in Table 2.

[0162] Table 2

[0163]

[0164] Among them, NCM is the number of correctly matched point pairs, SR is the ratio of successfully registered image pairs to the total number of image pairs, both of which reflect the stability of the algorithm; RMSE is the root mean square error, which reflects the accuracy of the algorithm. Figure 7 FIG is a schematic diagram of an image registration result provided by an embodiment of the present invention. Figure 7 As can be seen from the table, the registration method proposed in the present invention is superior to the other five existing algorithms in terms of algorithm stability and accuracy.

[0165] The present invention first performs anisotropic diffusion filtering preprocessing on the image and establishes anisotropic scale space, with the purpose of eliminating scale differences and performing preliminary noise suppression. Subsequently, a multi-angle bandwidth filter group is proposed for extracting phase-consistent information from the image, which effectively suppresses noise in the image through incoherent accumulation. Feature detection is completed in a phase-consistent multi-moment feature map. The feature point detection results are generated by combining FAST and multi-bandwidth feature point voting strategies. At the same time, the main direction is assigned to the feature points to eliminate directional differences. Then, the feature point detection results of the reference image and the image to be registered are matched to obtain transformation parameters. The transformation parameters are used to realize the registration of the image to be registered. The radiation characteristic differences, angle differences, noise interference differences, and scale differences between the multi-source images in the registration task can be suppressed or eliminated respectively. Image registration can be completed in a variety of scenarios. The registration algorithm is stable and the results are accurate.

[0166] Figure 8 is a block diagram of a multi-source remote sensing image registration device based on phase consistency information and multi-moment feature maps provided by an embodiment of the present invention. Figure 8 As shown, the apparatus 800 may include:

[0167] A multi-moment feature map acquisition module 801 is configured to calculate a plurality of first multi-moment feature maps of a reference image and a plurality of second multi-moment feature maps of an image to be registered based on a multi-angular bandwidth filter bank, wherein the multi-angular bandwidth filter bank is obtained by varying the angular filter bandwidth of a 2D Log-Gabor filter;

[0168] A detection result acquisition module 802 is configured to perform feature point detection on the plurality of first multi-moment feature maps and the plurality of second multi-moment feature maps, respectively, to obtain first feature point detection results and second feature point detection results;

[0169] A feature vector acquisition module 803 is configured to obtain a first feature vector according to a first feature point detection result;

[0170] A coarse matching module 804 is configured to perform coarse matching of feature points based on the first feature vector and the second feature point detection result to obtain a coarse matching point pair set;

[0171] A fine matching module 805 is configured to perform fine feature point matching on the coarse matching point pair set using cascaded sample consistency estimation to obtain a fine matching point pair set;

[0172] The registration module 806 is used to transform the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

[0173] It should be noted that the terms "first," "second," and the like are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention.

[0174] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0175] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings and the disclosed content. In the description of the present invention, the word "comprising" does not exclude other components or steps, "one" or "a" does not exclude multiple situations, and "multiple" means two or more, unless otherwise clearly and specifically limited. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0176] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps, characterized in that: The method comprises: A plurality of first multi-moment feature maps of the reference image and a plurality of second multi-moment feature maps of the image to be registered are calculated based on a multi-angular bandwidth filter bank and phase consistency information; wherein the multi-angular bandwidth filter bank is obtained by changing the angular filter bandwidth of a 2D Log-Gabor filter; Performing feature point detection on the multiple groups of first multi-moment feature maps and the multiple groups of second multi-moment feature maps respectively to obtain first feature point detection results and second feature point detection results; Obtaining a first feature vector according to the first feature point detection result; Performing rough matching of feature points based on the first feature vector and the second feature point detection result to obtain a rough matching point pair set; Performing fine feature point matching on the coarse matching point pair set using cascade sample consistency estimation to obtain a fine matching point pair set; The image to be registered is transformed according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

2. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: Before obtaining the plurality of first multi-moment feature maps of the reference image and the plurality of second multi-moment feature maps of the image to be registered by calculating the plurality of first multi-moment feature maps of the reference image and the plurality of second multi-moment feature maps of the image to be registered according to the multi-angle bandwidth filter bank, the method further comprises: constructing a first anisotropic scale space according to a first gradient of the reference image; constructing a second anisotropic scale space according to a second gradient of the image to be registered; The multiple groups of first multi-moment feature maps are calculated in the first anisotropic scale space, and the multiple groups of second multi-moment feature maps are calculated in the second anisotropic scale space.

3. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: The method of calculating multiple groups of first multi-moment feature maps of the reference image and multiple groups of second multi-moment feature maps of the image to be registered based on the multi-angle bandwidth filter group and the phase consistency information includes: Calculating multiple groups of first phase-consistent information of the reference image and multiple groups of second phase-consistent information of the image to be registered according to the multi-angle bandwidth filter group; Calculate multiple groups of first multi-moment feature maps based on multiple groups of first phase consistency information of the reference image; A plurality of groups of second multi-moment feature maps are calculated based on the plurality of groups of second phase consistency information of the images to be registered.

4. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: The multi-angle bandwidth filter bank is expressed as follows: Among them, LG2(ρ,θ,σ θ,i ) represents the multi-angle bandwidth filter bank, exp() represents the exponential function, ρ is the frequency in the logarithmic coordinate system, ρ0 is the center frequency in the logarithmic coordinate system, σ ρ is the filter bandwidth in the logarithmic coordinate system, θ is the angle, θ0 is the filter angle, σ θ,i is the bandwidth of the multi-angle filter, σ θ,i =π / N o ·d i , d i Represents the spacing of the multi-angle bandwidth filter bank in the angle dimension, N o Indicates the number of directions.

5. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: The performing feature point detection on the plurality of groups of first multi-moment feature maps and the plurality of groups of second multi-moment feature maps respectively to obtain first feature point detection results and second feature point detection results includes: Detect feature points of the maximum multi-moment feature map and the minimum multi-moment feature map in each group of first multi-moment feature maps in blocks, and obtain a first intermediate result and a second intermediate result of each group of first multi-moment feature maps; Combining the first intermediate result and the second intermediate result to obtain a first detection result under the current angular bandwidth; Analyze the first detection results under different angular bandwidths, and take the feature points whose occurrence times are greater than or equal to a preset threshold as the first feature point detection results; Detect feature points in blocks from the maximum multi-moment feature map and the minimum multi-moment feature map in each group of second multi-moment feature maps, and obtain a third intermediate result and a fourth intermediate result for each group of second multi-moment feature maps; Combining the third intermediate result and the fourth intermediate result to obtain a second detection result under the current angular bandwidth; The second detection results under different angular bandwidths are analyzed, and feature points whose occurrence times are greater than or equal to a preset threshold are taken as the second feature point detection results.

6. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: The performing rough matching of feature points according to the first feature vector and the second feature point detection result to obtain a rough matching point pair set includes: Select any feature point from the second feature point detection result, and calculate the closest Euclidean distance and the second closest Euclidean distance between the feature point and the first feature vector of each feature point in the first feature point detection result one by one; Obtaining a ratio threshold according to the nearest Euclidean distance and the second nearest Euclidean distance; The coarse matching point pair set is obtained according to the ratio threshold.

7. The multi-source remote sensing image registration method based on phase consistency information and multi-moment feature maps according to claim 1, characterized in that: The transforming the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered includes: Obtaining transformation parameters between the reference image and the image to be registered according to the set of fine matching point pairs and the image transformation model; The image to be registered is transformed according to the transformation parameters to complete the registration of the reference image and the image to be registered.

8. A multi-source remote sensing image registration device based on phase consistency information and multi-moment feature maps, characterized in that: The device comprises: a multi-moment feature map acquisition module, configured to calculate a plurality of first multi-moment feature maps of the reference image and a plurality of second multi-moment feature maps of the image to be registered based on a multi-angular bandwidth filter bank, wherein the multi-angular bandwidth filter bank is obtained by varying the angular filter bandwidth of a 2D Log-Gabor filter; A detection result acquisition module, configured to perform feature point detection on each of the plurality of first multi-moment feature maps and the plurality of second multi-moment feature maps to obtain a first feature point detection result and a second feature point detection result; A feature vector acquisition module, configured to obtain a first feature vector according to the first feature point detection result; a coarse matching module, configured to perform coarse matching of feature points based on the first feature vector and the second feature point detection result to obtain a coarse matching point pair set; A fine matching module, configured to perform fine feature point matching on the coarse matching point pair set using cascade sample consistency estimation to obtain a fine matching point pair set; A registration module is used to transform the image to be registered according to the set of fine matching point pairs to complete the registration of the reference image and the image to be registered.

Images