Infrared and visible image registration method based on phase feature and edge feature

Abstract

The application discloses an infrared and visible light image registration method based on phase features and edge features. Infrared and visible light images of the same scene are acquired, image scales are preprocessed, and phase consistency information is acquired. Maximum moment graphs and minimum moment graphs of the infrared and visible light images are respectively acquired, corresponding superimposed moment graphs are acquired, a scale space is constructed for the superimposed moment graphs, and finally, feature points are screened through a non-maximum suppression method. According to a multi-angle multi-scale convolution sequence in the phase consistency information of the infrared and visible light images, a maximum direction index is calculated at each scale, a multi-scale maximum index graph is constructed, and feature descriptors of the feature points are constructed in combination with phase consistency edges. The feature points are matched through the Euclidean distance between the feature descriptors, and an optimal transformation matrix between the infrared and visible light images is obtained. The application overcomes the problem that traditional methods fail to register due to too large differences between infrared and visible light images, and realizes high-precision and good-robustness infrared and visible light image registration.

Description

Technical Field

[0001] This invention belongs to the fields of image registration and computer vision, specifically an infrared and visible light image registration method based on phase features and edge features. Background Technology

[0002] In recent years, multi-source sensor imaging technology has matured, resulting in a massive amount of multimodal image data. Infrared and visible light image processing technology, as a hot branch of multimodal image processing, has wide applications in military, medical, and surveying fields. Infrared and visible light images differ significantly, mainly in the following aspects: from an imaging principle perspective, both infrared and visible light images exhibit nonlinear radiation distortion; visually, visible light images have rich textures, while infrared images lack detail. Due to the significant differences and lack of consistent information between infrared and visible light images, registration is quite challenging.

[0003] Current registration methods mainly include region-based and feature-based methods. Region-based methods primarily include mutual information and normalized cross-correlation. These methods have two main drawbacks: first, they are only applicable to image pairs containing translational changes; registration will fail when the images undergo rotation, scaling, or other transformations. Second, their registration accuracy depends on the texture richness within the region window; if the window contains no texture or weak texture, incorrect matching is likely. Feature-based methods establish reliable matching relationships between feature points to obtain the transformation relationships between images. These methods have significant advantages in accuracy and practicality. However, due to the large spectral distance between long-wave infrared and visible light, gradient inversion and intensity distortion exist in infrared and visible light images, leading to low registration success rates and difficulty in achieving stable registration results using traditional methods. Summary of the Invention

[0004] To address the technical problems mentioned in the background section, this invention provides an infrared and visible light image registration method based on phase and edge features.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] An infrared and visible light image registration method based on phase and edge features includes the following steps:

[0007] (1) Acquire infrared and visible light images of the same scene, preprocess the image scale, and then use Log-Gabor filter to perform convolution calculation on the infrared and visible light images respectively to obtain phase consistency information;

[0008] (2) Based on the phase consistency information of infrared and visible light images, the maximum and minimum moment maps of infrared and visible light images are obtained respectively. Then, the maximum and minimum moment maps are weighted and superimposed to obtain the corresponding superimposed moment map. A scale space is constructed on the superimposed moment map, and feature points are extracted evenly in blocks on images of different scales. Finally, the feature points are screened by non-maximum suppression method.

[0009] (3) Based on the multi-angle, multi-scale convolution sequence in the phase consistency information of infrared and visible light images, calculate the maximum orientation index at each scale, construct a multi-scale maximum index map, and construct feature descriptors of feature points based on the multi-scale maximum index map and phase consistency edge map of infrared and visible light images.

[0010] (4) Match feature points by Euclidean distance between feature descriptors, then filter out mismatched points, and finally obtain the optimal transformation matrix between infrared and visible light images.

[0011] Furthermore, in step (1), the specific process is as follows:

[0012] (101) For the input infrared and visible light images, downsample the image with higher resolution so that the infrared and visible light resolutions are the same.

[0013] (102) For the input image I(x,y), a two-dimensional Log-Gabor filter (LGF) is used to convolve the image to obtain the response values ​​and phase consistency information of the filter in different directions and scales for each pixel.

[0014] Furthermore, in step (102), the process of obtaining convolutional information is as follows:

[0015] [e s,o (x,y),o s,o (x,y)]=[I(x,y)*LGF s,o even ,I(x,y)*LGF s,o odd ]

[0016] Among them, LGF s,o even and LGF s,o odd Let represent even-symmetric and odd-symmetric Log-Gabor wavelets at scale s and direction o, respectively, and * denote convolution operation. Then, the magnitude A at pixel I(x,y) is calculated. s,o (x,y) and phase angle The response value is calculated using the following formula:

[0017]

[0018]

[0019] The definition of phase coherence is:

[0020]

[0021]

[0022] Where PC(x,y) represents the phase consistency response value at pixel I(x,y), W o (x, y) are weighting factors. T is a preset noise threshold. This indicates rounding down, where ε is the local minimum constant. The average phase angle.

[0023] Furthermore, in step (2), feature point extraction specifically includes the following steps:

[0024] (201) Using the response values ​​of filters of different directions and scales, calculate the maximum moment and minimum moment of each pixel response value to form a maximum moment map and a minimum moment map of the same size as the original image. Weighted superposition of the maximum moment map and the minimum moment map, and normalization to obtain the superimposed moment map.

[0025] (202) Construct a scale space for the superimposed matrix, divide each scale image into non-overlapping image blocks of equal size, extract Harris corner points in each image block, if the number of extracted corner points is less than n, reduce the Harris corner point extraction threshold and re-extract corner points, where n is a preset constant, and finally use the non-maximum suppression method to filter feature points.

[0026] Furthermore, in step (201), the construction process of the overlay matrix diagram includes the following steps:

[0027] The maximum and minimum moment diagrams are calculated using phase consistency information, using the following formula:

[0028]

[0029]

[0030] In the formula:

[0031]

[0032]

[0033]

[0034] The method for superimposing the maximum and minimum matrix plots is as follows:

[0035] M'=αM+(1-α)m

[0036] Where α is a weighting factor, and then M' is normalized to obtain the superimposed matrix.

[0037] Furthermore, in step (3), the feature descriptor is constructed as follows:

[0038] (301) Set the convolution sequence of the Log-Gabor filter to have N s Each scale and N o Construct a maximum index graph in each direction at each scale s.

[0039]

[0040] Among them, A s,o (x,y) is the response value of the filter with scale s and direction o at the image pixel (x,y), k Ns (x, y) is the maximum orientation index at scale s, with values ​​ranging from 1 to N. o , This forms a multi-scale maximum index map.

[0041] (302) Constructing phase features, specifically including: in the multi-scale maximum index map, dividing the selected rectangular region centered on the feature point into 6×6 sub-regions, each sub-region consisting of 16×16 pixels. In each sub-region, performing histogram statistics on the multi-scale maximum index map, and finally calculating N... s The results from each scale are weighted and accumulated. The specific calculation process is as follows:

[0042]

[0043] Among them, V s Let α be the phase feature at scale s, V be the phase feature of the sub-region, and α be the phase feature at scale s. s It is a scaling factor, and

[0044]

[0045] Then, the features of each sub-region are connected. This is because the values ​​of the index graph are 1 to N. o Each histogram has a size of N. o Then the length of the phase feature portion of each feature descriptor is 6×6×N. o .

[0046] (303) Constructing edge features, specifically including: the edge map is obtained from the maximum moment map of phase consistency; a rectangular region is selected on the edge map centered on the feature point, and divided into 4×4 sub-regions, each sub-region consisting of 16×16 pixels. N is statistically analyzed in each sub-region. edgeThe gradient orientation histograms are generated in each direction, and then the features of the sub-regions are concatenated. Each feature descriptor has a dimension of 4×4×N. edge N edge This is a preset constant.

[0047] (304) Normalize the two sets of descriptors and then concatenate them to form a new descriptor with a dimension of (6×6×N). o +4×4×N edge ).

[0048] Furthermore, in step (4), the specific process of feature matching is as follows:

[0049] (401) Assuming that there are m feature points in the infrared image and n feature points in the visible light image, calculate the Euclidean distance between the m feature points in the infrared image and the n feature points in the visible light image in sequence, and take the point with the smallest Euclidean distance as the matching point.

[0050] (402) Use the Fast Matching (FSC) algorithm to remove outliers and obtain the optimal transformation matrix.

[0051] Beneficial effects: This invention overcomes the problem of registration failure caused by the large difference between infrared and visible light images in traditional methods, and achieves high-precision and robust infrared and visible light image registration. Attached Figure Description

[0052] Figure 1 This is the basic flowchart of the present invention. Detailed Implementation

[0053] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0054] This invention proposes a registration method for infrared and visible light images based on phase and edge features. The registration flowchart is shown below. Figure 1 As shown, it includes the following steps:

[0055] Step 1: Acquire infrared and visible light images of the same scene, perform scale preprocessing on the images, and then use a Log-Gabor filter to perform convolution calculations on the infrared and visible light images respectively to obtain phase consistency information;

[0056] Step 2: Based on the phase consistency information of the infrared and visible light images, obtain the maximum and minimum moment maps of the infrared and visible light images respectively. Then, weightedly superimpose the maximum and minimum moment maps to obtain the corresponding superimposed moment map. Construct a scale space for the superimposed moment map, extract feature points uniformly in blocks on images of different scales, and finally filter feature points using the non-maximum suppression method.

[0057] Step 3: Based on the multi-angle, multi-scale convolutional sequences in the phase consistency information of infrared and visible light images, calculate the maximum orientation index at each scale, construct a multi-scale maximum index map, and construct feature descriptors for feature points based on the multi-scale maximum index map and phase consistency edge map of infrared and visible light images.

[0058] Step 4: Match feature points using Euclidean distance between feature descriptors, then filter out mismatched points, and finally obtain the optimal transformation matrix between infrared and visible light images.

[0059] In this embodiment, step one can be implemented using the following preferred method:

[0060] (1) For the input infrared and visible light images, downsampling of the higher resolution image makes the infrared and visible light images have the same resolution, which can reduce the scale difference between the infrared and visible light images.

[0061] (2) For the input image I(x,y), a two-dimensional Log-Gabor filter (LGF) is used to convolve the image:

[0062] [e s,o (x,y),o s,o (x,y)]=[I(x,y)*LGF s,o even ,I(x,y)*LGF s,o odd ]

[0063] Among them, LGF s,o even and LGF s,o odd Let represent even-symmetric and odd-symmetric Log-Gabor wavelets at scale s and direction o, respectively, and * denote convolution operation. Then, the magnitude A at each pixel I(x,y) is calculated. s,o (x,y) and phase angle The response value is calculated using the following formula:

[0064]

[0065]

[0066] The definition of phase coherence is as follows:

[0067]

[0068]

[0069] Where PC(x,y) represents the phase consistency response value at pixel I(x,y), W o(x, y) are weighting factors. T is a preset noise threshold. This indicates rounding down, where ε is the local minimum constant. The average phase angle.

[0070] In this embodiment, the specific process of uniformly extracting corner points using the superimposed matrix diagram in step two is as follows:

[0071] (1) Using the response values ​​of filters of different directions and scales, calculate the maximum and minimum moments of each pixel to construct the maximum moment map and the minimum moment map. The calculation method is as follows:

[0072]

[0073]

[0074] In the formula:

[0075]

[0076]

[0077]

[0078] The method for superimposing the maximum and minimum matrix plots is as follows:

[0079] M'=αM+(1-α)m

[0080] Where α is a weighting factor, and then M' is normalized to obtain the superimposed matrix.

[0081] (2) Construct a scale space for the superimposed matrix image, divide each scale image into non-overlapping image blocks of equal size, extract Harris corner points in each image block, if the number of extracted corner points is less than n, reduce the Harris corner point extraction threshold and re-extract corner points, where n is a preset constant, and finally use the non-maximum suppression method to filter feature points.

[0082] In this embodiment, the method for constructing the feature descriptor in step three is as follows:

[0083] (1) Set the convolution sequence of the Log-Gabor filter to have N s Each scale and N o Construct a maximum indexed graph in each of the following directions at each scale s:

[0084]

[0085] Among them, A s,o (x,y) is the response value of the filter with scale s and direction o at the image pixel (x,y), k Ns(x, y) is the maximum orientation index at scale s, with values ​​ranging from 1 to N. o , This forms a multi-scale maximum index map.

[0086] (2) Constructing phase features, specifically including: In the multi-scale maximum index map, dividing the selected rectangular region centered on the feature point into 6×6 sub-regions, each sub-region consisting of 16×16 pixels. For each sub-region, statistical analysis of the multi-scale maximum index map is performed, and finally, N... s The results from each scale are weighted and accumulated. The specific calculation process is as follows:

[0087]

[0088] Among them, V s Let α be the phase feature at scale s, V be the phase feature of the sub-region, and α be the phase feature at scale s. s It is a scaling factor, and

[0089]

[0090] Then, the features of each sub-region are connected. This is because the values ​​of the index graph are 1 to N. o Each histogram has a size of N. o Then the length of the phase feature portion of each feature point descriptor is 6×6×N. o .

[0091] (3) Constructing edge features, specifically including: the edge map is obtained from the maximum moment map of phase consistency; a rectangular region is selected on the edge map centered on the feature point, and divided into 4×4 sub-regions, each sub-region consisting of 16×16 pixels. N is statistically analyzed in each sub-region. edge The gradient orientation histogram has 4×4×N directions and a feature dimension of 4×4×N. edge N edge This is a preset constant.

[0092] (4) Normalize the two sets of descriptors and then concatenate them to form a new descriptor with a dimension of (6×6×N). o +4×4×N edge ).

[0093] In this embodiment, the specific process of step four is as follows:

[0094] (1) Assuming that there are m feature points in the infrared image and n feature points in the visible light image, calculate the Euclidean distance between the m feature points in the infrared image and the n feature points in the visible light image in sequence, and take the point with the smallest Euclidean distance as the matching point.

[0095] (2) Use the Fast Matching (FSC) algorithm to remove outliers and obtain the optimal affine transformation matrix.

[0096] The above description is merely an illustration of the technical concept of the present invention in conjunction with specific preferred embodiments, and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for infrared and visible light image registration based on phase and edge features, characterized in that, Includes the following steps: Step 1: Acquire infrared and visible light images of the same scene, perform scale preprocessing on the infrared and visible light images, and then use filters to perform convolution calculations on the infrared and visible light images respectively to obtain the phase consistency information of the infrared and visible light images; Step 2: Based on the phase consistency information of the infrared image and the visible light image, obtain the maximum and minimum moment maps of the infrared image and the visible light image respectively. Then, weightedly superimpose the maximum and minimum moment maps to obtain the superimposed moment map of the infrared image and the visible light image. Construct a scale space for the superimposed moment map, extract feature points uniformly in blocks on images of different scales, and finally filter feature points using the non-maximum suppression method. Step 3: Based on the multi-angle, multi-scale convolutional sequences in the phase consistency information of infrared and visible light images, construct a multi-scale maximum index map, and construct feature descriptors for feature points based on the multi-scale maximum index map and phase consistency edge map of infrared and visible light images; Step 4: Match feature points using the Euclidean distance between feature descriptors, then filter out mismatched points, and finally obtain the optimal transformation matrix between infrared and visible light images; The implementation process of step one is as follows: (1.1) Downsample the infrared image and the visible light image to make the resolution of the infrared image and the visible light image the same; (1.2) A two-dimensional Log-Gabor filter is used to perform convolution calculation on the downsampled infrared image and the visible light image to obtain the response value and phase consistency information of the two-dimensional Log-Gabor filter for each pixel in the downsampled infrared image and the visible light image in different directions and scales. The process of obtaining convolutional information is as follows: [e s,o (x,y),o s,o (x,y)]=[I(x,y)*LGF s,o even ,I(x,y)*LGF s,o odd ] Among them, LGF s,o even and LGF s,o odd Let I represent the even-symmetric and odd-symmetric Log-Gabor wavelets at scale s and direction o, respectively, and * denote the convolution operation; then calculate the magnitude A at pixel I(x,y). s,o (x,y) and phase angle The response value is calculated using the following formula: The definition of phase coherence is: Where PC(x,y) represents the phase consistency response value at pixel I(x,y), W o (x,y) are weighting factors; T is a preset noise threshold. This indicates rounding down, where ε is the local minimum constant. The average phase angle.

2. The infrared and visible light image registration method based on phase and edge features according to claim 1, characterized in that, Step two involves extracting feature points, which includes the following steps: (2.1) Calculate the maximum and minimum moments of the response values ​​to form a maximum moment map and a minimum moment map with the same resolution as the infrared and visible light images after downsampling. Weight the maximum and minimum moment maps and normalize them to obtain the superimposed moment maps of the infrared and visible light images respectively. (2.2) Scale space is constructed from the superimposed matrix of the infrared image and the visible light image to obtain scale images. Each scale image is divided into non-overlapping image blocks of equal size. Corner points are extracted in each image block using the Harris corner detection algorithm. If the number of extracted corner points is less than n, the threshold for corner point extraction by the Harris detection algorithm is reduced and corner points are extracted again, where n is a preset constant. Finally, feature points are screened using the non-maximum suppression method to obtain the feature points of the infrared image and the visible light image.

3. The infrared and visible light image registration method based on phase and edge features according to claim 2, characterized in that, Step three, the construction of feature descriptors, includes: (3.1) Set the multi-angle, multi-scale convolutional sequence in the phase consistency information to have N s Each scale and N o In each direction and at each scale, a maximum index map is constructed, calculated as follows: Among them, A s,o (x,y) is the response value of a two-dimensional Log-Gabor filter at pixel (x,y) in infrared and visible light images at scale s and direction 0, where k Ns (x,y) is the maximum index map of pixel (x,y) at scale s. This forms a multi-scale maximum index map; (3.2) Constructing phase features: In the multi-scale maximum index map, the selected rectangular region centered on the feature point is divided into 6×6 sub-regions, each sub-region consisting of 16×16 pixels; in each sub-region, histogram statistics are performed on the multi-scale maximum index map, and finally N... s The results from each scale are weighted and accumulated. The specific calculation process is as follows: Among them, V s Let α be the phase feature at scale s, V be the phase feature of the sub-region, and α be the phase feature at scale s. s For scale weights, and Then the phase features of each sub-region are connected, since the maximum index map has values ​​from 1 to N. o The histogram size obtained for each sub-region is N. o Then the phase feature length of each feature point descriptor is 6×6×N. o ; (3.3) Construct edge features and obtain an edge map, which is the maximum rectangle obtained in step (2.1). Select a rectangular region centered on the feature point on the edge map and divide it into 4×4 sub-regions. Each sub-region consists of 16×16 pixels. Count N in each sub-region. edge If the gradient direction histogram is calculated in N directions, then the phase feature length of each feature point descriptor is 4×4×N. edge N edge This is a preset constant; (4) Normalize the two sets of descriptors and then concatenate them to form a new descriptor. The phase feature length of the new descriptor is (6×6×N). o +4×4×N edge ).

4. The infrared and visible light image registration method based on phase features and edge features according to claim 3, characterized in that, Step four, feature matching, includes the following steps: (4.1) Assuming that the infrared image has m feature points and the visible light image has n feature points, calculate the Euclidean distance between the m feature points of the infrared image and the n feature points of the visible light image in turn, and take the point with the smallest Euclidean distance as the matching point. (4.2) Use the Fast Matching (FSC) algorithm to remove mismatched points and obtain the optimal transformation matrix.

Images