An image registration method that can adapt to thermal corona effect and nonlinear radiation differences
By combining edge-window Gaussian pyramid and asymmetric spatial offset techniques with Log-Gabor frequency response and phase consistency, the edge positioning drift and nonlinear radiation differences caused by infrared image thermal halo effect are solved, achieving high-precision registration between infrared and visible light images and improving the accuracy and robustness of the matching.
Patent Information
- Application Number
- CN202611620833.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-20
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies cannot simultaneously solve the problems of edge positioning drift, nonlinear grayscale inversion, and background thermal noise interference caused by thermal halo effect in infrared images, resulting in a small number of matching points or a high mismatch rate in the registration of infrared and visible light images.
By employing side-window Gaussian pyramid and asymmetric spatial offset techniques, feature descriptors are constructed using Log-Gabor frequency response and phase consistency information. Combined with asymmetric physical correction and phase weighting, feature point localization drift in infrared images is corrected, and mismatched points are eliminated, achieving sub-pixel-level geometric correction.
It improves the accuracy and robustness of infrared and visible light image registration, enhances the stability of cross-modal features, overcomes the effects of thermal halo and nonlinear radiation differences, and achieves high-precision image registration.
Smart Images

Figure CN123118705A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image processing technology, and particularly to the fields of multimodal remote sensing image processing and heterogeneous image fusion, specifically to an image registration method that can adapt to thermal coherence effects and nonlinear radiation differences. Background Technology
[0002] The fusion technology of infrared and visible light images has significant application value in fields such as drone reconnaissance, security monitoring, autonomous driving, and disaster relief. Visible light images possess rich texture details and color information, which better aligns with human visual habits; while infrared images, based on the thermal radiation of objects, can operate in all weather conditions and penetrate smoke, highlighting thermal targets. High-precision registration and fusion of the two can achieve complementary advantages. However, in practical applications, the registration of infrared and visible light images faces considerable challenges.
[0003] First, nonlinear radiometric differences (NRD) are one of the main obstacles. Due to the drastically different imaging mechanisms, the grayscale representation of the same scene is often inconsistent in the two modalities, and even a "grayscale inversion" phenomenon may occur (for example, a dark vehicle in visible light may appear bright white in an infrared image due to engine heat). Traditional gradient-based registration algorithms (such as SIFT and SURF) rely on the consistency of gradient directions, which often fails when processing such multimodal images, resulting in a sparse number of matching points or an extremely high mismatch rate.
[0004] Secondly, the unique "thermal blooming" effect of infrared images leads to severe edge localization errors. Infrared imaging systems are limited by the physical characteristics of the sensor and the influence of atmospheric transmission; the thermal radiation from high-temperature objects tends to diffuse into surrounding pixels, causing the edges of objects in infrared images to become blurred and indistinct, and the edge positions often expand outwards. Existing registration methods (such as RIFT based on phase consistency and HOPC based on structural features) overcome grayscale differences to some extent through frequency domain features, but they typically assume that feature points have strict geometric center consistency across different modalities. This means that existing algorithms, when extracting feature descriptors, still use the detected feature points as the geometric center and employ symmetrical sampling patterns (such as circular or rectangular neighborhoods) for information acquisition.
[0005] However, in scenarios with thermal halo effects, the "corner points" or "edge points" detected by infrared images often deviate from the true physical boundaries of the object. If the traditional center-symmetric sampling strategy continues to be used, the sampling window of the descriptor will inevitably cover a large amount of irrelevant background noise or blurred areas, resulting in a decrease in the discriminative power of the generated feature descriptor. Although the recently proposed side window filtering technique has solved the edge preservation problem to some extent, existing side window algorithms are mainly used for image smoothing or edge detection, and no research has yet applied them to adaptive correction of the physical center of the feature descriptor. That is, there is a lack of a mechanism that can actively "push" the sampling center of the descriptor towards a stable region inside the object based on the blur direction of the edge.
[0006] Furthermore, existing multimodal descriptors (such as GIFT and LGHD) often assign distance-based Gaussian weights to all pixels in the neighborhood during the construction process, ignoring the reliability of the pixels themselves. In infrared images, background regions are often accompanied by speckle noise or thermal noise. If these noise points are included in the descriptor construction without differentiation, the robustness of the matching will be further reduced.
[0007] In summary, existing technologies cannot simultaneously solve the three major problems of edge positioning drift caused by infrared thermal halo, nonlinear grayscale inversion, and background thermal noise interference. Therefore, there is an urgent need for an image registration method that can actively correct physical position and has anti-inversion and anti-noise capabilities. Summary of the Invention
[0008] To address the aforementioned technical problems, this invention provides an infrared and visible light image registration method that can adapt to the effects of thermal halo and nonlinear radiation differences. It aims to use side-window Gaussian pyramids and asymmetric spatial offset techniques to physically correct feature point positioning drift caused by infrared thermal halo, forcibly repositioning the sampling center from unstable, blurred edges to a flat area inside the object, thereby achieving sub-pixel-level geometric correction.
[0009] According to a first aspect of the present invention, the present invention first provides an image registration method that can adapt to thermal coherence effects and differences in nonlinear radiation, comprising: Step S1: Input the visible light image and infrared image to be registered, perform multi-scale Log-Gabor analysis on the visible light image and infrared image respectively, construct Log-Gabor frequency response map and phase consistency map, and construct a reliability weight matrix from the phase consistency map; Step S2: Based on the Log-Gabor frequency response map obtained in Step S1, a multi-layered multi-window Gaussian pyramid scale space is constructed using a side-window Gaussian filter (SWGF). Each layer of the image represents a pyramid image at the corresponding scale. Feature points are detected on each layer of the image in the scale space, and the optimal side-window index corresponding to each feature point is obtained. The optimal side-window index is the pointing index of the feature point relative to the direction of the internal flat region of the object edge. Step S3: Perform asymmetric physical correction on the feature points; wherein, the object of the asymmetric physical correction is the local structural shift caused by the difference in thermal radiation imaging between the infrared image and the visible light image. A spatial offset vector pointing to the texture-flat region in the neighborhood of the feature point is calculated based on the optimal side window index, and the sampling center of the feature point is moved from its original position using the spatial offset vector. Translate to repositioning center ; Step S4: Construct a phase-weighted asymmetric descriptor at the relocation center, establish a local sampling grid, and combine it with the reliability weight matrix generated in step S1. The amplitude of the Log-Gabor frequency response plot Weighted encoding is performed to generate feature descriptors for infrared and visible light images; Step S5: Calculate the similarity between the feature descriptors of the infrared image and the visible light image using a normalized cross-correlation metric. Obtain the initial set of matching point pairs between the feature descriptors of the two; Step S6: Construct position-direction-scale geometric feature vectors, use density clustering algorithm to remove mismatched points in the initial matching point set due to smoke flow or local parallax, calculate the affine transformation matrix from infrared image to visible light image, and use the affine transformation matrix to perform spatial transformation on the image to be registered, thus completing the registration of infrared and visible light images.
[0010] Furthermore, the construction of the Log-Gabor frequency response map and phase consistency map in step S1 specifically includes: using a multi-scale, multi-directional Log-Gabor filter to convolve the image to obtain each pixel. Frequency response components at scale s and direction o And calculate its corresponding amplitude. ; Based on the above amplitude The phase consistency at each pixel is calculated using the following formula. :
[0011] At the same time, the total amplitude of the Gabor frequency response corresponding to this pixel is calculated using the following formula. :
[0012] in, The frequency domain response amplitude function is the log-Gabor frequency domain response function. It is the phase deviation function, where T is the noise threshold. It is the frequency spread weighting factor. To prevent small constants where the numerator is zero; the aforementioned That is, as described in step S4 The value is taken at pixel x.
[0013] Further, in step S2, the construction of a multi-layered Gaussian pyramid scale space with side windows specifically includes: defining a side window model with 8 directions, including up, down, left, right, and four diagonal directions; for each layer of the image in the pyramid, filtering is performed using the side window model with the 8 directions respectively, and the pixel gray-level variance within each side window is calculated; the side window with the smallest variance is selected as the optimal side window, the filtering result of the optimal side window is used as the output value of the current pixel, and the direction index of the optimal side window is recorded as the optimal side window index; the next set of images is constructed through downsampling operation, and the above filtering steps are repeated to form a multi-resolution pyramid structure that preserves edge features.
[0014] Furthermore, the specific formula for calculating the spatial offset vector pointing to the texture flat region within the neighborhood of the feature point is as follows:
[0015] in Indicates the first The spatial offset vector corresponding to the best side window index, where α is the preset offset coefficient and σ is the scale value of the current feature point. To point to the first The unit direction vector of the geometric center of each side window; the repositioning The calculation formula is: ,in These are the original coordinates of the feature point.
[0016] Further, in step S4, generating feature descriptors for the infrared image and the visible light image specifically includes: using the repositioning center... Establish a logarithmic polar coordinate grid around the origin; calculate the characteristic statistics of each sub-region within the grid. The formula is as follows:
[0017] in, For the set of pixels within the sub-region, For Gaussian weighted parameters, This represents the phase consistency weight corresponding to that point. The value is the Log-Gabor frequency response amplitude corresponding to that point; the feature statistics of all sub-regions are combined into a vector, sorted along the main feature direction, and then normalized by L2 to obtain the feature descriptor.
[0018] Further, in step S5, the formula for calculating the normalized cross-correlation metric is:
[0019] in, and These are the feature descriptor vectors of the reference image and the image to be registered, respectively. The L2 norm is represented; the point with the largest NCC value is selected as the matching point, and the initial matching point pair is screened by combining the two-way ratio test.
[0020] Furthermore, in step S6, the elimination using the density clustering algorithm specifically includes: for each pair of initial matching points, calculating their positional deviation. Logarithmic scale ratio Difference from the main direction Construct a four-dimensional feature vector In four-dimensional space, cluster centers are found, and matching points within the neighborhood of each cluster center are retained as interior points, while other outliers are removed. The specific formula is as follows:
[0021] in, These represent the positional deviations of the matching point pair in the x and y directions, respectively. To match the scale ratio of point pairs, Its logarithm; The principal direction difference for matching point pairs.
[0022] According to a second aspect of the present invention, an electronic device is provided, the electronic device including a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the above-described method.
[0023] According to a third aspect of the present invention, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0024] Compared with the prior art, the advantages of the present invention are: 1. This invention utilizes Log-Gabor frequency response and phase consistency information to construct a basic feature map, which can enhance the structural consistency between infrared and visible light images and improve cross-modal feature stability.
[0025] 2. This invention combines Gaussian filtering with optimal window indexing to achieve asymmetric correction of local structural shifts caused by differences in thermal radiation, thereby improving feature localization accuracy.
[0026] 3. This invention improves the accuracy and robustness of infrared and visible light image registration through phase-weighted feature description and mismatch elimination mechanism. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a schematic diagram of an image registration method that can adapt to the differences between thermal coma and nonlinear radiation, provided by an embodiment of the present invention. Figure 2 This is a block diagram of the infrared and visible light image registration system according to an embodiment of the present invention; Figure 3 Examples of infrared and visible light images according to embodiments of the present invention; Figure 4 A magnified comparison of the local area before and after relocation of the sampling center of feature points in the edge thermal halo region of an infrared image; Figure 5 A schematic diagram illustrating the construction process and log-polar coordinate partitioning sampling for the phase-weighted asymmetric descriptor (PW-ASWD). Figure 6 This is a schematic diagram of feature matching connections between infrared and visible light images according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the infrared and visible light image registration and fusion results according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the registered checkerboard mosaic fusion effect in an embodiment of the present invention. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0030] It should be noted that the illustrations provided in the embodiments of the present invention are only schematic representations of the basic concept of the present invention. Therefore, the illustrations only show the components related to the present invention and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0031] In this invention, it should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used only for descriptive and distinguishing purposes and should not be construed as indicating or implying relative importance.
[0032] Furthermore, it should be noted that the features of the various embodiments of the present invention can be combined or integrated in whole or in part, and as those skilled in the art will understand, they can interact and operate in different ways. Each embodiment can be implemented independently of each other or in association with one another.
[0033] This invention provides an image registration method that adapts to thermal halo effects and nonlinear radiation differences, aiming to solve the problems of inaccurate positioning due to blurred edges in infrared images caused by thermal halo effects, and high mismatch rates between multimodal images due to nonlinear radiation differences. Figure 1 and Figure 2 As shown, where Figure 1 This is a schematic diagram of the registration method. Figure 2 To register the system structure diagram, this method mainly includes the following steps: S1. Input the visible light image to be registered. With infrared images Multi-scale Log-Gabor analysis was performed on the visible light and infrared images respectively to construct Log-Gabor frequency response maps and phase consistency maps, and a reliability weight matrix was constructed from the phase consistency maps. ; S2. Based on the Log-Gabor frequency response map obtained in step S1, a multi-layered side-window Gaussian pyramid scale space is constructed using a side-window Gaussian filter (SWGF). Each layer of the image represents a pyramid image at the corresponding scale. Feature points are detected on each layer of the image in the scale space. And obtain the optimal side window index corresponding to each feature point. The optimal edge window index is the pointing index of the feature point relative to the direction of the internal flat area of the object edge; the object refers to a continuous region in the image defined by the gray-level gradient change, with a closed or semi-closed edge contour, and its interior has relatively uniform gray-level or texture distribution characteristics. In infrared images, it appears as an imaging region with similar thermal radiation intensity, and in visible light images, it appears as an imaging region with similar reflectivity. S3. Perform asymmetric physical correction on the feature points; wherein, the object of the asymmetric physical correction is the local structural shift caused by the difference in thermal radiation imaging between the infrared image and the visible light image, and calculates the spatial offset vector pointing to the texture flat region in the neighborhood of the feature point based on the optimal side window index. The spatial offset vector is used to move the sampling center of the feature points from their original positions. Translate to repositioning center ; S4. Construct a phase-weighted asymmetric descriptor at the relocation center, establish a local sampling grid, and combine it with the reliability weight matrix generated in step S1. The amplitude of the Log-Gabor frequency response plot Weighted encoding is performed to generate feature descriptors for infrared and visible light images; S5. Calculate the similarity between feature descriptors of infrared and visible light images using a normalized cross-correlation metric. Obtain the initial set of matching point pairs between the feature descriptors of the two; S6. Construct position-direction-scale geometric feature vectors Density clustering algorithm is used to remove mismatched points in the initial matching point set caused by smoke flow or local parallax. The affine transformation matrix from infrared image to visible light image is then calculated. The affine transformation matrix is used to perform spatial transformation on the image to be registered, thereby completing the registration of infrared and visible light images.
[0034] In one embodiment of the present invention, firstly, in step S1, a visible light image of the same scene is acquired. With infrared images This addresses additive thermal noise in infrared images (such as thermal airflow disturbances around a fire) and grayscale inversion between visible and infrared images. Figure 3 The two images were processed as follows: Constructing the Log-Gabor frequency response map: The image is convolved using a multi-scale (e.g., scale S=4) and multi-directional (direction O=6) Log-Gabor filter bank to obtain the amplitude response of each pixel in the frequency domain. The amplitudes from all scales and directions are then superimposed to obtain the overall Log-Gabor frequency response amplitude map. This feature primarily reflects the texture energy of an image and is naturally robust to changes in illumination and grayscale inversion.
[0035] The following formula calculates the total amplitude of the Gabor frequency response corresponding to this pixel. :
[0036] Constructing a phase consistency weight map: Based on the Log-Gabor filter components, calculate the phase consistency (PC) value for each pixel to generate a phase consistency map. The values (0-1) in this figure reflect the structural saliency of the image features. A higher value indicates that the point is more likely to be an object edge or corner, while a lower value indicates that the point is more likely to be background noise.
[0037] Based on the above amplitude The phase consistency at each pixel is calculated using the following formula. :
[0038] in, The frequency domain response amplitude function is the Log-Gabor function. It is the phase deviation function, where T is the noise threshold. It is the frequency spread weighting factor. To prevent small constants where the numerator is zero; the aforementioned That is, as described in step S4 At pixel The value is taken at that location.
[0039] In step S2, a Gaussian pyramid of the side window is constructed and feature points are detected. To accurately locate the edge of the infrared heat source at different scales, a Gaussian pyramid scale space of the side window is constructed: Pyramid Construction: A Side Window Gaussian Filter (SWGF) is used instead of a traditional Gaussian kernel to perform multiple sets and layers of downsampling and filtering on the image. SWGF calculates the gray-level variance within eight side windows surrounding a pixel and selects the side window with the smallest variance as the filtering criterion. Specifically, a pixel is defined... These correspond to the left, right, top, bottom, and four diagonal directions, respectively. Calculate each side window. gray variance within And determine the optimal filtered output:
[0040]
[0041] In the formula, This represents the total number of pixels in the side window. For the first The coverage area of each side window This is the average grayscale value within the side window. This allows the pyramid to maintain the sharpness of the fire point edges and prevent heat halo diffusion even in high-resolution (low-resolution) images.
[0042] Feature detection and indexing: At each layer of the pyramid image, corner points are detected as feature points using the Harris operator. The Harris corner response value for each pixel is calculated. :
[0043] in The autocorrelation matrix is... This is an empirical constant (usually taken as 0.04-0.06). Specifically, during the detection process, each feature point is recorded simultaneously. Corresponding optimal side window index :
[0044] That is, the direction index with the smallest variance during SWGF filtering, which physically indicates the direction of the feature point relative to the flat area inside the object's edge.
[0045] In step S3, the asymmetric spatial offset vector (physical correction) is calculated. To address the virtual edge phenomenon caused by thermal radiation at the fire point edges in the infrared image, physical relocation of the feature point sampling center is performed. Calculate the offset vector: based on the optimal side window index recorded in step S2. Calculate the spatial offset vector pointing to the flat region of texture inside the object. The calculation formula is: in This is the offset coefficient (0.5 in this embodiment). The scale at which the current feature point is located. To point to the first The unit vector of the geometric center of each side window.
[0046] Calculate the relocation center of feature points Through this step, the sampling center of the feature points is physically shifted from the unstable, blurred edges to a flat, textured area of the object (such as the center of a flame or the inside of a tree trunk), thus avoiding the positioning error caused by edge thermal halos. Example results are shown below. Figure 4 .
[0047] Physical relocation: using the formula:
[0048] in Indicates the first The spatial offset vector corresponding to the optimal side window index. α is the preset offset coefficient, and σ is the scale value of the current feature point. To point to the first The unit direction vector of the geometric center of each side window; the repositioning The calculation formula is: ,in These are the original coordinates of the feature point.
[0049] In step S4, the physical relocation of the feature point sampling center is completed according to step S3 (obtaining the relocation center). After that, in order to extract features with rotation invariance and radiation resistance differences, the PW-ASWD descriptor is constructed according to the following steps: Determine the principal orientation of feature points (imparting rotation invariance): To ensure the descriptor is unaffected by image rotation, the local principal orientation of feature points needs to be calculated. This is done at the repositioning center. Within the neighborhood of a pixel, the gradient direction of the Log-Gabor frequency response is statistically analyzed. A pixel is defined. gradient value and gradient , 0° 360° direction is divided into DIR columns, Gaussian weighted window statistical histogram of directions. Main direction. The direction corresponding to the peak value of the histogram is determined by the following formula:
[0050] Where Ω represents the neighborhood range. When constructing the descriptor mesh subsequently, the coordinate system is rotated to this principal direction to achieve rotation invariance.
[0051] Establish a logarithmic polar coordinate sampling grid: such as Figure 5 As shown, with the repositioning center A logarithmic polar coordinate grid is established with the origin as the origin. This grid contains... A radial ring and Each angle sector ( The overall plan is divided into = × Within each sub-region (Bin), weighted response values at four scales are calculated. Compared to traditional rectangular grids, the logarithmic polar coordinate grid features denser sampling in the central region and sparser sampling in the peripheral region. This design better aligns with human visual characteristics, prioritizing the capture and preservation of stable texture structures in the central region even when infrared thermal halo causes peripheral blurring. For any sampling point within the neighborhood... Calculate its polar radius ρ and polar angle φ in the rotated coordinate system. Determine the sub-region index to which it belongs:
[0052] Weighted encoding: when determining any sampling point Sub-region index Then, in each sub-region For binning the histogram, weighted cumulative statistics are performed on sampling points falling into the same sub-region. A phase consistency map is introduced during this process. As a reliability weight, the Log-Gabor frequency amplitude generated in step S1 Weighting is applied. For thermal noise in the infrared background ( (When the value is low), its contribution to the descriptor is automatically suppressed; for true structural features ( (Higher values) enhance their contribution. A 128-dimensional feature vector is ultimately generated and sorted along the principal direction to achieve rotation invariance. Specifically, for the ... Sub-regions Histogram statistics are defined as follows:
[0053] in, For the relocated sampling center, For Gaussian weighted parameters, This represents the phase consistency weight corresponding to that point. This represents the Log-Gabor frequency response amplitude at that point. Therefore, The histogram statistical results are constructed by dividing the sub-regions into bins; the statistical values of all sub-regions are arranged in angular order after being aligned with the main direction and then concatenated sequentially to obtain the final feature descriptor vector. .
[0054] Descriptor vector generation and normalization: All The feature statistics of each subregion are arranged into a vector. To eliminate the overall contrast difference between infrared and visible light images caused by exposure differences and sensor gain variations, L2 normalization was performed on the vectors to obtain the final 128-dimensional feature descriptor. :
[0055] Through the above steps, the generated descriptor not only contains the fine texture of the object's interior (logarithmic sampling) but also effectively removes background thermal noise (phase weighting) and is unaffected by grayscale inversion (Log-Gabor amplitude), such as... Figure 6 As shown.
[0056] In step S5, correlation-based anti-inversion matching addresses the nonlinear radiometric difference between the visible light image (white smoke) and the infrared image (black background) by employing Normalized Cross-Correlation (NCC) as a similarity metric, replacing the traditional Euclidean distance. The NCC metric focuses on the waveform trend of feature vectors rather than absolute intensity, effectively matching corresponding feature points with gray-scale inversion characteristics. The NCC score between the descriptors of the image to be registered and the reference image is calculated, and combined with a bidirectional ratio test, an initial set of matching point pairs is obtained.
[0057] The formula for calculating the normalized cross-correlation metric is as follows:
[0058] in, and These are the feature descriptor vectors of the reference image and the image to be registered, respectively. The L2 norm is represented; the point with the largest NCC value is selected as the matching point, and the initial matching point pair is screened by combining the two-way ratio test.
[0059] In step S6, to eliminate mismatched points caused by smoke flow or local parallax and to calculate the final registration parameters, this step performs the following operations: Constructing geometric feature vectors: For each pair of initial matching points obtained in step S5 (let the infrared feature points be...), Visible light feature points are Calculate its positional deviation, scale ratio, and principal orientation difference to construct a four-dimensional geometric feature vector. :
[0060] in, This is the coordinate translation amount. As a feature scale, Rotation angle in the main direction.
[0061] Geometric consistency screening (outlier removal): finding the center of the densest region of samples in four-dimensional space. Calculate the vector corresponding to each matching point. Euclidean distance from the center If the distance is less than the preset threshold If the match is true, it is considered an interior point (correct match); otherwise, it is considered an outlier (false match).
[0062] Only matching points that meet the above conditions are retained for subsequent calculations.
[0063] Affine transformation matrix calculation: Using the filtered set of interior points, the affine transformation matrix from the infrared image to the visible light image is solved by the least squares method. Assuming points on an infrared image Points mapped onto a visible light image The transformation formula is as follows:
[0064] The coordinate transformation equations after expansion are:
[0065] in, , , , For rotation and scaling parameters, , These are the horizontal and vertical translation parameters. Based on the obtained matrix... By performing spatial transformation on infrared images, precise registration with visible light images can be achieved, such as... Figure 7 and Figure 8 As shown, where Figure 7 This is a schematic diagram of the registration and fusion results of infrared and visible light images. Figure 8 This is a schematic diagram of the inlaid and blended effect of the registered checkerboard pattern.
[0066] In summary, the image registration method proposed in this invention, which can adapt to thermal coherence effects and nonlinear radiation differences, systematically solves the multimodal registration problem in complex scenarios such as UAV fire detection through innovative asymmetric physical correction and phase weighted description strategies.
[0067] Specifically, this embodiment of the invention utilizes a side-window Gaussian pyramid and asymmetric spatial offset techniques to physically correct feature point localization drift caused by infrared thermal halo effects, forcibly repositioning the sampling center from unstable, blurred edges to a flat area inside the object, achieving sub-pixel-level geometric correction. Simultaneously, by constructing a PW-ASWD descriptor, it cleverly combines the anti-reversal characteristics of Log-Gabor frequency amplitude with the anti-noise characteristics of phase consistency, effectively overcoming nonlinear radiation differences caused by smoke obstruction and drastic lighting changes. Finally, POS geometric constraints are used to eliminate mismatches, ensuring high reliability of the registration results.
Claims
1. An image registration method adaptable to thermal coherence effects and differences in nonlinear radiation, characterized in that, Includes the following steps: Step S1: Input the visible light image and infrared image to be registered, perform multi-scale Log-Gabor analysis on the visible light image and infrared image respectively, construct Log-Gabor frequency response map and phase consistency map, and construct a reliability weight matrix from the phase consistency map; Step S2: Based on the Log-Gabor frequency response map obtained in Step S1, a multi-layered multi-window Gaussian pyramid scale space is constructed using a side-window Gaussian filter (SWGF). Each layer of the image represents a pyramid image at the corresponding scale. Feature points are detected on each layer of the image in the scale space, and the optimal side-window index corresponding to each feature point is obtained. The optimal side-window index is the pointing index of the feature point relative to the direction of the internal flat region of the object edge. Step S3: Perform asymmetric physical correction on the feature points; wherein, the object of the asymmetric physical correction is the local structural shift caused by the difference in thermal radiation imaging between the infrared image and the visible light image. A spatial offset vector pointing to the texture-flat region in the neighborhood of the feature point is calculated based on the optimal side window index, and the sampling center of the feature point is moved from its original position using the spatial offset vector. Translate to repositioning center ; Step S4: Construct a phase-weighted asymmetric descriptor at the relocation center, establish a local sampling grid, and combine it with the reliability weight matrix generated in step S1. The amplitude of the Log-Gabor frequency response plot Weighted encoding is performed to generate feature descriptors for infrared and visible light images; Step S5: Calculate the similarity between the feature descriptors of the infrared image and the visible light image using a normalized cross-correlation metric. Obtain the initial set of matching point pairs between the feature descriptors of the two; Step S6: Construct position-direction-scale geometric feature vectors, use density clustering algorithm to remove mismatched points in the initial matching point set due to smoke flow or local parallax, calculate the affine transformation matrix from infrared image to visible light image, and use the affine transformation matrix to perform spatial transformation on the image to be registered, thus completing the registration of infrared and visible light images.
2. The method according to claim 1, characterized in that... The construction of the Log-Gabor frequency response map and phase consistency map in step S1 specifically includes: using a multi-scale, multi-directional Log-Gabor filter to convolve the image to obtain each pixel. Frequency response components at scale s and direction o And calculate its corresponding amplitude. ; Based on the above amplitude The phase consistency at each pixel is calculated using the following formula. : At the same time, the total amplitude of the Gabor frequency response corresponding to this pixel is calculated using the following formula. : in, The frequency domain response amplitude function is the log-Gabor frequency domain response function. It is the phase deviation function, where T is the noise threshold. It is the frequency spread weighting factor. To prevent small constants where the numerator is zero; the aforementioned That is, as described in step S4 The value is taken at pixel x.
3. The method according to claim 1, characterized in that, In step S2, the construction of a multi-layered Gaussian pyramid scale space with side windows specifically includes: defining a side window model with eight directions, including up, down, left, right, and four diagonal directions; for each layer of the image in the pyramid, filtering is performed using the side window models with the eight directions, and the pixel gray-level variance within each side window is calculated; the side window with the smallest variance is selected as the optimal side window, and the filtering result of the optimal side window is used as the output value of the current pixel, and the direction index of the optimal side window is recorded as the optimal side window index; the next set of images is constructed through downsampling, and the above filtering steps are repeated to form a multi-resolution pyramid structure that preserves edge features.
4. The method according to claim 1, characterized in that... In step S3, the specific formula for calculating the spatial offset vector pointing to the texture flat region within the neighborhood of the feature point is as follows: in Indicates the first The spatial offset vector corresponding to the best side window index, where α is the preset offset coefficient and σ is the scale value of the current feature point. To point to the first The unit direction vector of the geometric center of each side window; the repositioning The calculation formula is: ,in These are the original coordinates of the feature point.
5. The method according to claim 1, characterized in that, In step S4, generating feature descriptors for the infrared and visible light images specifically includes: using the repositioning center... Establish a logarithmic polar coordinate grid around the origin; calculate the characteristic statistics of each sub-region within the grid. The formula is as follows: in, For the set of pixels within the sub-region, For Gaussian weighted parameters, This represents the phase consistency weight corresponding to that point. The value is the Log-Gabor frequency response amplitude corresponding to that point; the feature statistics of all sub-regions are combined into a vector, sorted along the main feature direction, and then normalized by L2 to obtain the feature descriptor.
6. The method according to claim 1, characterized in that, In step S5, the formula for calculating the normalized cross-correlation metric is as follows: in, and These are the feature descriptor vectors of the reference image and the image to be registered, respectively. The L2 norm is represented; the point with the largest NCC value is selected as the matching point, and the initial matching point pair is screened by combining the two-way ratio test.
7. The method according to claim 1, characterized in that, In step S6, the elimination using density clustering specifically includes: for each pair of initial matching points, calculating their positional deviation. Logarithmic scale ratio Difference from the main direction Construct a four-dimensional feature vector In four-dimensional space, cluster centers are found, and matching points within the neighborhood of each cluster center are retained as interior points, while other outliers are removed. The specific formula is as follows: in, These represent the positional deviations of the matching point pair in the x and y directions, respectively. To match the scale ratio of point pairs, Its logarithm; The principal direction difference for matching point pairs.
8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.
