A method and system for matching heterogeneous image region features, and computer equipment.
By using a rotation- and scale-independent region feature descriptor method, the problem of suboptimal feature extraction and matching in heterogeneous image matching is solved, achieving efficient matching of corresponding points and adapting to image rotation and scale changes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2023-12-29
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies do not yield ideal results in feature extraction and matching in heterogeneous image matching, especially since the differences between thermal infrared and visible light images lead to poor performance of traditional methods.
We employ a rotation- and scale-independent region feature descriptor method to obtain feature regions through edge detection, calculate gray-level centroid and principal direction, generate feature vectors, and perform Euclidean distance matching to improve matching accuracy.
It effectively narrows the search range of corresponding points, improves the efficiency and accuracy of heterogeneous image matching, and solves the problems of image rotation and scale changes.
Smart Images
Figure CN117830672B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and in particular to a technical solution for matching corresponding locations between heterogeneous images using regional feature matching. Background Technology
[0002] Image matching technology is one of the key technologies in computer vision, remote sensing image processing, and related fields. Image matching includes high-precision matching of corresponding points and coarse matching of corresponding locations. Matching strategies include template matching based on correlation coefficients and feature matching based on descriptors. The accuracy and effectiveness of image matching are directly related to the success of 3D model construction, robot visual navigation and localization, and processing of multi-source sensor joint observations.
[0003] Taking thermal infrared (TIR) and visible light (VIS) images as examples, thermal infrared images can help the human eye distinguish between targets and backgrounds, while visible light images can provide high-resolution material and edge information of targets. Combining the two can greatly improve the efficiency and accuracy of image interpretation.
[0004] However, the imaging mechanisms and recorded information of heterogeneous images differ, posing a significant obstacle to their matching. Taking thermal infrared and visible light images as examples, thermal infrared images capture electromagnetic waves in the thermal infrared band emitted by the target, while visible light images record electromagnetic waves in the visible light band reflected from the target. This causes two problems: First, during feature extraction, the edges of thermal infrared images become blurred due to thermal diffusion, and the image gradient magnitude exhibits a smoothing effect. Second, during feature matching, there is almost no correlation between the radiation values recorded by thermal infrared and visible light images, leading to significant differences in the gradient magnitude and direction of corresponding points on the two images. These two problems are almost fatal to traditional feature extraction methods and various operators. For example, the classic SIFT operator is highly dependent on the gradient magnitude and direction of the image, resulting in poor matching results when applied to heterogeneous image matching.
[0005] In recent years, some studies have utilized neural networks and deep learning methods to achieve heterogeneous image matching. For example, the CMM-Net heterogeneous image matching method based on a pseudo-Siamese neural network extracts unique features from two heterogeneous images using two branches of the pseudo-Siamese neural network, and then uses a fully connected layer to allow the two branches to share parameters, thereby extracting common invariant features between the two images. Other methods improve the stability of feature extraction and matching results by refining the gradient calculation method of the SIFT operator and the feature descriptor generation algorithm. However, the practical results of these methods are still not ideal.
[0006] To improve matching efficiency and reduce the time required for heterogeneous image matching, this invention proposes a rotation- and scale-independent heterogeneous image region matching method, which can narrow the processing range for further feature point extraction and homonymous point matching. Summary of the Invention
[0007] This invention addresses the problem of unsatisfactory feature extraction and matching results for heterogeneous images in existing technologies by proposing a rotation- and scale-independent region feature descriptor to achieve a new matching technique.
[0008] To achieve the above objectives, the technical solution proposed in this invention provides a heterogeneous image region feature matching method, comprising the following steps:
[0009] Step 1: Acquire two heterogeneous images, extract the image edges, and obtain the feature regions as the targets to be matched;
[0010] Step 2: Calculate the grayscale centroid for each region to be matched;
[0011] Step 3: Determine the principal direction for each region to be matched. This involves obtaining a minimum circle containing the region, centered on the gray-level centroid obtained in Step 2 and with the maximum distance from the centroid to the region edge as the radius. Calculate the gradient magnitude and direction for each pixel within the circle. Weight the gradient magnitude of each pixel with a Gaussian kernel and accumulate the values within the interval containing its direction to form a histogram. Use the peak value of the histogram and the statistical values in the neighborhood to perform parabolic fitting to improve the accuracy of the principal direction estimation.
[0012] Step 4: Taking the main direction as the starting direction and clockwise as the positive direction, draw m rays clockwise from the origin of each region, covering a range from 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m. Calculate the Euclidean distance between each of these m points and the origin of the region; m is a preset value.
[0013] Step 5: Arrange the m Euclidean distances obtained in Step 4 in the order of generation, remove some of the maximum and minimum values respectively, and normalize the remaining ordered array to form a feature vector.
[0014] Step 6: Calculate the Euclidean distance between the feature vectors of each region to be matched. According to the preset threshold, when the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
[0015] Furthermore, in step 1, edge detection is used to obtain image edges, thereby determining the number and location of regions to be matched in the image.
[0016] Furthermore, the grayscale centroid obtained in step 2 represents the center of the region, in order to confirm the center of the region before determining the main direction of the region.
[0017] Furthermore, in step 3, a minimum circle containing the entire region is drawn with the region center obtained in step 2 as the center. The gradient of each pixel within the circle is counted, and a histogram is formed by weighting it using a Gaussian kernel function. The accurate principal direction is obtained by parabolic fitting using the peak value of the histogram and its n-neighboring neighbor values. The feature vector is given rotation invariance, and the value of n is 1 or 2.
[0018] Furthermore, in step 4, m = 36 and dk = 10 are set.
[0019] Furthermore, in step 5, the obtained m Euclidean distances are arranged in the order of generation, the largest and smallest values in the first 5% are removed respectively, and the remaining ordered array is normalized to form a feature vector, thus giving the feature vector scale invariance.
[0020] Furthermore, the location of the region center determines the number of pixels involved in gradient calculation, and together with the Gaussian kernel function, determines the weighting of the gradient magnitude of each pixel when accumulating the histogram.
[0021] On the other hand, the present invention also provides a heterogeneous image region feature matching system, comprising the following modules:
[0022] The region edge information extraction module is used to acquire two heterogeneous images, extract the image edges, and obtain the feature regions as the target to be matched;
[0023] The region initialization module is used to calculate the grayscale centroid of each region to be matched;
[0024] The main direction extraction module is used to determine the main direction of each region to be matched. This includes obtaining a minimum circle containing the region, centered on the gray-level centroid obtained in step 2 and with the maximum distance from the gray-level centroid to the edge of the region as the radius; calculating the gradient magnitude and direction of each pixel within the circle; weighting the gradient magnitude of each pixel with a Gaussian kernel and accumulating it within the interval of its direction to form a histogram; and using the peak value of the histogram and the statistical values in the neighborhood to perform parabolic fitting to improve the accuracy of the main direction estimation.
[0025] The region feature descriptor generation module is used to generate m rays from the region origin, starting from the main direction and clockwise as the positive direction, covering a range of 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m. The module calculates the Euclidean distance between each of the m points and the region origin, where m is a preset value. The m Euclidean distances are then arranged in the generation order, and several maximum and minimum values are removed. The remaining ordered array is normalized to form a feature vector.
[0026] The region matching module is used to calculate the Euclidean distance between the feature vectors of each region to be matched. According to a preset threshold, when the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
[0027] On the other hand, the present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a heterogeneous image region feature matching method as described above.
[0028] On the other hand, the present invention also provides a readable storage medium, including a readable storage medium on which a computer program is stored, wherein when the computer program is executed, it implements a heterogeneous image region feature matching method as described above.
[0029] This invention provides region matching results for feature point extraction and matching of heterogeneous images, offering a clear search range for further refined matching and reducing the difficulty of finding corresponding points while improving search efficiency. Because the principal direction of the region is calculated and the feature vector is normalized before matching, the region feature descriptor exhibits strong stability under image rotation and scale changes. It effectively solves the region matching problem under image rotation and scale changes, even when heterogeneous images face various complex challenges, thus significantly improving region matching results.
[0030] The present invention is simple and convenient to implement, highly practical, and solves the problems of low practicality and inconvenience in actual application of related technologies. It can improve user experience and has significant market value. Attached Figure Description
[0031] Figure 1 This is a general flowchart of an embodiment of the present invention;
[0032] Figure 2 This is a schematic diagram of the main direction calculation in an embodiment of the present invention.
[0033] Figure 3 This is a schematic diagram illustrating the generation of region feature descriptors in an embodiment of the present invention. Detailed Implementation
[0034] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0035] This invention relates to a method for matching corresponding locations between heterogeneous images using region feature matching technology. The core of this method is the construction of a rotation- and scale-independent region feature descriptor. The method first enhances the two original images to be matched, segmenting regions containing salient features. Then, it calculates the principal orientation of each region and generates normalized descriptors for the region features. Finally, it uses the minimum block distance between the region descriptors of the two images as the similarity metric to achieve matching of corresponding features. This invention effectively solves the region matching problem caused by image rotation and scale changes in heterogeneous images, significantly improving the region matching results.
[0036] See Figure 1 The present invention provides a heterogeneous image region feature matching method, comprising the following steps:
[0037] Step 1: Acquire two heterogeneous images and use edge extraction (detection) technology in digital image processing to obtain the image edges, thereby extracting feature regions from the images as the target to be matched.
[0038] In this embodiment, a fast Fourier transform is used to convert the image to the frequency domain, and after discarding the low-frequency part, an inverse transform is performed to extract the edge of the image.
[0039] Step 2: Calculate the grayscale centroid for each region to be matched.
[0040] Before determining the main direction of a region, it is necessary to identify the region center, which is represented by its gray-scale centroid.
[0041] Specifically, the grayscale centroid of each region can be calculated using the following formula:
[0042]
[0043] in,
[0044] R represents the set of points in the target region.
[0045] x and y represent the x and y coordinates of the current point, respectively.
[0046] h(x,y) represents the gray value of the point with coordinates (x,y).
[0047] These represent the horizontal and vertical coordinates of the grayscale centroid of the target area, respectively.
[0048] Step 3, see Figure 2 The principal direction is determined for each region to be matched. This involves taking the gray-level centroid obtained in step 2 as the center and the maximum distance from the gray-level centroid to the edge of the region as the radius to obtain the smallest circle containing the region. The gradient magnitude and direction of each pixel within the circle are then calculated. The gradient magnitude of each pixel is weighted by a Gaussian kernel and accumulated within the interval of its direction to form a histogram. The peak value of the histogram and the statistical values in the neighborhood are used to perform parabolic fitting to improve the accuracy of the principal direction estimation.
[0049] The resulting grayscale centroid represents the center of the region, which is used to confirm the center of the region before determining the main direction of the region.
[0050] The present invention further proposes that when calculating the principal direction of the region, the gradient of each pixel is statistically analyzed to form a histogram, and the peak value and its n-neighboring neighbor values are used to perform parabolic fitting to obtain the accurate principal direction, thus giving the feature vector rotation invariance. The value of n can generally be 1 or 2.
[0051] The location of the region center determines the number of pixels involved in gradient calculation, and together with the Gaussian kernel function, determines the weighting of the gradient magnitude of each pixel when accumulating the histogram.
[0052] In this embodiment, the smallest circle containing the region is obtained by using the gray-level centroid obtained in step 2 as the center and the maximum distance from the gray-level centroid to the edge of the region as the radius. The gradient magnitude and direction are then calculated for each pixel within the circle.
[0053] Specifically, the formulas for calculating the gradient magnitude and gradient direction in this embodiment are as follows:
[0054]
[0055] θ(x,y)=tan -1 ((h(x,y+1)-h(x,y-1)) / (h(x+1,y)-h(x-1,y)))
[0056] in,
[0057] g(x,y) represents the gradient magnitude at the current point.
[0058] θ(x,y) represents the gradient direction at the current point.
[0059] h(x-1,y) represents the gray value of the point (x-1,y), and the meanings of the other gray values h(,) are similar.
[0060] A histogram is formed using the calculated gradient magnitude and direction of each pixel. In this embodiment, the group spacing of the histogram is 36, covering a range from 0 to 360 degrees. The gradient magnitude of each pixel is accumulated within its corresponding interval; the accumulated value is a weighted value obtained by weighting the gradient magnitude with a Gaussian kernel. In this embodiment, σ is set to 2.28. The two-dimensional Gaussian kernel function is as follows:
[0061]
[0062] Parabolic fitting is performed using the peak value of the histogram and its left and right adjacent values to improve the accuracy of the principal direction estimation.
[0063] In practice, the peak value of the histogram and the statistical values in its n-neighborhood (e.g., n=1, i.e., taking the left and right adjacent values) can be used to fit a parabola, thereby improving the accuracy of the main direction estimation.
[0064] Step 4, see Figure 3 For each region, draw m rays clockwise from the region origin, intersecting the region boundary at m points. Calculate the Euclidean distance between each of these m points and the region origin.
[0065] Specifically, taking the main direction as the starting direction and clockwise as the positive direction, each region draws m rays clockwise from the region origin, covering a range from 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m (e.g., m = 36, dk = 10). Calculate the Euclidean distance between each of these m points and the region origin.
[0066] In this embodiment, m = 36 is preferably chosen, meaning that with the main direction as the starting direction and clockwise as the positive direction, 36 rays are drawn clockwise from the origin of each region, covering a range from 0 to 360 degrees, intersecting the region boundary at 36 points, with an angular interval of dk = 10 between each ray. The Euclidean distances between these 36 points and the centroid of the region are then calculated.
[0067] Step 5, see Figure 3 Arrange the m Euclidean distances obtained in step 4 in the order of generation, remove the maximum and minimum values of the first few proportions respectively, and normalize the remaining ordered array to form a feature vector.
[0068] In this embodiment, the top 5% of the maximum and the top 5% of the minimum values are removed from the 36 Euclidean distances obtained in step 4, and then normalized to form the feature vector. That is, when m=36, the two largest and two smallest values are removed.
[0069] Step 6: Calculate the Euclidean distance between the feature vectors of each region to be matched. A suitable threshold can be set in advance. When the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
[0070] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.
[0071] In some possible embodiments, a heterogeneous image region feature matching system is provided, comprising the following modules:
[0072] The region edge information extraction module is used to acquire two heterogeneous images, extract the image edges, and obtain the feature regions as the target to be matched;
[0073] The region initialization module is used to calculate the grayscale centroid of each region to be matched;
[0074] The main direction extraction module is used to determine the main direction of each region to be matched. This includes obtaining a minimum circle containing the region, centered on the gray-level centroid obtained in step 2 and with the maximum distance from the gray-level centroid to the edge of the region as the radius; calculating the gradient magnitude and direction of each pixel within the circle; weighting the gradient magnitude of each pixel with a Gaussian kernel and accumulating it within the interval of its direction to form a histogram; and using the peak value of the histogram and the statistical values in the neighborhood to perform parabolic fitting to improve the accuracy of the main direction estimation.
[0075] The region feature descriptor generation module is used to generate m rays from the region origin, starting from the main direction and clockwise as the positive direction, covering a range of 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m. The module calculates the Euclidean distance between each of the m points and the region origin, where m is a preset value. The m Euclidean distances are then arranged in the generation order, and several maximum and minimum values are removed. The remaining ordered array is normalized to form a feature vector.
[0076] The region matching module is used to calculate the Euclidean distance between the feature vectors of each region to be matched. According to a preset threshold, when the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
[0077] The specific module implementations and method implementations correspond to each step, which will not be elaborated upon in this invention.
[0078] In some possible embodiments, a heterogeneous image region feature matching system is provided, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a heterogeneous image region feature matching method as described above.
[0079] In some possible embodiments, a heterogeneous image region feature matching system is provided, including a readable storage medium on which a computer program is stored. When the computer program is executed, it implements a heterogeneous image region feature matching method as described above.
[0080] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
Claims
1. A method for matching features in heterogeneous image regions, the method comprising: Includes the following steps: Step 1: Acquire two heterogeneous images, extract the image edges, and obtain the feature regions as the target to be matched; Step 2: Calculate the grayscale centroid for each region to be matched; Step 3: Calculate the main direction for each region to be matched. This includes obtaining the smallest circle containing the region, with the gray-level centroid obtained in Step 2 as the center and the maximum distance from the gray-level centroid to the edge of the region as the radius. Calculate the gradient magnitude and direction for each pixel within the circle. The gradient magnitude of each pixel is weighted by the Gaussian kernel and then accumulated within the interval of its direction to form a histogram. Parabolic fitting is performed using histogram peak values and neighborhood statistics to improve the accuracy of principal direction estimation; Step 4: Taking the main direction as the starting direction and clockwise as the positive direction, draw m rays clockwise from the origin of each region, covering a range from 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m. Calculate the Euclidean distance between each of these m points and the origin of the region; m is a preset value. Step 5: Arrange the m Euclidean distances obtained in Step 4 in the order of generation, remove some of the maximum and minimum values respectively, and normalize the remaining ordered array to form a feature vector. Step 6: Calculate the Euclidean distance between the feature vectors of each region to be matched. According to the preset threshold, when the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
2. The method of claim 1, wherein: In step 1, edge detection is used to obtain image edges, and the number and location of regions to be matched in the image are determined.
3. The method of claim 1, wherein: The grayscale centroid obtained in step 2 represents the center of the region, which is used to confirm the center of the region before determining the main direction of the region.
4. The method of claim 3, wherein: In step 3, the smallest circle containing the entire region is drawn with the center of the region obtained in step 2 as the center. The gradient of each pixel within the circle is counted, and a histogram is formed by weighting it with a Gaussian kernel function. The accurate principal direction is obtained by parabolic fitting using the peak value of the histogram and its n-neighboring neighbor values. The feature vector is given rotation invariance, and the value of n is 1 or 2.
5. The method of claim 1, wherein: In step 4, set m = 36 and dk = 10.
6. The method of claim 1, wherein: In step 5, the m Euclidean distances obtained are arranged in the order of generation, the largest and smallest values in the first 5% are removed, the remaining ordered array is normalized to form a feature vector, and the feature vector is given scale invariance.
7. The method of claim 1 or 3 or 4, wherein: The location of the region center determines the number of pixels involved in the gradient calculation, and together with the Gaussian kernel function, determines the weighting of the gradient magnitude of each pixel when accumulating the histogram.
8. A heterogeneous image region feature matching system, characterized in that: Includes the following modules, The region edge information extraction module is used to acquire two heterogeneous images, extract the image edges, and obtain the feature regions as the target to be matched; The region initialization module is used to calculate the grayscale centroid of each region to be matched; The main direction extraction module is used to find the main direction of each region to be matched. This includes obtaining a minimum circle containing the region with the gray-level centroid obtained in step 2 as the center and the maximum distance from the gray-level centroid to the edge of the region as the radius. The gradient magnitude and direction of each pixel within the circle are calculated. The gradient magnitude of each pixel is weighted by the Gaussian kernel and then accumulated in the interval where its direction is located to form a histogram. Parabolic fitting is performed using histogram peak values and neighborhood statistics to improve the accuracy of principal direction estimation; The region feature descriptor generation module is used to generate m rays from the region origin, starting from the main direction and clockwise as the positive direction, covering a range of 0 to 360 degrees, intersecting the region boundary at m points. The angular interval between each ray is dk = 360 / m. The module calculates the Euclidean distance between each of the m points and the region origin, where m is a preset value. The m Euclidean distances are then arranged in the generation order, and several maximum and minimum values are removed. The remaining ordered array is normalized to form a feature vector. The region matching module is used to calculate the Euclidean distance between the feature vectors of each region to be matched. According to a preset threshold, when the Euclidean distance between the feature vectors of two regions to be matched is less than the threshold, the match is considered successful.
9. A computer device, comprising: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute the heterogeneous image region feature matching method as described in any one of claims 1-7.
10. A readable storage medium, characterized by: The method includes a readable storage medium on which a computer program is stored, and when the computer program is executed, it implements a heterogeneous image region feature matching method as described in any one of claims 1-7.