Image splicing quality improvement algorithm based on secondary matching
By establishing a simple Gaussian pyramid and adjusting feature point extraction thresholds, quadratic matching between images is solved, and the problem of insufficient feature point extraction capability in low-texture image processing is achieved, which significantly improves the image stitching quality.
Patent Information
- Application Number
- CN202411942869.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-05-13
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When processing low-texture images, the SIFT algorithm has weak feature point extraction capabilities, and the number of feature points in high-level images is reduced, making it difficult to achieve the ideal stitching effect.
By establishing a simple Gaussian pyramid, the neighborhood threshold for extracting feature points of high-group images in the DOG Gaussian differential pyramid is changed, and the quadratic matching of feature points is performed, the scale relationship between images and the main and secondary adjustment groups are determined, and the number of feature points is optimized to improve matching accuracy.
The SSIM and PSNR indicators of image stitching have been significantly improved, increasing by 19.39% and 15.65% respectively, and can effectively deal with image stitching problems in various complex situations.
Smart Images
Figure BDA0005212949700000021 
Figure BDA0005212949700000022 
Figure BDA0005212949700000041
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image stitching, and in particular relates to an image stitching quality improvement algorithm based on secondary matching. Background Art
[0002] Image stitching technology is a technology that calculates the transformation relationship between multiple images and fuses them to generate a larger field of view image. It is an important part of image processing. Image stitching technology is widely used in the fields of size measurement, remote sensing, motion detection, resolution enhancement and medical imaging.
[0003] Among various image stitching algorithms, the SIFT algorithm has attracted much attention from researchers due to its excellent performance and strong robustness. However, the SIFT algorithm has a weak ability to extract feature points when processing low-texture images. In addition, the Gaussian pyramid of the SIFT algorithm is generated by downsampling. As the number of groups increases, the number of feature points decreases. Especially when the number of groups is high, it is often difficult to achieve an ideal stitching effect. Summary of the invention
[0004] In order to solve the above problems, this paper provides an image stitching quality improvement algorithm based on quadratic matching. By establishing a simple Gaussian pyramid, changing the neighborhood threshold for extracting feature points of the high group of images in the DOG Gaussian difference pyramid, and performing secondary matching of feature points, the scale relationship between images and the primary and secondary adjustment groups are determined, and finally the image stitching quality is improved.
[0005] To achieve the above purpose, the technical solution adopted by the present invention is:
[0006] An image stitching quality improvement algorithm based on secondary matching includes the following steps:
[0007] (1) Construction of a simple Gaussian pyramid: The first four layers of each group in the DOG Gaussian difference pyramid are selected to construct the simple pyramid. After obtaining the simple pyramid, feature points are extracted from large to small according to the contrast, and the same number of feature points are extracted from each group;
[0008] (2) Adjust the feature point extraction threshold of high-level images: Reduce the neighborhood threshold of the high-level image detection feature points in the DOG Gaussian difference pyramid to increase the number of extractable feature points, perform feature matching on the extracted feature points, and calculate the matching rate of each group of feature points. The calculation formula is as follows:
[0009]
[0010] Where N c,j and N iThe correct matching number and total number of the i-th group of the simple pyramid are respectively obtained. The maximum value of the calculated matching rate is obtained to know the scale relationship between the images. The group corresponding to the maximum value is set as the main adjustment group, and the group with k times the maximum value is set as the secondary adjustment group.
[0011] (3) Image fusion: The distribution of the feature points of the DOG Gaussian difference pyramid is derived, the number of feature points is adjusted, and the image fusion is completed. The formula for deriving the number of feature points in each group and layer is as follows:
[0012]
[0013] Where N sum is the sum of the number of feature points in each group and each layer, m and n are the number of groups and the number of layers in each group of the DOG Gaussian difference pyramid, respectively, N o，l is the number of feature points of the oth group and the lth layer;
[0014] N o,l =N 1,2 α o-1 β l-2 l≥2
[0015] Where N 1,2 is the number of feature points in the second layer of the first group of the DOG Gaussian difference pyramid, α is the extraction ratio of feature points in the same layer of two adjacent groups in the DOG Gaussian difference pyramid, and β is the extraction ratio of feature points in two adjacent layers;
[0016] (4) Secondary matching: After determining the distribution of feature points in the DOG Gaussian difference pyramid and the primary and secondary adjustment groups, the number of feature points is optimized so that the number of feature points in the high-level pyramid is consistent with that in the first group. The number of adjusted feature points increases, thereby improving the matching accuracy.
[0017] Furthermore, the DOG Gaussian difference pyramid is obtained by downsampling with a step size of 2, and the extraction ratio of feature points of two adjacent groups of the same layer in the DOG Gaussian difference pyramid is set to α=0.30, that is, the number of feature points extracted between two adjacent groups decreases by 30%; during matching, the possibility of matching the extracted feature points of each layer in each group of the DOG Gaussian difference pyramid is the same, so the extraction ratio of feature points of two adjacent layers is set to β=1.
[0018] The value range of the multiple k is: 0.75 <k<1。
[0019] Compared with the prior art, the beneficial effects of the present invention are:
[0020] 1. Compared with the traditional SIFT algorithm and other improved algorithms, the present invention has significantly improved the SSIM (structural similarity index) and PSNR (peak signal-to-noise ratio) indicators by 19.39% and 15.65% respectively;
[0021] 2. This method is not only applicable to conventional images, but can also effectively deal with image stitching problems under various complex conditions such as blur transformation, perspective transformation, brightness transformation, rotation transformation and compression transformation. DETAILED DESCRIPTION
[0022] The technical solutions in the embodiments of the present invention are described in detail below.
[0023] An image stitching quality improvement algorithm based on secondary matching includes the following steps:
[0024] (1) Construction of a simple Gaussian pyramid: The first four layers of each group in the DOG Gaussian difference pyramid are selected to construct the simple pyramid. After obtaining the simple pyramid, feature points are extracted from large to small according to the contrast, and the same number of feature points are extracted from each group;
[0025] (2) Adjust the feature point extraction threshold of high-level images: Reduce the neighborhood threshold of the high-level image detection feature points in the DOG Gaussian difference pyramid to increase the number of extractable feature points, perform feature matching on the extracted feature points, and calculate the matching rate of each group of feature points. The calculation formula is as follows:
[0026]
[0027] Where N c,j and N i The correct matching number and total number of the i-th group of the simple pyramid are respectively obtained. The maximum value of the calculated matching rate is obtained to know the scale relationship between the images. The group corresponding to the maximum value is set as the main adjustment group, and the group with k times the maximum value is set as the secondary adjustment group.
[0028] (3) Image fusion: The distribution of the feature points of the DOG Gaussian difference pyramid is derived, the number of feature points is adjusted, and the image fusion is completed. The formula for deriving the number of feature points in each group and layer is as follows:
[0029]
[0030] Where N sum is the sum of the number of feature points in each group and each layer, m and n are the number of groups and the number of layers in each group of the DOG Gaussian difference pyramid, respectively, N o，l is the number of feature points of the oth group and the lth layer;
[0031] N o,l =N 1,2 α o-1β l-2 l≥2
[0032] Where N 1,2 is the number of feature points in the second layer of the first group of the DOG Gaussian difference pyramid, α is the extraction ratio of feature points in the same layer of two adjacent groups in the DOG Gaussian difference pyramid, and β is the extraction ratio of feature points in two adjacent layers;
[0033] (4) Secondary matching: After determining the distribution of feature points in the DOG Gaussian difference pyramid and the primary and secondary adjustment groups, the number of feature points is optimized so that the number of feature points in the high-level pyramid is consistent with that in the first group. The number of adjusted feature points increases, thereby improving the matching accuracy.
[0034] Since the DOG Gaussian difference pyramid is obtained by downsampling with a step size of 2, the feature point extraction ratio of two adjacent groups of the same layer in the DOG Gaussian difference pyramid is set to α=0.30, that is, the number of feature points extracted between two adjacent groups decreases by 30%; by reducing the feature point extraction ratio, the situation of insufficient feature points in the high-level image is avoided, ensuring that the high-level image can still provide enough feature points for accurate matching; during matching, the possibility of matching the feature points extracted from each layer in each group of the DOG Gaussian difference pyramid is the same, so the extraction ratio of feature points of two adjacent layers is set to β=1.
[0035] The value range of the multiple k is: 0.75 <k<1。
[0036] In one embodiment, a simple Gaussian pyramid is established by selecting the first four layers of each group to avoid extra time consumption caused by repeated Gaussian pyramid establishment. After the simple pyramid is obtained, feature points are extracted from large to small according to the contrast, and the same number of 180 feature points are extracted from each group.
[0037] Perform feature point detection and matching, calculate the matching rate of each group of feature points, and determine the scale relationship between images and the primary and secondary adjustment groups. Since the Gaussian pyramid is obtained by downsampling and Gaussian convolution kernel, the image size of the upper group of the Gaussian pyramid is small and relatively blurred, and the number of extractable feature points gradually decreases. Therefore, in order to increase the number of feature points, the present invention reduces the neighborhood threshold of the high group image detection feature points in the DOG Gaussian difference pyramid. Specifically: the neighborhood thresholds of the 3rd and 4th groups are changed to 23 and 15 respectively, and the neighborhood thresholds of 5 groups and above are changed to 10 to increase the number of extractable feature points. Feature matching is performed on the extracted feature points, and the matching rate of each group of feature points is calculated. The calculation formula is as follows:
[0038]
[0039] Where N c,j and N iThey are respectively the number of correct matches and the total number of the i-th group of the simple pyramid. The calculated matching rate is maximized to determine the scale relationship between the images, and the main adjustment group for feature point detection is selected accordingly. Since the Gaussian pyramid is generated by downsampling with a step size of 2, the scale relationship between adjacent groups is 2:1. When the scale relationship of the image is not a multiple of 2, it usually leads to a higher matching rate between two adjacent groups. Based on this, the present invention sets the group corresponding to the maximum matching rate as the main adjustment group, and sets the group with a matching rate greater than 0.75 times the maximum value as the secondary adjustment group. The main adjustment group and the secondary adjustment group will adopt the same feature point quantity optimization strategy to ensure the consistency of matching accuracy.
[0040] The distribution of the feature points of the DOG Gaussian difference pyramid is derived, the number of feature points is adjusted, and the image fusion is completed. For the derivation of the number of feature point distribution, after obtaining the main and secondary adjustment groups between the images, the number of feature points needs to be adjusted, but before the number of feature points is adjusted, the distribution of feature points in the DOG Gaussian difference pyramid needs to be determined. The present invention derives the formula for the distribution of feature points in the DOG Gaussian difference pyramid, and the method for determining the number of feature points in each group and layer is derived as follows:
[0041] First, let the total number of feature points be N sum , which is equal to the sum of the number of feature points in each group and layer:
[0042]
[0043] In the formula, m and n are the number of DOG groups and the number of layers in each group, respectively. o,l is the number of feature points in the oth group and the lth layer.
[0044] In order to ensure that the number of feature points in the high-level image is not less than that in the lower level, the feature point extraction ratio of two adjacent groups of the same layer in the DOG Gaussian difference pyramid is set to α=0.30 through a downsampling method with a step size of 2, that is, the number of feature points extracted between two adjacent groups decreases by 30%. When matching feature points, the matching probability of feature points in each group and each layer is equal, so the feature point extraction ratio of two adjacent layers is set to β=1. At the same time, the number of feature points in the second layer of the first group of DOG is set to N 1,2 , the number of feature points in any layer of DOG can be obtained as:
[0045] N o,l =N 1,2 α o-1 β l-2 l≥2
[0046] From the above formula, we can see that the number of feature points in different layers of each group and in the same layer of different groups are distributed in geometric proportion. Therefore, the sum of the number of feature points of any layer in each group is:
[0047]
[0048] From the above formula, we can see that N sum,l = {N sum ,2,N sum ,3...N sum ,n-1} presents geometric distribution, and β is 1. Therefore, the total number of feature points in the Gaussian difference pyramid is:
[0049]
[0050] N 1,2 for:
[0051]
[0052] It can be concluded that the number of feature points in any group and any layer is:
[0053]
[0054] Under the premise that the total number of feature points is set to 850 in this embodiment, the number of feature points of any group and any layer can be obtained by using this formula. In order to control the number of feature points, the total number is first determined, and then the required number of feature points is calculated layer by layer according to the formula, and the feature points are extracted according to the contrast. Since each extreme point may have multiple auxiliary directions, an extreme point may be identified as multiple feature points during extraction. Therefore, they need to be detected one by one during the extraction process until the number of feature points reaches the set upper limit. In this process, due to the existence of auxiliary directions, the final number of feature points may be slightly higher than the set value.
[0055] For the optimization of the number of feature points, after determining the distribution of feature points in the DOG Gaussian difference pyramid and the main and secondary adjustment groups, the number of feature points needs to be further adjusted so that the number of feature points in the high-level pyramid is consistent with the first group. The feature point distribution in DOG is determined according to the aforementioned method, and the adjusted feature point distribution (assuming that the ratio of the main adjustment group to the secondary adjustment group is 2) is shown in Table 1. Table 1 shows the optimized feature point distribution, which significantly increases the number of feature points in the second group, thereby improving the success rate of matching. Since the number of feature points in high-level images is small, the present invention increases the number of feature points in high-level images by adjusting the neighborhood threshold. Specifically, the neighborhood thresholds of the third and fourth groups are adjusted to 23 and 15, respectively, and the neighborhood thresholds of the fifth and higher groups are adjusted to 10 to ensure that more feature points are extracted, thereby optimizing the image matching process.
[0056] Table 1 Comparison of feature point adjustment
[0057]
Claims
1. An image stitching quality improvement algorithm based on secondary matching, characterized in that: The following steps are involved: (1) Construction of a simple Gaussian pyramid: The first four layers of each group in the DOG Gaussian difference pyramid are selected to construct the simple pyramid. After obtaining the simple pyramid, feature points are extracted from large to small according to the contrast, and the same number of feature points are extracted from each group; (2) Adjust the feature point extraction threshold of high-level images: Reduce the neighborhood threshold of the high-level image detection feature points in the DOG Gaussian difference pyramid to increase the number of extractable feature points, perform feature matching on the extracted feature points, and calculate the matching rate of each group of feature points. The calculation formula is as follows: Where N c,j and N i The correct matching number and total number of the i-th group of the simple pyramid are respectively obtained. The maximum value of the calculated matching rate is obtained to know the scale relationship between the images. The group corresponding to the maximum value is set as the main adjustment group, and the group with k times the maximum value is set as the secondary adjustment group. (3) Image fusion: The distribution of the feature points of the DOG Gaussian difference pyramid is derived, the number of feature points is adjusted, and the image fusion is completed. The formula for deriving the number of feature points in each group and layer is as follows: Where N sum is the sum of the number of feature points in each group and each layer, m and n are the number of groups and the number of layers in each group of the DOG Gaussian difference pyramid, respectively, N o，l is the number of feature points of the oth group and the lth layer; N o,l =N 1,2 a o-1 b l-2 l≥2 Where N 1,2 is the number of feature points in the first group and the second layer of the DOG Gaussian difference pyramid, α is the extraction ratio of feature points in the same layer of two adjacent groups in the DOG Gaussian difference pyramid, and β is the extraction ratio of feature points in two adjacent layers; (4) Secondary matching: After determining the distribution of feature points in the DOG Gaussian difference pyramid and the primary and secondary adjustment groups, the number of feature points is optimized so that the number of feature points in the high-level pyramid is consistent with that in the first group. The number of adjusted feature points increases, thereby improving the matching accuracy.
2. The image stitching quality improvement algorithm based on secondary matching according to claim 1 is characterized in that: The DOG Gaussian difference pyramid is obtained by downsampling with a step size of 2. The feature point extraction ratio of two adjacent groups of the same layer in the DOG Gaussian difference pyramid is set to α=0.30, that is, the number of feature point extractions between two adjacent groups decreases by 30%; during matching, the possibility of matching the feature points extracted from each layer in each group of the DOG Gaussian difference pyramid is the same, so the extraction ratio of feature points of two adjacent layers is set to β=1.
3. The image stitching quality improvement algorithm based on secondary matching according to claim 1 is characterized in that: The value range of the multiple k is: 0.75 <k<1。