A method, system, device and medium for improving image feature matching accuracy

Abstract

The application provides a method for improving image feature matching accuracy, which introduces image cutting technology to eliminate the influence of regions with excessively large or small gradient values on feature point matching, improve the accuracy of feature point matching in regions with normal gradient values, introduces contrast enhancement technology to improve the number of feature points, and further improve the accuracy of feature point matching, and further introduces feature point deduplication technology to remove repeated feature points and improve the accuracy of feature point matching, thereby solving the technical problems that current image splicing and superimposition technologies are either simple to operate but have poor effects or have good effects but are complex to operate and have large calculation amount, and achieving good splicing and superimposition effects while simplifying the operation.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method, system, device, and medium for improving the accuracy of image feature matching. Background Technology

[0002] Computer vision uses various imaging systems to replace visual organs as the means of input sensing, allowing computers to process and interpret data instead of the brain. Image stitching and overlay techniques in computer vision are widely used in map making, satellite remote sensing, landscape photography, interior design, medical imaging, and weld seam imaging. However, for images with poor quality, such as blurriness, overexposure, or unclear features, the feature matching accuracy is very low, or even completely incorrect, leading to image stitching and overlay failure.

[0003] Therefore, existing image stitching and overlay technologies face the dilemma of either being simple to operate but having poor results, or having good results but being complex to operate and requiring a large amount of computation. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method, system, device, and medium for improving the accuracy of image feature matching. This addresses the technical problems faced by existing image stitching and overlay technologies, which are either simple to operate but have poor results, or have good results but are complex to operate and have a large computational load.

[0005] This invention provides a method for improving the accuracy of image feature matching, comprising:

[0006] S1. Traverse the rows and columns of the image to be overlaid. Based on the gradient value distribution of each row or column, and combined with the upper and lower limits of the gradient value threshold, crop out the rows or columns with extreme gradient values ​​exceeding the proportional threshold.

[0007] S2. Use contrast enhancement to transform the grayscale value of the cropped image to obtain an image with greater contrast.

[0008] S3. Use feature extraction algorithms to extract features from the cropped image with enhanced contrast to obtain feature points;

[0009] S4. Transform the coordinates of the feature points and map them onto the image to be superimposed before cropping;

[0010] S5. Use a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, and remove duplicate feature points.

[0011] S6. Use the feature matching algorithm to match the deduplicated feature points to obtain matching points, and use the matching points to calculate the perspective transformation matrix between the two images to be superimposed.

[0012] S7. Invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, and add it to the registration map to obtain the superimposed image. Invert the superimposed image to obtain the target image.

[0013] Optionally, the step of pruning rows or columns with extreme gradient value proportions exceeding a certain threshold, based on the gradient value distribution of each row or column and in conjunction with the upper and lower limits of the gradient value threshold, includes:

[0014] The number of pixels whose gradient values ​​are less than the lower limit of the gradient value threshold or greater than the upper limit of the gradient value threshold in each row or column is counted out of the total number of pixels in each row or column. If the proportion is greater than the proportion threshold, the pixels are cropped; otherwise, they are retained.

[0015] Optionally, the step of calculating the proportion of pixels whose gradient values ​​in each row or column are less than the lower limit of the gradient value threshold or greater than the upper limit of the gradient value threshold to the total number of pixels in each row or column, wherein if the proportion is greater than a proportion threshold, the pixels are cropped, otherwise they are retained, includes:

[0016] Create two new lists R and C. When iterating through all rows of the image to be overlaid, if a row needs to be cropped, add element -1 to list R; otherwise, add element n to list R. n is equal to the row number minus the number of elements with a value of -1 in list R. When iterating through all columns of the image to be overlaid, if a column needs to be cropped, add element -1 to list C; otherwise, add element n to list C. n is equal to the column number minus the number of elements with a value of -1 in list C.

[0017] Optionally, the feature extraction algorithm includes:

[0018] The feature extraction algorithm used is the SIFT algorithm.

[0019] Optionally, transforming the coordinates of the feature points and mapping them to the image to be overlaid before cropping includes:

[0020] Traverse all feature points and find the index values ​​of the row and column coordinates of each feature point in the lists R and C obtained in step 1. This gives the coordinates of the feature point on the image to be overlaid before cropping.

[0021] Optionally, the feature point deduplication algorithm includes:

[0022] The feature point deduplication algorithm uses the K-Means clustering algorithm.

[0023] The present invention also provides a system for improving the accuracy of image feature matching, comprising:

[0024] The cropping module is used to traverse the rows and columns of the image to be overlaid, and based on the gradient value distribution of each row or column, combined with the upper and lower limits of the gradient value threshold, to crop out rows or columns with extreme gradient values ​​exceeding the proportional threshold.

[0025] The contrast enhancement module is used to perform grayscale transformation on the cropped image to obtain an image with greater contrast.

[0026] The feature extraction module is used to extract features from the cropped, contrast-enhanced image using a feature extraction algorithm to obtain feature points.

[0027] The conversion module is used to convert the coordinates of the feature points and map them onto the image to be superimposed before cropping;

[0028] The feature point deduplication module uses a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, removing duplicate feature points.

[0029] The feature matching module is used to match the deduplicated feature points using a feature matching algorithm to obtain matching points, and to use the matching points to calculate the perspective transformation matrix between the two images to be superimposed.

[0030] The output module is used to invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, add it to the registration map to obtain the superimposed image, and invert the superimposed image to obtain the target image.

[0031] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned method.

[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. This invention introduces image cropping technology to eliminate the influence of regions with excessively large or small gradient values ​​on feature point matching, thereby improving the accuracy of feature point matching in regions with normal gradient values.

[0035] 2. This invention improves the accuracy of feature point matching by introducing contrast enhancement technology to increase the number of feature points detected.

[0036] 3. This invention improves the accuracy of feature point matching by introducing feature point deduplication technology to remove duplicate feature points. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the method flow in this invention;

[0040] Figure 2 The image to be superimposed in this invention;

[0041] Figure 3 This is the gradient map of the image to be superimposed in this invention;

[0042] Figure 4 This is the image after cropping the image to be superimposed in this invention;

[0043] Figure 5 This is a schematic diagram of the images before and after contrast enhancement and their grayscale distribution after cropping the images to be superimposed in this invention;

[0044] Figure 6 This is a map showing the location of feature points in the image to be overlaid in this invention.

[0045] Figure 7 This is an identifier map of the feature point matching results of the images to be overlaid in this invention;

[0046] Figure 8 This is a schematic diagram of the superposition effect of the images to be superimposed in this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Functional units with the same reference numerals in the examples of this invention have the same and similar structures and functions.

[0048] See Figure 1 This invention provides a method for improving the accuracy of image feature matching, comprising:

[0049] S1. Traverse the rows and columns of the image to be overlaid. Based on the gradient value distribution of each row or column, and combined with the upper and lower limits of the gradient value threshold, crop out the rows or columns with extreme gradient values ​​exceeding the proportional threshold.

[0050] S2. Use contrast enhancement to transform the grayscale value of the cropped image to obtain an image with greater contrast.

[0051] S3. Use feature extraction algorithms to extract features from the cropped image with enhanced contrast to obtain feature points;

[0052] S4. Transform the coordinates of the feature points and map them onto the image to be superimposed before cropping;

[0053] S5. Use a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, and remove duplicate feature points.

[0054] S6. Use the feature matching algorithm to match the deduplicated feature points to obtain matching points, and use the matching points to calculate the perspective transformation matrix between the two images to be superimposed.

[0055] S7. Invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, and add it to the registration map to obtain the superimposed image. Invert the superimposed image to obtain the target image.

[0056] In this embodiment, S1, traverse the rows and columns of the image to be overlaid, and based on the gradient value distribution of each row or column, and combined with the upper and lower limits of the gradient value threshold, cut off the rows or columns with extreme gradient values ​​exceeding the proportional threshold.

[0057] The image to be superimposed in this invention is a weld seam radiographic digital image, see [link / reference]. Figure 2 Both images depict the same weld, but the second image shows some translation and rotation compared to the first. First, see... Figure 3 The algorithm calculates the proportion of pixels in each row or column whose gradient values ​​are less than or equal to the lower threshold (lower threshold = minimum gradient value * coefficient, e.g., coefficient equal to 1, 1.1, 1.2, ...) or greater than or equal to the upper threshold (upper threshold = maximum gradient value * coefficient, e.g., coefficient equal to 1, 0.9, 0.8, ...) out of the total number of pixels in that row or column. If this proportion is greater than a threshold, the pixels are cropped; otherwise, they are retained. The preferred threshold is 10%, but it can be adjusted as needed. See [link to previous section] for details. Figure 4This yields the cropped pattern to be overlaid. When performing the aforementioned gradient value judgment, two new lists, R and C, need to be created. When traversing all rows of the gradient image, if a row needs to be cropped, add element -1 to list R; otherwise, add element n to list R, where n equals the row number minus the number of elements in list R with a value of -1. Similarly, when traversing all columns of the image to be overlaid, if a column needs to be cropped, add element -1 to list C; otherwise, add element n to list C, where n equals the column number minus the number of elements in list C with a value of -1.

[0058] Suppose we have an 8x8 image, and we need to crop the first, second, and sixth rows and the first, fifth, and sixth columns. Then the value of list R is [-1, -1, 1, 2, 3, -1, 4, 5], and the value of list C is [-1, 1, 2, 3, -1, -1, 4, 5]. Here, the row and column numbers are counted starting from 1.

[0059] S2. Use contrast enhancement to transform the grayscale values ​​of the cropped image to obtain an image with greater contrast.

[0060] See Figure 5 Contrast enhancement techniques (such as histogram equalization) are used to transform the grayscale values ​​of the cropped images to be superimposed, resulting in images with greater contrast. As shown in the figure, a1 and b1 are the cropped images of the original images, a2 and b2 are the grayscale value distributions of the cropped images, a3 and b3 are the contrast-enhanced images, and a4 and b4 are the grayscale value distributions of the contrast-enhanced images. The comparison reveals that the grayscale distribution is more uniform after contrast enhancement.

[0061] S3. Use a feature extraction algorithm to extract features from the cropped image with enhanced contrast to obtain feature points.

[0062] Feature extraction algorithms (such as SIFT) are used to extract features from cropped images with enhanced contrast, resulting in several feature points. SIFT features are local image features that are invariant to rotation, scaling, and brightness changes, and also maintain a certain degree of stability against viewpoint changes, affine transformations, and noise. They are also rich in information and suitable for fast and accurate processing in massive feature databases.

[0063] S4. Transform the coordinates of the feature points and map them onto the image to be overlaid before cropping.

[0064] Iterate through all feature points and find the index values ​​of the row and column coordinates of each feature point in lists R and C, respectively. This gives the coordinates of the feature point on the image to be overlaid before cropping. Continuing with the example in step S1, assuming the feature points before transformation are (1, 3) and (4, 2), then the feature points after transformation are (3, 4) and (7, 3).

[0065] S5. Use a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, and remove duplicate feature points.

[0066] See Figure 6 We use circles to mark the locations of feature points. a1 and b1 are the feature point detection results of the original image, a2 and b2 are the feature point detection results after cropping the original image, a3 and b3 are the feature point detection results after cropping the original image and enhancing the contrast, and a4 and b4 are the feature point detection results after cropping the original image, enhancing the contrast, and removing duplicate features.

[0067] S6. Use a feature matching algorithm to match the deduplicated feature points to obtain matching points, and use the matching points to calculate the perspective transformation matrix between the two images to be superimposed.

[0068] The aforementioned feature points are matched using a feature matching algorithm (brute force matcher) to obtain matching points. Then, the perspective transformation matrix between the two unprocessed images to be superimposed is calculated using these matching points.

[0069] See Figure 7 We use straight lines to mark the matching points. 7-1 shows the feature matching effect of the original image, 7-2 shows the feature matching effect after cropping the original image, 7-3 shows the feature matching effect after cropping the original image and enhancing the contrast, and 7-4 shows the feature matching effect after cropping the original image, enhancing the contrast, and removing duplicate feature points.

[0070] S7. Invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, and add it to the registration map to obtain the superimposed image. Invert the superimposed image to obtain the target image.

[0071] In this example, when inverting one of the images to be overlaid, since the image selected is an 8-bit image, 255 is subtracted from the image for inversion. When inverting the other image to be overlaid and adding it to the registration image, if the sum is greater than 255, 255 is taken as the grayscale value of that pixel. The overlaid image is then inverted to obtain the final target image. See [link to documentation]. Figure 88-1 is the effect of superimposing the feature matching effect of the original image; 8-2 is the effect of superimposing the feature matching effect after cropping the original image; 8-3 is the effect of superimposing the feature matching effect after cropping the original image and enhancing the contrast; 8-4 is the effect of superimposing the feature matching effect after cropping the original image, enhancing the contrast, removing duplicate feature points, and then cropping the original image.

[0072] This invention not only improves the accuracy of feature point matching in regions with normal gradient values ​​by introducing image cropping technology to eliminate the influence of regions with excessively large or small gradient values ​​on feature point matching, but also introduces contrast enhancement technology to increase the number of detected feature points, thereby improving the accuracy of feature point matching. Furthermore, it introduces feature point deduplication technology to remove duplicate feature points, further improving the accuracy of feature point matching. This invention solves the technical problems faced by existing image stitching and overlay technologies, which are either simple to operate but have poor results, or have good results but are complex to operate and have a large amount of computation. It achieves simplification while achieving good stitching and overlay effects.

[0073] The present invention also provides a system for improving the accuracy of image feature matching, comprising:

[0074] The cropping module is used to traverse the rows and columns of the image to be overlaid, and based on the gradient value distribution of each row or column, combined with the upper and lower limits of the gradient value threshold, to crop out rows or columns with extreme gradient values ​​exceeding the proportional threshold.

[0075] The contrast enhancement module is used to perform grayscale transformation on the cropped image to obtain an image with greater contrast.

[0076] The feature extraction module is used to extract features from the cropped, contrast-enhanced image using a feature extraction algorithm to obtain feature points.

[0077] The conversion module is used to convert the coordinates of the feature points and map them onto the image to be superimposed before cropping;

[0078] The feature point deduplication module uses a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, removing duplicate feature points.

[0079] The feature matching module is used to match the deduplicated feature points using a feature matching algorithm to obtain matching points, and to use the matching points to calculate the perspective transformation matrix between the two images to be superimposed.

[0080] The output module is used to invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, add it to the registration map to obtain the superimposed image, and invert the superimposed image to obtain the target image.

[0081] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned method.

[0082] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for improving the accuracy of image feature matching, characterized in that, include: S1. Traverse the rows and columns of the image to be overlaid. Based on the gradient value distribution of each row or column, and combined with the upper and lower limits of the gradient value threshold, crop out the rows or columns with extreme gradient values ​​exceeding the proportional threshold. S2. Use contrast enhancement to transform the grayscale value of the cropped image to obtain an image with greater contrast. S3. Use feature extraction algorithms to extract features from the cropped image with enhanced contrast to obtain feature points; S4. Transform the coordinates of the feature points and map them onto the image to be superimposed before cropping; S5. Use a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, and remove duplicate feature points. S6. Use the feature matching algorithm to match the deduplicated feature points to obtain matching points, and use the matching points to calculate the perspective transformation matrix between the two images to be superimposed. S7. Invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, and add it to the registration map to obtain the superimposed image. Invert the superimposed image to obtain the target image.

2. The method for improving image feature matching accuracy as described in claim 1, characterized in that, The step of pruning rows or columns with extreme gradient value proportions exceeding a certain threshold, based on the gradient value distribution of each row or column and combined with the upper and lower limits of the gradient value threshold, includes: The number of pixels whose gradient values ​​are less than the lower limit of the gradient value threshold or greater than the upper limit of the gradient value threshold in each row or column is counted out of the total number of pixels in each row or column. If the proportion is greater than the proportion threshold, the pixels are cropped; otherwise, they are retained.

3. The method for improving image feature matching accuracy as described in claim 2, characterized in that, The method involves calculating the proportion of pixels in each row or column whose gradient value is less than the lower limit of the gradient value threshold or greater than the upper limit of the gradient value threshold, relative to the total number of pixels in each row or column. If this proportion is greater than a certain threshold, the pixels are cropped; otherwise, they are retained. This includes: Create two new lists R and C. When iterating through all rows of the image to be overlaid, if a row needs to be cropped, add element -1 to list R; otherwise, add element n to list R. n is equal to the row number minus the number of elements with a value of -1 in list R. When iterating through all columns of the image to be overlaid, if a column needs to be cropped, add element -1 to list C; otherwise, add element n to list C. n is equal to the column number minus the number of elements with a value of -1 in list C.

4. The method for improving image feature matching accuracy as described in claim 3, characterized in that, The feature extraction algorithm includes: The feature extraction algorithm used is the SIFT algorithm.

5. The method for improving image feature matching accuracy as described in claim 3, characterized in that, The step of transforming the coordinates of the feature points and mapping them to the image to be overlaid before cropping includes: Traverse all feature points and find the index values ​​of the row and column coordinates of each feature point in the lists R and C obtained in step 1. This gives the coordinates of the feature point on the image to be overlaid before cropping.

6. The method for improving image feature matching accuracy as described in claim 1, characterized in that, The feature point deduplication algorithm includes: The feature point deduplication algorithm uses the K-Means clustering algorithm.

7. A system for improving the accuracy of image feature matching, characterized in that, include: The cropping module is used to traverse the rows and columns of the image to be overlaid, and based on the gradient value distribution of each row or column, combined with the upper and lower limits of the gradient value threshold, to crop out rows or columns with extreme gradient values ​​exceeding the proportional threshold. The contrast enhancement module is used to perform grayscale transformation on the cropped image to obtain an image with greater contrast. The feature extraction module is used to extract features from the cropped, contrast-enhanced image using a feature extraction algorithm to obtain feature points. The conversion module is used to convert the coordinates of the feature points and map them onto the image to be superimposed before cropping; The feature point deduplication module uses a feature point deduplication algorithm to deduplicate the feature points of the image to be superimposed, removing duplicate feature points. The feature matching module is used to match the deduplicated feature points using a feature matching algorithm to obtain matching points, and to use the matching points to calculate the perspective transformation matrix between the two images to be superimposed. The output module is used to invert one of the two images to be superimposed, perform a transmission transformation on it using the transmission transformation function and the perspective transformation matrix to obtain a registration map, invert the other image, and add it to the registration map to obtain the superimposed image. The superimposed image is then inverted to obtain the target image.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images