A method for judging and removing invalid feature points under a multi-level pyramid structure

Abstract

This invention discloses a method for judging and removing invalid feature points under a multi-level pyramid structure, comprising: extracting feature points from an image; dividing the image into ideal and non-ideal regions; performing a uniqueness judgment on each feature point in the ideal region, filtering out feature points with poor uniqueness, and forming bottom-level invalid feature points together with feature points in the non-ideal region; establishing a multi-level image pyramid structure and extracting feature points at each upper level; performing downsampling mapping discrimination on each upper-level feature point based on the bottom-level invalid feature points and the pyramid level mapping relationship, classifying feature points that satisfy the downsampling mapping discrimination relationship as invalid feature points; thereby removing invalid feature points at each level. This invention, by performing uniqueness judgment and filtering after extracting image feature points and filtering non-ideal regions under a multi-level pyramid structure, eliminates invalid feature points, thereby reducing the invalid resource consumption of the matching algorithm and significantly improving the matching accuracy and speed.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a method for judging and removing invalid feature points under a multi-level pyramid structure. Background Technology

[0002] Image matching technology plays a crucial role in image processing, with wide applications in computer vision, pattern recognition, remote sensing image processing, and medical image analysis. Image matching methods are mainly categorized into grayscale-based matching algorithms, transform domain-based matching algorithms, and feature-based matching algorithms. Among these, feature-based matching algorithms are the mainstream. The matching effectiveness is influenced not only by the feature detection algorithm and matching method but also by the quality of the feature points. Extracting high-quality, effective feature points helps improve the accuracy and efficiency of image matching. High-quality, effective feature points have three main characteristics: first, they can detect stable feature points regardless of image interference, meaning they are unique; second, the feature information contained within the feature points is representative; and third, the feature points must be located on the edges or corners of the matching region, while ensuring a certain distance between points to avoid dense distribution.

[0003] To obtain high-quality feature points and enhance image matching, relevant research has been conducted. Regarding the effectiveness of feature points, methods such as using the Harris corner detection operator to filter SIFT feature points and eliminate a large number of pseudo-feature points in image edge regions have proven effective, but the Harris corner detection is sensitive to manually set thresholds, limiting its effectiveness in eliminating pseudo-feature points. Regarding the stability of feature points, a stability criterion based on the gray-level entropy difference in the feature point's neighborhood has been proposed. While this criterion effectively reduces unstable feature points, multiple points in a local neighborhood may satisfy the stability criterion and be retained, resulting in a dense distribution of feature points. Regarding the density of feature points, methods utilizing image information entropy can improve the problem of excessively dense feature point distribution. This method ensures the uniformity of feature points but does not solve the matching problem of complex and interfering images.

[0004] Image pyramids are a method for representing images at multiple scales. They interpret images through multiple resolutions and are primarily used in image matching and segmentation. The construction method of an image pyramid involves starting with the original image and progressively downsampling to obtain a series of sub-images with gradually decreasing resolution. These sub-images are then arranged in a pyramid shape. This structure is simple and effective, allowing the extraction of feature points at different scales across different pyramid levels. Therefore, to improve matching accuracy and speed, many matching algorithms employ image pyramid structures for layered image processing. However, when filtering out invalid feature points, each layer of the pyramid needs to remove the corresponding invalid feature points.

[0005] Currently, there is no good method to screen for highly unique and significant feature points and remove invalid feature points in multi-level pyramid structures. This will inevitably affect the quality and efficiency of the image matching process when performing feature matching. Summary of the Invention

[0006] The purpose of this invention is to provide a method for judging and removing invalid feature points under a multi-level pyramid structure. After extracting feature points, the method first judges the uniqueness of all feature points and removes feature points with poor uniqueness; then, a multi-level image pyramid structure is established, and feature points in non-ideal regions under the multi-level pyramid structure are screened to remove invalid feature points, thereby improving accuracy and speed.

[0007] The technical solution adopted in this invention is as follows:

[0008] A method for identifying and removing invalid feature points in a multi-level pyramid structure includes the following steps:

[0009] Extract feature points from the image;

[0010] Divide the image into ideal and non-ideal regions;

[0011] For each feature point in the ideal region, a uniqueness check is performed, and feature points with poor uniqueness are selected and combined with feature points in the non-ideal region to form the bottom layer of invalid feature points.

[0012] Establish a multi-level image pyramid structure and extract feature points from each upper level;

[0013] Based on the invalid feature points at the bottom level and the pyramid hierarchy mapping relationship, downsampling mapping discrimination is performed on each upper-level feature point, and feature points that satisfy the downsampling mapping discrimination relationship are classified as invalid feature points; whereby the downsampling mapping discrimination formula is:

[0014]

[0015] In the formula, x n With y n y represents the coordinates of each upper-level feature point, x and y represent the coordinates of the lower-level invalid feature points, N1, N2, N3 and N4 represent the neighborhood scale, and M represents the downsampling factor.

[0016] This removes invalid feature points at each level.

[0017] Furthermore, the image pyramid structure is a Gaussian image pyramid structure, and a feature point extraction algorithm is used to extract feature points of the image and its upper levels.

[0018] Furthermore, for each feature point within the ideal region, a uniqueness check is performed, and feature points with poor uniqueness are filtered out. Specifically:

[0019] For each feature point within the ideal region, a neighborhood search is performed. If there is a feature point in the neighborhood whose cross-entropy with the feature point is less than a preset threshold, then the uniqueness of the feature point at the center position is poor.

[0020] Furthermore, the formula for calculating cross-entropy is:

[0021]

[0022] In the formula, D(g,h) is the cross-entropy, m is the dimension of the descriptor, g(p,i) is the descriptor of feature point p, and h(p′,i) is the descriptor of feature point p′.

[0023] Furthermore, an M×M neighborhood search is performed on each feature point within the ideal region, where M is 48.

[0024] Furthermore, the preset threshold value is 0.3.

[0025] Furthermore, when establishing a multi-level image pyramid structure, the multi-level image pyramid structure should be established as high as possible while ensuring that the number of feature points in the highest pyramid layer is not less than 4.

[0026] Furthermore, N1, N2, N3, and N4 are each 8.

[0027] Furthermore, the downsampling factor M is 2. n , where n is the level of the upper-level feature point, and the lower level is level 0.

[0028] Furthermore, the bounding boxes of the ideal and non-ideal regions are downsampled at each level to obtain the range of the non-ideal region at each level, and the feature points within the range are classified as invalid feature points.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] This invention performs uniqueness judgment and screening and non-ideal region screening under a multi-level pyramid structure after extracting image feature points, thereby eliminating invalid feature points and reducing the invalid resource consumption of the matching algorithm, which can significantly improve the matching accuracy and speed. Attached Figure Description

[0031] Figure 1 This is a flowchart of the method for judging and removing invalid feature points under a multi-level pyramid structure according to the present invention;

[0032] Figure 2 This is a schematic diagram of the feature point screening scheme of the present invention;

[0033] Figure 3 This is a schematic diagram showing the process before and after removing feature points with poor uniqueness according to the present invention; wherein, Figure 3 (a) is a schematic diagram of the feature points of the template image. Figure 3 (b) is a schematic diagram for removing feature points with poor uniqueness;

[0034] Figure 4 This is a schematic diagram showing the non-ideal region feature points before and after removal according to the present invention; wherein, Figure 4 (a) Schematic diagram of feature points below the 0th level of the pyramid. Figure 4 (b) is a schematic diagram of removing invalid feature points;

[0035] Figure 5 This is a comparison chart of the matching accuracy of the algorithm before and after removing invalid feature points according to the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0037] Existing feature point extraction techniques do not target the feature pyramid structure to screen for highly unique and significant feature points, nor do they provide a method for eliminating invalid feature points based on the feature pyramid structure.

[0038] This invention provides a method for identifying and removing invalid feature points in a multi-level pyramid structure, comprising the following steps:

[0039] Extract feature points from the image;

[0040] Divide the image into ideal and non-ideal regions;

[0041] For each feature point in the ideal region, a uniqueness check is performed, and feature points with poor uniqueness are selected and combined with feature points in the non-ideal region to form the bottom layer of invalid feature points.

[0042] Establish a multi-level image pyramid structure and extract feature points from each upper level;

[0043] Based on the invalid feature points at the bottom level and the pyramid hierarchy mapping relationship, downsampling mapping discrimination is performed on each upper-level feature point, and feature points that satisfy the downsampling mapping discrimination relationship are classified as invalid feature points; whereby the downsampling mapping discrimination formula is:

[0044]

[0045] In the formula, x n With y nare the coordinates of the upper-level feature points, x and y are the coordinates of the bottom-level invalid feature points, N1, N2, N3, and N4 are neighborhood scales, and M is the downsampling factor;

[0046] Thus, the invalid feature points at each level are removed.

[0047] Furthermore, the downsampling factor M is 2 n , where n is the level where the upper-level feature points are located, and the bottom level is level 0.

[0048] Furthermore, the division boxes of the ideal region and the non-ideal region are correspondingly downsampled level by level to obtain the non-ideal region range at each level, and the feature points within the range are classified as invalid feature points.

[0049] The method for judging and removing invalid feature points under the multi-level pyramid structure of the embodiments of the present invention, as Figure 1 and Figure 2 shown, includes the following steps:

[0050] S1. Extract the feature points of the template image.

[0051] Use a feature point extraction algorithm to extract the feature points of the template image.

[0052] S2. Perform uniqueness judgment on all feature points, and filter out the feature points with poor uniqueness.

[0053] Perform an M×M neighborhood search for each feature point, and calculate the cross-entropy between two points in the neighborhood. The larger the cross-entropy, the better the uniqueness of the feature point, and the feature point is representative.

[0054] The calculation formula of the cross-entropy is:

[0055]

[0056] where m is the dimension of the descriptor, g(p, i) is the descriptor of feature point p, and h(p′, i) is the descriptor of feature point p′.

[0057] If there is a feature point in the local neighborhood whose cross-entropy D < P, where P is an adjustable threshold, it is considered that the uniqueness of the feature point at the center position is poor and needs to be filtered out. Traverse all feature points and remove the feature points with poor uniqueness. In the uniqueness determination, the local neighborhood M takes a value of 48, and the threshold P takes a value of 0.3.

[0058] S3. Establish a multi-level image pyramid structure, and construct a feature point screening rule for the non-ideal region under the multi-level pyramid structure to screen the feature points in the non-ideal region under the multi-level pyramid structure.

[0059] The number of image pyramid levels is determined based on the number of extracted template feature points. Under the premise of ensuring that the number of feature points in the highest pyramid level is not less than 4, the image pyramid model is created as high as possible.

[0060] Based on the pyramid model determined above, generate the feature point set corresponding to each level;

[0061] At the original resolution, the template image is divided into ideal and non-ideal regions. Based on the pyramid hierarchy mapping relationship, feature point selection rules for non-ideal regions under a multi-level pyramid structure are established.

[0062] The feature points are labeled layer by layer for distinguishing non-ideal regions, which is used to differentiate between valid and invalid feature points in each layer.

[0063] The process involves establishing feature point filtering rules for non-ideal regions within a multi-level pyramid structure based on the pyramid hierarchy mapping relationship. The steps include: visualizing the feature point distribution at the original resolution (level 0); dividing the region into non-ideal and ideal regions at this level; and selecting feature points from the non-ideal regions. The selected feature points at the original resolution and the non-ideal regions defined by the selection boxes are then recorded.

[0064] It should be noted that the ideal area is the target area or the desired area.

[0065] S4. Remove invalid feature points according to the filtering rules, and retain valid feature points. Valid feature points can be used for image matching, thus significantly reducing computer resource consumption and improving matching accuracy and speed.

[0066] The multi-level filtering formula for feature points in the multi-level image pyramid feature point downsampling mapping is as follows: (The formula is not provided in the original text.)

[0067]

[0068] Where n is the image pyramid level, N is the neighborhood scale with a value of 8, and x and y are the coordinates of the feature points at the original resolution.

[0069] A downsampling mapping discriminant is used to quickly search for upper-level feature points, filtering out invalid feature points mapped from the lower levels. If, within a local neighborhood, an upper-level feature point and a feature point at the original resolution satisfy the discriminant formula, then the feature point in the higher-level pyramid is considered a labeled feature point at the original resolution. This process is repeated for all levels of feature points, and points that satisfy the mapping discriminant relationship are classified as invalid points.

[0070] Alternatively, the remaining feature points can be directly filtered out using bounding box filtering rules. The selection box is downsampled at each level to obtain the non-ideal region range at each level. The remaining unclassified feature points at the corresponding level are traversed, and feature points within the filtering box range, i.e., in the non-ideal region, are classified as invalid feature points.

[0071] Based on the above, invalid feature points in the image are removed according to the criteria of non-ideal regions and uniqueness, while valid feature points are retained for matching. The key point of this invention is that ideal regions are divided at the original resolution, and invalid feature points are filtered out layer by layer according to the mapping relationship obtained from the established feature pyramid structure. There are various alternative methods for feature point extraction in this invention, and this invention does not limit these. Since this invention establishes a feature pyramid for the original image, which is essentially image downsampling, the preferred image pyramid structure is a Gaussian image pyramid structure.

[0072] In summary, this invention provides a method for judging and removing invalid feature points under a multi-level pyramid structure. After extracting template feature points, it performs uniqueness judgment and screening, as well as screening for non-ideal regions under a multi-level pyramid structure. Invalid feature points are deleted layer by layer according to the original resolution image and the established screening rules, thereby reducing the invalid resource consumption of the matching algorithm and significantly improving the matching accuracy and speed.

[0073] The template area is selected from the test image. First, the corresponding level is selected according to the pyramid level discriminant. The following table shows the number of feature points extracted from each layer and the number of invalid feature points after removing invalid feature points according to the uniqueness principle and the non-ideal region removal principle under the multi-level pyramid structure.

[0074] Table 1. Number of Feature Points

[0075] Pyramid level 0 1 2 Original feature point count 33 18 7 Number of feature points after removal 20 12 4

[0076] Figure 3 (a) is a schematic diagram of template feature points in this embodiment, including the result of calculating the feature point attributes of the ideal region according to the uniqueness determination criterion. The white-marked points are feature points in their neighborhood where the cross-entropy is less than a set threshold. Figure 3 (b) is a schematic diagram for removing invalid feature points with poor uniqueness. Figure 4 (a) and Figure 4 (b) is a comparison image before and after the removal of feature points in the non-ideal region under the pyramid structure according to the filtering rules in this embodiment (taking the 0th layer as an example), where the hand-shaped region is the non-ideal region and the box region is the ideal region.

[0077] Under the same controlled conditions, the matching speed and matching accuracy before and after removing the invalid feature points are compared. Specifically, using the same image to be detected, the algorithm's matching time is tested with two template feature point sets before and after removing invalid feature points, under the same rotation angle, similarity score, and maximum detection quantity; and when the similarity score is adjusted to a lower level, the ratio of the number of correct matches to the total number of matches at the same similarity score is calculated, which is the comparison of the matching accuracy before and after.

[0078] This embodiment selects 12 sets of images to be detected, each containing a template image, with a size of 2448*2048. The template size is 136*96. To verify the experimental results of this invention, the experimental test environment uses a PC with an AMD Ryzen 7 6800H CPU with Radeon Graphics, 3.20GHz, 16GB of memory, and a 64-bit Windows 11 operating system. The experimental test algorithm of this embodiment is based on OpenCV 4.3.0, the programming language is C++, and the programming environment is Visual Studio 2019.

[0079] Matching accuracy and matching time were used as performance evaluation metrics for the matching algorithm. Experiments were conducted on both metrics under the aforementioned constraints. The experimental data were the average of 24 experiments performed twice on 12 images to be detected. Figure 5 It can be seen that both the original algorithm and the algorithm after removing some feature points improve the matching accuracy to a certain extent. Under the same conditions, compared with the original algorithm, the improved algorithm removes some invalid feature points, reducing the number of feature points. The improved matching accuracy and significantly reduced matching time indicate that this invention successfully removes some duplicate or poorly significant feature points that are ineffective for algorithm performance. The retained feature points are effective and highly characteristic for matching, which is effective in improving the matching accuracy and reducing the matching time as intended by this invention.

[0080] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0081] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying and removing invalid feature points in a multi-level pyramid structure, characterized in that, Includes the following steps: Extract feature points from the image; Divide the image into ideal and non-ideal regions; For each feature point in the ideal region, a uniqueness check is performed, and feature points with poor uniqueness are selected and combined with feature points in the non-ideal region to form the bottom layer of invalid feature points. Establish a multi-level image pyramid structure and extract feature points from each upper level; Based on the invalid feature points at the bottom level and the pyramid hierarchy mapping relationship, downsampling mapping discrimination is performed on each upper-level feature point, and feature points that satisfy the downsampling mapping discrimination relationship are classified as invalid feature points; whereby the downsampling mapping discrimination formula is: Where x n With y n y represents the coordinates of each upper-level feature point, x and y represent the coordinates of the lower-level invalid feature points, N1, N2, N3 and N4 represent the neighborhood scale, and M represents the downsampling factor. This removes invalid feature points at each level.

2. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, The image pyramid structure is a Gaussian image pyramid structure, and a feature point extraction algorithm is used to extract feature points of the image and its upper levels.

3. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, For each feature point within the ideal region, a uniqueness check is performed, and feature points with poor uniqueness are filtered out. Specifically: For each feature point within the ideal region, a neighborhood search is performed. If there is a feature point in the neighborhood whose cross-entropy with the feature point is less than a preset threshold, then the uniqueness of the feature point at the center position is poor.

4. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 3, characterized in that, The formula for calculating cross-entropy is: In the formula, D(g,h) is the cross-entropy, m is the dimension of the descriptor, g(p,i) is the descriptor of feature point p, and h(p′,i) is the descriptor of feature point p′.

5. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 3 or 4, characterized in that, For each feature point within the ideal region, perform an M×M neighborhood search, where M is 48.

6. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 3 or 4, characterized in that, The preset threshold value is 0.

3.

7. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, When constructing a multi-level image pyramid structure, the multi-level image pyramid structure should be constructed as high as possible, while ensuring that the number of feature points in the highest pyramid layer is not less than 4.

8. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, N1, N2, N3 and N4 take values ​​of 8.

9. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, The downsampling factor M is 2 n , where n is the level of the upper-level feature point, and the lower level is level 0.

10. The method for judging and removing invalid feature points in a multi-level pyramid structure according to claim 1, characterized in that, The bounding boxes of the ideal and non-ideal regions are downsampled at each level to obtain the range of the non-ideal region at each level. Feature points within the range are classified as invalid feature points.

Images