Image processing method and device for multi-threshold feature point matching

Abstract

The application provides an image processing method and device for multi-threshold feature point matching, which comprises the following steps: obtaining a feature point set of multiple images; performing first matching on each feature point in the feature point set of a first image according to a first preset threshold, and the result of the first matching contains the Hamming distance ratio of each feature point in the feature point set of the first image; optimizing the feature point set of the first image and the feature point set of a second image according to a first set to obtain a first optimized feature point set of the first image and a second optimized feature point set of the second image; performing second matching on each feature point in the first optimized feature point set according to a second preset threshold according to the result of the first matching; and obtaining the feature point matching result of the multiple images according to the first set and a second set. The application solves the problem that the matching effect is not ideal in the image processing process due to unreasonable threshold setting of the distance ratio in the current feature point matching.

Description

Technical Field

[0001] This application relates to the technical field of image processing, and in particular to an image processing method and apparatus for multi-threshold feature point matching. Background Technology

[0002] Feature matching is a crucial and essential part of visual tracking and image fusion technologies. Features are the "interesting" part of a digital image, serving as the starting point for many computer image analysis algorithms. The success of an algorithm is often determined by the features it uses and defines. Image matching is the process of identifying corresponding points between two or more images using a specific matching algorithm. Because feature points are invariant to rotation, scale changes, and brightness variations, and also maintain a certain degree of stability against viewpoint changes, affine transformations, and noise, image matching is often accomplished through feature point matching. In feature point matching, brute-force algorithms (BF) are commonly used.

[0003] The Browser-Filler (BF) algorithm works by selecting the target point from the feature set, finding the two points in another feature set with the closest Hamming distance to the target point, and then calculating the distances between these two points and the target point. In this process, a threshold for the distance ratio plays a crucial role. However, in current feature point matching, an unreasonable threshold setting for the distance ratio often leads to unsatisfactory matching results during image processing.

[0004] Currently, there is an urgent need for an image processing method and apparatus for multi-threshold feature point matching to solve the problems existing in related technologies. Summary of the Invention

[0005] This application provides an image processing method and apparatus for multi-threshold feature point matching, which addresses the problem that unsatisfactory matching results occur during image processing due to unreasonable threshold settings for distance ratios in current feature point matching. By employing the above method, this application can achieve more accurate feature point matching in images.

[0006] This application provides a multi-threshold feature point matching image processing method, the method comprising: acquiring feature point sets of multiple images, the feature point sets of multiple images including a first image feature point set and a second image feature point set; performing a first matching on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature point set that satisfies the first preset threshold, wherein the result of the first matching contains the Hamming distance ratio of each feature point in the first image feature point set; optimizing the first image feature point set and the second image feature point set according to the first set to obtain a first optimized image feature point set and a second optimized image feature point set respectively; performing a second matching on each feature point in the first optimized image feature point set according to a second preset threshold according to the result of the first matching to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold; wherein the second preset threshold is greater than the first preset threshold; and obtaining feature point matching results of multiple images according to the first set and the second set.

[0007] By employing the above method, this application performs multiple matchings by varying the threshold during the feature point matching process of an image, and optimizes the feature point set before the next matching based on the results of the previous matching, thereby making the feature point matching effect of the image more accurate and the matching efficiency more efficient.

[0008] In one possible implementation, before performing a second matching on each feature point of the first optimized image feature point set according to a second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, the first optimized image feature point set is sorted according to the result of the first matching to determine the matching order when performing the second matching.

[0009] This application employs the aforementioned method to sort each feature point in the image feature point set according to the results of the previous matching before matching. This ensures that during the current matching, each feature point in the first optimized image feature point set can be matched in ascending order of matching ratio, thereby improving the matching speed.

[0010] In one possible implementation, before performing a first match on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of feature points in the second image feature point set that satisfies the first preset threshold, the method further includes: obtaining the number of feature points in the first image feature point set and the second image feature point set; when the number of feature points in the first image feature point set meets the matching requirements, determining whether the number of feature points in the second image feature point set meets the matching requirements; when the number of feature points in the second image feature point set meets the matching requirements, determining that the second matching can be performed normally.

[0011] By employing the above method, this application first detects the number of feature points in the image feature point set before performing feature point matching. After the number of feature points meets the matching requirements, feature point matching is then performed, thus avoiding the situation where the matching process fails and no matching result is generated.

[0012] In one possible implementation, each feature point in the first image feature point set is matched according to a first preset threshold to obtain a first set of the second image feature point set that satisfies the first preset threshold. Specifically, this includes: obtaining a first feature point in the first image feature point set according to a first preset order; obtaining a second feature point and a third feature point in the second image feature point set; wherein the second feature point is the feature point in the second image feature point set that has the closest Hamming distance to the first feature point; the third feature point is the feature point in the second image feature point set that has the second closest Hamming distance to the first feature point; obtaining a first Hamming distance ratio between the first feature point and the second feature point and the first Hamming distance between the first feature point and the third feature point; when the first Hamming distance ratio is less than the first preset threshold, determining the matching relationship between the first feature point and the second feature point, so as to add the second feature point into the first set.

[0013] This application uses the above method to match a set of feature points from a second set of image feature points that match the matching relationships of each feature point in the first set of image feature points by comparing a first Hamming distance ratio with a first preset threshold.

[0014] In one possible implementation, based on the result of the first matching, each feature point in the first optimized image feature point set is matched a second time according to a second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold. Specifically, this includes: selecting feature points sequentially from the first optimized image feature point set according to the matching order, wherein the first optimized image feature point set includes a fourth feature point; querying the result of the first matching for the second Hamming distance ratio corresponding to the fourth feature point; and adding the fourth feature point to the second set when the second Hamming distance ratio is less than the second preset threshold.

[0015] This application, by adopting the above method, directly obtains the feature point matching result based on the relationship between the Hamming distance ratio of each feature point in the first image feature point set obtained in the previous matching and the second preset threshold, thereby reducing the steps of calculating the Hamming distance ratio of feature points and improving the efficiency of feature point matching.

[0016] In one possible implementation, based on the first set, the first image feature point set and the second image feature point set are optimized to obtain the first optimized image feature point set and the second optimized image feature point set, respectively. Specifically, this includes: when the first ratio is less than a first preset threshold, removing the first feature point from the first image feature point set to obtain the first optimized image feature point set; and removing the second feature point from the second image feature point set to obtain the second optimized image feature point set.

[0017] This application optimizes the feature point set of each image for the next matching by adopting the above method based on the previous matching result, thereby reducing the number of features in the feature point set and improving the matching efficiency.

[0018] In one possible implementation, before performing a second matching on each feature point of the first optimized image feature point set according to a second preset threshold based on the result of the first matching, to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, the method further includes: when the number of feature points in the first optimized image feature point set is zero, determining the first set as the feature point matching result of multiple images; when the number of feature points in the second optimized image feature point set is less than a preset number, determining the first set as the feature point matching result of multiple images.

[0019] A second aspect of this application provides an image processing apparatus for multi-threshold feature point matching. The apparatus is an image processing server, and the server includes: a feature point acquisition unit, a first matching unit, a set optimization unit, a second matching unit, and a matching result acquisition unit. The feature point acquisition unit is used to acquire feature point sets of multiple images, including a first image feature point set and a second image feature point set. The first matching unit is used to perform a first matching on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature point set that satisfies the first preset threshold. The result of the first matching contains the Hamming distance ratio of each feature point in the first image feature point set. The set optimization unit is used to optimize the first image feature point set and the second image feature point set according to the first set to obtain a first optimized image feature point set and a second optimized image feature point set, respectively. The second matching unit is used to perform a second matching on each feature point in the first optimized image feature point set according to a second preset threshold based on the result of the first matching to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold. The second preset threshold is greater than the first preset threshold. The matching result acquisition unit is used to obtain feature point matching results for multiple images based on the first set and the second set.

[0020] In one possible implementation, the server further includes a sorting unit; the sorting unit is used to sort the first optimized image feature point set according to the result of the first matching, so as to determine the matching order when performing the second matching.

[0021] In one possible implementation, the server further includes a matching requirement detection unit; the matching requirement detection unit is used to obtain the number of feature points in the first image feature point set and the second image feature point set; when the number of feature points in the first image feature point set meets the matching requirement, it determines whether the number of feature points in the second image feature point set meets the matching requirement; when the number of feature points in the second image feature point set meets the matching requirement, it determines that the second matching can be performed normally.

[0022] In one possible implementation, the first matching unit includes a first feature point acquisition module, a second feature point acquisition module, a ratio acquisition module, and a first matching relationship judgment module. The first feature point acquisition module is used to acquire a first feature point in a first image feature point set according to a first preset order. The second feature point acquisition module is used to acquire a second feature point and a third feature point in a second image feature point set. The second feature point is the feature point in the second image feature point set that has the closest Hamming distance to the first feature point. The third feature point is the feature point in the second image feature point set that has the second closest Hamming distance to the first feature point. The ratio acquisition module is used to acquire a first Hamming distance ratio between the first feature point and the second feature point and the first Hamming distance between the first feature point and the third feature point. The first matching relationship judgment module is used to determine the matching relationship between the first feature point and the second feature point when the first Hamming distance ratio is less than a first preset threshold, so as to add the second feature point to the first set.

[0023] In one possible implementation, the second matching unit includes a feature point selection module, a ratio query module, and a second matching relationship judgment module; the feature point selection module is used to select feature points sequentially from a first optimized image feature point set according to the matching order, the first optimized image feature point set including a fourth feature point; the ratio query module is used to query the second Hamming distance ratio corresponding to the fourth feature point in the result of the first matching; the second matching relationship judgment module is used to add the fourth feature point to the second set when the second Hamming distance ratio is less than a second preset threshold.

[0024] In one possible implementation, the set optimization unit includes a first feature point removal module and a second feature point removal module; the first feature point removal module is used to remove a first feature point from the first image feature point set when the first ratio is less than a first preset threshold, to obtain a first optimized image feature point set; the second feature point removal module is used to remove a second feature point from the second image feature point set when the first ratio is less than the first preset threshold, to obtain a second optimized image feature point set.

[0025] In one possible implementation, the server further includes a result output unit, which includes a first result output module and a second result output module; the first result output module is used to determine that the first set is a feature point matching result of multiple images when the number of feature points in the first optimized image feature point set is zero; the second result output module is used to determine that the first set is a feature point matching result of multiple images when the number of feature points in the second optimized image feature point set is less than a preset number.

[0026] A third aspect of this application provides an electronic device, which includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform any of the methods described above.

[0027] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform any of the methods described above.

[0028] Compared with related technologies, the beneficial effects of this application are:

[0029] 1. In the process of feature point matching in an image, multiple matchings are performed by changing the threshold. At the same time, the feature point set before the next matching is optimized based on the results of the previous matching. This can make the feature point matching effect of the image more accurate and the matching efficiency more efficient.

[0030] 2. Before matching, the feature points in the image feature point set are sorted according to the results of the previous matching. This ensures that during the current matching, the feature points in the first optimized image feature point set are matched in ascending order of matching ratio, thus improving the matching speed.

[0031] 3. Based on the previous matching, the feature point matching result is obtained directly by using the relationship between the Hamming distance ratio of each feature point in the first image feature point set obtained in the previous matching and the second preset threshold. This reduces the steps of calculating the Hamming distance ratio of feature points and improves the efficiency of feature point matching.

[0032] 4. Based on the previous matching results, the feature point set of each image for the next matching is optimized, which reduces the number of features in the feature point set and improves the matching efficiency. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a first scene of an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0034] Figure 2 This is a schematic diagram of a second scene of an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0035] Figure 3 This is a first flowchart illustrating an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the second process of an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the third process of an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the fourth process of an image processing method for multi-threshold feature point matching provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of an image processing device for multi-threshold feature point matching provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0041] Reference numerals: 11, Feature point acquisition unit; 12, First matching unit; 13, Set optimization unit; 14, Second matching unit; 15, Matching result acquisition unit; 800, Electronic device; 801, Processor; 802, Communication bus; 803, User interface; 804, Network interface; 805, Memory. Detailed Implementation

[0042] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0043] In the description of the embodiments of this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0044] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0045] The method described in this application is applied to an image processing server to address the problem that, in current feature point matching, the matching effect is unsatisfactory due to an unreasonable threshold setting for the distance ratio. By employing the above method, this application can achieve more accurate feature point matching in images.

[0046] In the embodiments of this application, Hamming distance is commonly used in signal processing to indicate the minimum operation (bit substitution) required to transform one signal into another. In practice, it involves comparing how many bits of two bit strings are different. A simplified operation is to count the number of 1s after XORing the two bit strings. Therefore, Hamming distance can be used to calculate the similarity of binary descriptors.

[0047] In this embodiment, ORB features extracted from two images will form 256-bit binary descriptors. If two feature points in the two images are considered to have similar binary descriptors, their Hamming distance is calculated. If the Hamming distances are close and no other feature points are close to them, they are considered the same feature point. To describe the degree of Hamming distance matching, the nearest and second-nearest Hamming distance points are typically found in set {B}. When the nearest distance is smaller than the second-nearest distance, the point corresponding to the nearest distance is considered to match the feature points in set {A} better than other feature points, thus completing the matching of {Ai, Bj}.

[0048] The aforementioned standard matching suffers from the problem of setting a hyperparameter threshold. If the hyperparameter threshold T is set too large, some non-optimal matches may not meet the threshold range, resulting in insufficient matching accuracy and even numerous false matches, significantly hindering subsequent image computation and analysis. If the hyperparameter threshold T is set too small, the matching will be more accurate, but there may be fewer matching feature pairs, or even matches that do not meet the hyperparameter threshold T, directly preventing subsequent image computation and analysis. Furthermore, due to the diverse and complex application scenarios and the vast differences in images, it is difficult to use a uniform hyperparameter threshold T for constraint.

[0049] like Figure 1 The diagram shown is a matching scenario with a large threshold provided in an embodiment of this application. Figure 2 This is a schematic diagram of a matching scenario with a relatively small threshold, provided as an embodiment of this application. Figure 1 Image 1 shows many incorrect matches, which is due to a high threshold setting; Image 2 shows fewer matches, which is due to a low threshold setting and overly strict conditions. Due to the complexity of the scene, it is difficult to find a universally suitable optimal threshold.

[0050] This application proposes a method of trying different thresholds sequentially, from small to large, to find the optimal match. In this calculation process, the threshold changes from a smaller value to a larger one, rather than remaining fixed. Smaller values ​​are used to achieve some precise matches, while larger values ​​are used to achieve more matching points. Since matching is performed at multiple thresholds, a simple loop would significantly increase the computational load, especially for Hamming distance, leading to poor real-time performance and failing to meet online detection and matching requirements. This application proposes a method that utilizes previous calculation results and avoids unnecessary Hamming distance calculations. The matching ratio F is only recalculated when the matching source fails, reducing potential loop calls caused by multiple domain values, greatly reducing CPU computation, fully utilizing the calculation results of previous loops for matching verification, and adding a multi-threaded algorithm to significantly improve system response. Furthermore, it proposes a method of quickly sorting by the smallest matching ratio F, prioritizing matches with the smallest F value, greatly improving matching accuracy.

[0051] Figure 3 This is a flowchart illustrating an adaptive multi-threshold feature point image processing method provided in an embodiment of this application. Figure 3 As shown, the method includes S11-S15.

[0052] S11, obtain feature point sets of multiple images, including feature point sets of the first image and feature point sets of the second image.

[0053] For example, such as Figure 1 as well as Figure 2 As shown in the figure, this embodiment extracts feature points from the left and right images using ORB (Oriented Fast and Rotated Brief) feature points. ORB feature points are a set of corner points calculated from pyramid images at different scales, reflecting feature scale invariance. The 256-bit binary descriptor is generated by selecting extraction points with a certain offset along the moment direction based on the image density around the corner point, calculating the information pixel intensity within a 5*5 window at that point, and comparing their sizes to form binary information that describes the identity characteristics of the corner point, exhibiting rotation invariance.

[0054] S12, perform a first match on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature point set that satisfies the first preset threshold. The result of the first match contains the Hamming distance ratio of each feature point in the first image feature point set.

[0055] In this embodiment, when matching feature points in the first image and the second image, an initial matching is performed by setting a small first preset threshold as the threshold for the initial matching. Simultaneously, the result of the first matching includes the Hamming distance ratios of each feature point in the feature point set of the first image.

[0056] In one possible implementation, such as Figure 4 As shown, step S12 specifically includes S121-S124.

[0057] S121, according to the first preset order, obtain the first feature point in the first image feature point set.

[0058] S122, obtain the second feature point and the third feature point in the second image feature point set; wherein, the second feature point is the feature point in the second image feature point set that has the closest Hamming distance to the first feature point; the third feature point is the feature point in the second image feature point set that has the second closest Hamming distance to the first feature point.

[0059] S123, obtain the first Hamming distance ratio between the first feature point and the second feature point and the first Hamming distance between the first feature point and the third feature point;

[0060] S124, when the first Hamming distance ratio is less than the first preset threshold, determine the matching relationship between the first feature point and the second feature point, so as to add the second feature point into the first set.

[0061] Specifically, the first feature points in the first image feature point set are obtained according to a first preset order. That is, they are sorted in ascending order of Hamming distance ratio. Before the first matching, the Hamming distance ratio of each feature point in the first image is set to 0, and the sorting method for the first matching is the default sorting. After the first matching, since the result of the first matching contains the Hamming distance ratios of each feature point in the first image feature point set, in subsequent matching processes that adjust the matching threshold, the feature points are sorted in ascending order of the Hamming distance ratios obtained from the first matching.

[0062] In one possible implementation, when the first Hamming distance ratio is greater than or equal to the first preset threshold, it is determined that the first feature point and the second feature point do not match, and the corresponding first Hamming distance ratio of the first feature point is recorded to facilitate the next matching.

[0063] In one possible implementation, before performing a first match on each feature point in the first image feature point set according to a first preset threshold in step S12 to obtain a first set of the second image feature points that satisfies the first preset threshold, the method further includes step S16.

[0064] S16, obtain the first image feature point set and the number of feature points in the second image feature point set; when the number of feature points in the first image feature point set meets the matching requirements, determine whether the number of feature points in the second image feature point set meets the matching requirements; when the number of feature points in the second image feature point set meets the matching requirements, determine that the second matching can proceed normally.

[0065] For example, in this embodiment, before matching, it is detected whether the first image feature point set is empty, and simultaneously, it is detected whether the number of feature points in the second image feature point set is less than two. Matching ends directly when the first image feature point set is empty or the number of feature points in the second image feature point set is less than two. In this embodiment, during multiple matching processes, it is necessary to confirm that the number of feature points in the set meets the matching requirements before each matching.

[0066] S13, based on the first set, optimize the first image feature point set and the second image feature point set to obtain the first optimized image feature point set and the second optimized image feature point set, respectively.

[0067] In one possible implementation, step S13 specifically includes S131-S132.

[0068] S131, when the first ratio is less than the first preset threshold, the first feature point is removed from the first image feature point set to obtain the first optimized image feature point set.

[0069] S132, when the first ratio is less than the first preset threshold, the second feature point is removed from the second image feature point set to obtain the second optimized image feature point set.

[0070] Specifically, embodiments of this application provide a method for updating and optimizing the set of image feature points for the next matching operation based on the previous matching result during multiple matching processes, thereby reducing the number of feature points that do not need to be matched again.

[0071] In one possible implementation, when the first ratio is greater than or equal to a first preset threshold, a first feature point is retained from the first image feature point set to obtain a first optimized image feature point set; and two feature points are retained from the second image feature point set to obtain a second optimized image feature point set.

[0072] S14, based on the result of the first matching, perform a second matching on each feature point of the first optimized image feature point set according to a second preset threshold, so as to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold; wherein, the second preset threshold is greater than the first preset threshold.

[0073] In one possible implementation, such as Figure 5 As shown, step S14 specifically includes S141-S143.

[0074] S141, according to the matching order, select feature points sequentially from the first optimized image feature point set, the first optimized image feature point set including the fourth feature point.

[0075] S142, query the second Hamming distance ratio corresponding to the fourth feature point in the first matching result.

[0076] S143, when the second Hamming distance ratio is less than the second preset threshold, the fourth feature point is added to the second set.

[0077] In one possible implementation, the fourth feature point is added to the next matching set when the second Hamming distance ratio is greater than or equal to the second preset threshold.

[0078] In one possible implementation, such as Figure 6 As shown, before step S14, which performs a second matching on each feature point of the first optimized image feature point set according to a second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, step S17 is also included.

[0079] S17, Based on the result of the first matching, sort the first optimized image feature point set to determine the matching order when performing the second matching.

[0080] Specifically, since matching is performed under multiple thresholds, a simple loop would significantly increase the computational load, especially for Hamming distance, leading to poor real-time performance and failing to meet online detection and matching requirements. This application proposes a method that utilizes previous calculation results and avoids unnecessary Hamming distance calculations. The matching ratio is only recalculated when the matching source fails, reducing potential loop calls caused by multiple domain values, greatly reducing CPU computation, fully utilizing the calculation results of previous loops for verification matching, and adding a multi-threaded algorithm to significantly improve system response. This invention also proposes a fast sorting method based on the minimum matching ratio, prioritizing matching with the minimum ratio to improve matching accuracy.

[0081] In one possible implementation, before step S14, where each feature point of the first optimized image feature point set is matched according to a second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, the method further includes step S18.

[0082] S18, when the number of feature points in the first optimized image feature point set is zero, the first set is determined to be the feature point matching result of multiple images; when the number of feature points in the second optimized image feature point set is less than a preset number, the first set is determined to be the feature point matching result of multiple images.

[0083] In one possible implementation, when the number of feature points in the first optimized image feature point set is not zero, and when the number of feature points in the second optimized image feature point set is greater than or equal to 2, a third preset threshold is set to continue the next matching process. The third preset threshold is greater than the second preset threshold.

[0084] In one possible implementation, when feature point matching is performed multiple times, the matching threshold increases by an equal amount based on the number of matching attempts.

[0085] In the embodiments of this application, the increment of the matching threshold is set according to the actual situation, which will not be elaborated on in this application.

[0086] S15. Based on the first set and the second set, obtain the feature point matching results of multiple images.

[0087] As shown in Table 1, the improved algorithm proposed in this embodiment achieves better matching results and avoids erroneous matching, making it adaptable to complex and ever-changing environments. Table 1 compares execution times. Due to the improved BF algorithm time, the addition of quicksort, and multiple rounds of matching threshold attempts, the execution time is longer than the standard BF algorithm. However, this embodiment reduces unnecessary computational overhead, resulting in only a slight increase in execution time compared to the standard BF algorithm, without a significant increase of several times. This achieves better matching quality and demonstrates the efficiency of this method.

[0088] Robustness analysis: Since sets {A} and {B} have a finite number of elements, the number of times set {A} needs to be traversed to retrieve an element is finite. Retrieving the nearest and second-most-approaching elements from set {B} has a time complexity of O(n), so computation is prioritized. When a matching pair is obtained, the number of elements in sets {A} and {B} decreases accordingly, further reducing computation. When no match can be found, the matching process exits, and the threshold increases, limiting the number of loop iterations. Therefore, the overall system computation time is finite, and the system is in a stable convergent state.

[0089] In summary, this application addresses the difficulty of setting feature point matching thresholds by proposing an improved cyclic threshold solution for matching. This method achieves multi-threshold coverage and reduces the computational load of the system. It can efficiently and accurately perform feature point matching calculations, meeting the needs of real-time online computing.

[0090] Table 1

[0091]

[0092] The beneficial effects that can be achieved by adopting the above-described method embodiments are as follows:

[0093] 1. In the process of feature point matching in an image, multiple matchings are performed by changing the threshold. At the same time, the feature point set before the next matching is optimized based on the results of the previous matching, which can make the feature point matching effect of the image more accurate and the matching efficiency more efficient.

[0094] 2. Before matching, the feature points in the image feature point set are sorted according to the results of the previous matching. This ensures that during the current matching, the feature points in the first optimized image feature point set are matched in ascending order of matching ratio, thus improving the matching speed.

[0095] 3. Based on the previous matching, the feature point matching result is obtained directly by using the relationship between the Hamming distance ratio of each feature point in the first image feature point set obtained in the previous matching and the second preset threshold. This reduces the steps of calculating the Hamming distance ratio of feature points and improves the efficiency of feature point matching.

[0096] 4. Based on the previous matching results, the feature point set of each image for the next matching is optimized, which reduces the number of features in the feature point set and improves the matching efficiency.

[0097] This application provides an adaptive multi-threshold feature point image processing device, such as... Figure 7 As shown, the image processing server includes: a feature point acquisition unit 11, a first matching unit 12, a set optimization unit 13, a second matching unit 14, and a matching result acquisition unit 15. The feature point acquisition unit 11 is used to acquire feature point sets of multiple images, including a first image feature point set and a second image feature point set. The first matching unit 12 is used to perform a first matching on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature points that satisfies the first preset threshold. The result of the first matching contains the first set of feature points from the first image feature point set. The distance ratio; set optimization unit 13, used to optimize the first image feature point set and the second image feature point set according to the first set, to obtain the first optimized image feature point set and the second optimized image feature point set respectively; second matching unit 14, used to perform a second matching on each feature point of the first optimized image feature point set according to a second preset threshold according to the result of the first matching, to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold; wherein, the second preset threshold is greater than the first preset threshold; matching result acquisition unit 15, used to obtain the feature point matching results of multiple images according to the first set and the second set.

[0098] In one possible implementation, the server further includes a sorting unit; the sorting unit is used to sort the first optimized image feature point set according to the result of the first matching, so as to determine the matching order when performing the second matching.

[0099] In one possible implementation, the server further includes a matching requirement detection unit; the matching requirement detection unit is used to obtain the number of feature points in the first image feature point set and the second image feature point set; when the number of feature points in the first image feature point set meets the matching requirement, it determines whether the number of feature points in the second image feature point set meets the matching requirement; when the number of feature points in the second image feature point set meets the matching requirement, it determines that the second matching can be performed normally.

[0100] In one possible implementation, the first matching unit 12 includes a first feature point acquisition module, a second feature point acquisition module, a ratio acquisition module, and a first matching relationship judgment module. The first feature point acquisition module is used to acquire a first feature point in a first image feature point set according to a first preset order. The second feature point acquisition module is used to acquire a second feature point and a third feature point in a second image feature point set. The second feature point is the feature point in the second image feature point set that has the closest Hamming distance to the first feature point. The third feature point is the feature point in the second image feature point set that has the second closest Hamming distance to the first feature point. The ratio acquisition module is used to acquire a first Hamming distance ratio between the first feature point and the second feature point and the first Hamming distance between the first feature point and the third feature point. The first matching relationship judgment module is used to determine the matching relationship between the first feature point and the second feature point when the first Hamming distance ratio is less than a first preset threshold, so as to add the second feature point into the first set.

[0101] In one possible implementation, the second matching unit 14 includes a feature point selection module, a ratio query module, and a second matching relationship judgment module; the feature point selection module is used to select feature points sequentially from a first optimized image feature point set according to the matching order, the first optimized image feature point set including a fourth feature point; the ratio query module is used to query the second Hamming distance ratio corresponding to the fourth feature point in the result of the first matching; the second matching relationship judgment module is used to add the fourth feature point to the second set when the second Hamming distance ratio is less than a second preset threshold.

[0102] In one possible implementation, the set optimization unit 13 includes a first feature point removal module and a second feature point removal module; the first feature point removal module is used to remove a first feature point from the first image feature point set when the first ratio is less than a first preset threshold, to obtain a first optimized image feature point set; the second feature point removal module is used to remove a second feature point from the second image feature point set when the first ratio is less than the first preset threshold, to obtain a second optimized image feature point set.

[0103] In one possible implementation, the server further includes a result output unit, which includes a first result output module and a second result output module; the first result output module is used to determine that the first set is a feature point matching result of multiple images when the number of feature points in the first optimized image feature point set is zero; the second result output module is used to determine that the first set is a feature point matching result of multiple images when the number of feature points in the second optimized image feature point set is less than a preset number.

[0104] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0105] Please see Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 800 may include: at least one processor 801, at least one network interface 804, a user interface 803, a memory 805, and at least one communication bus 802.

[0106] The communication bus 802 is used to enable communication between these components.

[0107] The user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.

[0108] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0109] The processor 801 may include one or more processing cores. The processor 801 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling data stored in the memory 805. Optionally, the processor 801 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 801 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 801.

[0110] The memory 805 may include random access memory (RAM) or read-only memory. Optionally, the memory 805 may include a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned processor 801. Figure 8 As shown, the memory 805, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for adaptive multi-threshold feature point image processing.

[0111] exist Figure 8In the electronic device 800 shown, the user interface 803 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 801 can be used to call the image processing application stored in the memory 805, which stores adaptive multi-threshold feature points. When executed by one or more processors, the electronic device 800 performs one or more of the methods described in the above embodiments.

[0112] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0113] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0118] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

Claims

1. An image processing method for multi-threshold feature point matching, characterized in that, Applied to an image processing server, the method includes: Acquire feature point sets of multiple images, wherein the feature point sets of multiple images include a first image feature point set and a second image feature point set; Each feature point in the first image feature point set is matched according to a first preset threshold to obtain a first set of the second image feature point set that satisfies the first preset threshold. The result of the first matching contains the Hamming distance ratio of each feature point in the first image feature point set. Based on the first set, the first image feature point set and the second image feature point set are optimized to obtain the first optimized image feature point set and the second optimized image feature point set, respectively. Based on the result of the first matching, each feature point in the first optimized image feature point set is matched a second time according to a second preset threshold to obtain a second set in the second optimized image feature point set that satisfies the second preset threshold; wherein, the second preset threshold is greater than the first preset threshold. Based on the first set and the second set, feature point matching results of multiple images are obtained; The method further includes, before performing a second matching on each feature point of the first optimized image feature point set according to a second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold: Based on the result of the first matching, the first optimized image feature point set is sorted to determine the matching order when performing the second matching. The step of performing a second match on each feature point of the first optimized image feature point set according to a second preset threshold based on the result of the first matching, to obtain a second set of the second optimized image feature points that satisfies the second preset threshold, specifically includes: According to the matching order, feature points are selected sequentially from the first optimized image feature point set, wherein the first optimized image feature point set includes a fourth feature point; In the query results of the first match, the second Hamming distance ratio corresponding to the fourth feature point; When the second Hamming distance ratio is less than the second preset threshold, the fourth feature point is added to the second set; When the second Hamming distance ratio is greater than or equal to the second preset threshold, the fourth feature point is added to the next matching set.

2. The method according to claim 1, characterized in that, Before performing a first matching on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature points that satisfies the first preset threshold, the method further includes: Obtain the number of feature points in the first image feature point set and the second image feature point set; When the number of feature points in the first image feature point set meets the matching requirements, it is determined whether the number of feature points in the second image feature point set meets the matching requirements. When the number of feature points in the second image feature point set meets the matching requirements, it is determined that the second matching can proceed normally.

3. The method according to claim 1, characterized in that, The step of performing a first matching on each feature point in the first image feature point set according to a first preset threshold to obtain a first set of the second image feature points that satisfies the first preset threshold specifically includes: According to a first preset order, the first feature point in the first image feature point set is obtained; Obtain the second feature point and the third feature point from the second image feature point set; wherein, The second feature point is the feature point in the second image feature point set that has the closest Hamming distance to the first feature point; The third feature point is the feature point in the second image feature point set that has the second closest Hamming distance to the first feature point; Obtain the first Hamming distance ratio between the first feature point and the second feature point and the first Hamming distance between the first feature point and the third feature point; When the first Hamming distance ratio is less than the first preset threshold, the matching relationship between the first feature point and the second feature point is determined so that the second feature point is added to the first set.

4. The method according to claim 3, characterized in that, The step of optimizing the first image feature point set and the second image feature point set based on the first set to obtain the first optimized image feature point set and the second optimized image feature point set, respectively, specifically includes: When the first Hamming distance ratio is less than the first preset threshold, the first feature point is removed from the first image feature point set to obtain the first optimized image feature point set. When the first Hamming distance ratio is less than the first preset threshold, the second feature point is removed from the second image feature point set to obtain the second optimized image feature point set.

5. The method according to claim 1, characterized in that, Before performing a second matching on each feature point of the first optimized image feature point set according to a second preset threshold based on the result of the first matching, to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, the method further includes: When the number of feature points in the first optimized image feature point set is zero, the first set is determined to be the feature point matching result of multiple images; When the number of feature points in the second optimized image feature point set is less than a preset number, the first set is determined to be the feature point matching result of multiple images.

6. An image processing apparatus for multi-threshold feature point matching, characterized in that, The device is an image processing server, which includes a feature point acquisition unit (11), a first matching unit (12), a set optimization unit (13), a second matching unit (14), and a matching result acquisition unit (15). The feature point acquisition unit (11) is used to acquire feature point sets of multiple images, wherein the feature point sets of multiple images include a first image feature point set and a second image feature point set; The first matching unit (12) is used to perform a first matching on each feature point in the first image feature point set according to a first preset threshold, so as to obtain a first set in the second image feature point set that satisfies the first preset threshold. The result of the first matching contains the Hamming distance ratio of each feature point in the first image feature point set. The set optimization unit (13) is used to optimize the first image feature point set and the second image feature point set according to the first set, so as to obtain the first optimized image feature point set and the second optimized image feature point set respectively. The second matching unit (14) is configured to perform a second matching on each feature point of the first optimized image feature point set according to a second preset threshold based on the result of the first matching, so as to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold; wherein the second preset threshold is greater than the first preset threshold; before performing the second matching on each feature point of the first optimized image feature point set according to the second preset threshold to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold, the first optimized image feature point set is sorted according to the result of the first matching to determine the matching order when performing the second matching; the first matching is performed according to the result of the first matching. The matching result involves performing a second matching on each feature point in the first optimized image feature point set according to a second preset threshold, to obtain a second set of the second optimized image feature point set that satisfies the second preset threshold. Specifically, this includes: selecting feature points sequentially from the first optimized image feature point set according to the matching order, wherein the first optimized image feature point set includes a fourth feature point; querying the second Hamming distance ratio corresponding to the fourth feature point in the first matching result; adding the fourth feature point to the second set when the second Hamming distance ratio is less than the second preset threshold; and adding the fourth feature point to the next matching set when the second Hamming distance ratio is greater than or equal to the second preset threshold. The matching result acquisition unit (15) is used to obtain feature point matching results of multiple images based on the first set and the second set.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method of any one of claims 1-5.

8. An electronic device, characterized in that, The device includes a processor (801), a user interface (803), a network interface (804), and a memory (805), wherein the memory (805) is used to store instructions, the user interface (803) and the network interface (804) are used to communicate with other devices, and the processor (801) is used to execute the instructions stored in the memory (805) to cause the electronic device (800) to perform the method as described in any one of claims 1-5.

Images