A High-Precision Image Feature Matching Method Based on Global Structural Consistency Using Game Theory

Abstract

This invention discloses a high-precision matching method for globally structurally consistent image features based on game theory. The method includes: acquiring feature points from two images to be matched using a feature detection algorithm and constructing an initial coarse matching set; calculating affine transformations based on the feature information of the feature points in the matching terms, calculating the payoff values ​​between each pair of matching terms under the same affine transformation, and constructing a payoff matrix; transforming the matching payoff problem into a probability estimation problem for a mixed strategy; calculating the expected and average payoffs of the pure strategy under the mixed strategy, and finding the optimal strategy in the pure strategy set; finally, adjusting the probability distribution of the mixed strategy according to the optimal strategy until it reaches a Nash equilibrium point, obtaining an evolutionarily stable strategy, and eliminating mismatched terms. This invention has strong adaptability to application environments, high robustness, simplifies the complexity of matching calculations, has a fast response speed, and can effectively improve matching accuracy and computational resource utilization.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a high-precision image feature matching method, which is a game theory-based method for high-precision matching of globally structurally consistent image features. Background Technology

[0002] Computer vision is a key research area in the fields of robot navigation, autonomous driving, and artificial intelligence. Image feature matching plays a crucial role in core vision tasks such as estimating 3D structure and camera pose. For example, in tasks such as panoramic stitching, structure-of-motion (SFM), simultaneous localization and mapping (SLAM), binocular stereo vision, and 3D reconstruction, feature matching provides key support for building 3D models and determining camera pose.

[0003] Currently, image matching methods can be broadly classified into two categories:

[0004] One type is traditional algorithms, which are mainly based on image feature matching, such as Fast Corner Detection (FAST), Scale Invariant Feature Transform (SIFT), and Speed-Up Robust Features (SURF). Feature matching-based methods perform image matching by extracting feature descriptors of feature points in different images.

[0005] Another type is based on deep learning, which automatically extracts image features through deep models such as convolutional neural networks (CNN) and achieves high-precision image matching through feature comparison.

[0006] Compared to deep learning-based methods, feature matching-based methods do not require large datasets for training, while better reflecting the essential structure of images, simplifying image representation, and improving algorithm speed.

[0007] In practical applications, variations in the surface of the imaged object, changes in the imaging viewpoint, or non-rigid transformations can cause distortions in the geometric structure of the same object at different viewpoints. This makes it difficult for traditional feature matching methods to guarantee a global one-to-one match and structural consistency. Secondly, in scenes with varying lighting or illumination differences, a single image feature descriptor may not be accurate enough to describe the image features, potentially leading to mismatches when dealing with repetitive patterns and weak textures. Finally, due to the combinatorial nature of feature matching, matching algorithms typically have high time complexity, making it difficult to maintain matching efficiency for larger datasets. Therefore, how to maintain structural consistency while ensuring high-precision matching of local features is a significant challenge for feature matching. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by providing a high-precision image feature matching method based on game theory and global structural consistency. This matching method considers the geometric structural similarity of the affine transformation of a single matching item relative to the group of matching items, and uses evolutionary game theory to select high-precision matching items. This maximizes the balance between feature matching accuracy and global structural consistency, thereby giving the algorithm stronger adaptability and enabling high-precision image feature matching without increasing the complexity of feature descriptors.

[0009] The technical solution adopted by this invention to solve its technical problem is:

[0010] A high-precision matching method for global structural consistency image features based on game theory includes the following steps:

[0011] Step 1) Use a feature detection algorithm to obtain feature points of two images to be matched, establish feature descriptors and generate feature information, and construct an initial coarse matching set based on the similarity of feature descriptors in the two images;

[0012] Step 2) Calculate the affine transformation based on the feature information between the source feature points and the target feature points in the matching items, find the reprojection deviation between the feature points of the two matching items under the same affine transformation, and then define the profit function based on the deviation to construct the profit matrix.

[0013] Step 3) Transform the matching payoff problem into a probability estimation problem for the mixed strategy and generate the initial probability distribution of the mixed strategy;

[0014] Step 4) Calculate the expected and average returns of the pure strategy under the mixed strategy, and find the optimal strategy in the pure strategy set according to the Nash equilibrium condition.

[0015] Step 5): Based on the found optimal strategy, calculate the magnitude factor and response value of the hybrid strategy, and adjust the probability distribution of the hybrid strategy accordingly. Repeat the dynamic adjustment mechanism consisting of steps 4)-5) to update the probability distribution of the hybrid strategy.

[0016] Step 6) While repeating steps 4)-5), calculate the Nash equilibrium point of the mixed strategy, determine whether the mixed strategy has reached an evolutionary stable state, obtain the evolutionary stable strategy, and retain the fine matching terms according to the probability distribution of the evolutionary stable strategy; repeat steps 2)-6) for the coarse matching terms, and then obtain more fine matching terms.

[0017] Advantages or beneficial effects of the present invention:

[0018] (1) The present invention can effectively deal with problems such as lighting changes, weak texture scenes and repetitive patterns through the evolutionary game matching method, thereby improving the robustness of the system;

[0019] (2) The evolutionary game matching method of the present invention achieves a balance among multiple strategies and considers the global consistency among matching items, making the system more stable when facing the undulations of the imaging object surface, changes in imaging viewpoint or non-rigid transformations.

[0020] (3) The evolutionary game matching model can adaptively adjust its strategy by using a dynamic adjustment mechanism, which can effectively reduce the false matching rate and maintain a high matching accuracy and a sufficient number of fine matching items.

[0021] (4) The dynamic adjustment mechanism of the evolutionary game matching model can simplify the complexity of matching calculation, quickly calculate the results in the application of large data volume, improve the system response speed, and reduce the waste of computing resources. Attached Figure Description

[0022] Figure 1 This is a flowchart of the high-precision matching method for global structural consistency image features based on game theory in the embodiments.

[0023] Figure 2 These are binocular images of the water surface taken in the embodiment.

[0024] Figure 3 This is a schematic diagram of affine transformation and reprojection in the embodiment;

[0025] Figure 4 This is a matching result diagram of the high-precision matching method for global structural consistency image features based on game theory in the embodiment. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments, examples of which are shown in the drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0027] Example:

[0028] To illustrate the high-precision matching method for global structural consistency image features based on game theory designed in this invention, a water surface environment is used as a typical scenario and binocular stereo vision is used as a typical task, but it is not limited to this scenario and task. This matching method can also be used in other scenarios and other tasks.

[0029] like Figure 1 As shown, the high-precision matching method for global structural consistency image features based on game theory includes the following steps:

[0030] Step (1): Use a feature detection algorithm to obtain feature points of two images to be matched, establish feature descriptors and generate feature information, and construct an initial coarse matching set based on the similarity of feature descriptors in the two images.

[0031] The specific implementation method is as follows:

[0032] For two stereo images to be matched with a pixel size of 2448×2048, approximately 2000 feature points are extracted from each image using the SURF (Speeded Up RobustFeatures) feature detection algorithm. Feature descriptors for the feature points are established, and feature information for the corresponding feature points is generated. The feature information includes the position coordinates p, feature scale s, and principal gradient direction θ.

[0033] Taking the left image of the binocular images as the source image and the right image as the target image, the feature information of the source feature points is (p l ,s l ,θ l The feature information of the target feature points is (p) r ,s r ,θ r );

[0034] Then, based on the similarity of feature descriptors in the source image and the target image, the K-nearest neighbor algorithm (KNN) is used for initial matching, and the resulting N matching items are constructed into an initial coarse matching set;

[0035] In this example, one source feature point matches three target feature points, generating a total of N coarse matching terms. The water surface image to be processed is as follows: Figure 2 As shown, the coarse matching terms are finally optimized using the evolutionary game matching model constructed in steps (2)-(6).

[0036] Step (2): Calculate the affine transformation based on the feature information between the source feature points and the target feature points in the matching items, find the reprojection deviation between the feature points of the two matching items under the same affine transformation, and then define the profit function based on the deviation to construct the profit matrix.

[0037] The specific implementation method is as follows:

[0038] Step (2-1) involves calculating the affine transformation of the matching term by combining the position coordinates, feature scale, and principal gradient direction information of the source and target feature points. The affine transformation is shown in equation (1):

[0039]

[0040] p l For matching items (p) l ,p r The source feature points, p r For matching items (p) l ,p r ) target feature points; s l For the source feature point p l Feature scale, sr For the target feature point p r The characteristic scale;

[0041] The source feature point p is obtained through affine transformation. l and virtual point p′ r The affine relation is shown in equation (2):

[0042] p′ r =T(p l ,p r )p l (2)

[0043] Step (2-2): Under the same affine transformation, calculate the reprojection deviation between the two matching terms, as shown in equation (3):

[0044]

[0045] Let O = {1, 2, 3, ..., N} be an enumeration of the matching set, i ∈ O, j ∈ O; d i Represents the affine transformation T(p) li ,p ri Under ) the target feature point p ri and virtual point p′ ri The true error distance p ri -p′ ri With target feature point p rj and virtual point T(p li ,p ri )p lj Prediction error distance p rj -T(p li ,p ri )p lj Distance deviation between; d j Represents the affine transformation T(p) lj ,p rj Under ) the target feature point p rj and virtual point p′ rj The true error distance p rj -p′ rj With target feature point p ri and virtual point T(p lj ,p rj )p li Prediction error distance p ri -T(p lj ,p rj )p li The distance deviation between them; Equation (3) represents the distance deviation between the two matching terms (p li ,p ri ) and (p lj ,p rjThe degree of reprojection deviation between the two can be regarded as a relaxed geometric constraint, and its affine transformation and reprojection diagram is shown in the figure. Figure 3 As shown;

[0046] Figure 3 In the middle, p li and p lj For left image I l Source feature points on p ri and p rj For the right image I r The target feature point on T(p) li ,p ri ) is the matching item (p li ,p ri Affine transformation of ), T(p) lj ,p rj ) is the matching item (p lj ,p rj The affine transformation of p′ is shown in the diagram, where the solid black line connects the source and target feature points, the green and yellow dots represent virtual points obtained from the source feature points under different affine transformations, and the dashed line connects the source and virtual points; where p′ ri It is point p li In affine transformation T(p) li ,p ri The virtual point p′ under ) ri =T(p li ,p ri )p li ;p′ rj It is point p lj In affine transformation T(p) lj ,p rj The virtual point p′ under ) rj =T(p lj ,p rj )p lj ;T(p lj ,p rj )p li It is point p li In affine transformation T(p) lj ,p rj Virtual point under ); T(p li ,p ri )p lj It is point p lj In affine transformation T(p) li ,p ri The virtual points are shown below; the black arrows indicate the feature scale and principal gradient direction of the feature points.

[0047] Step (2-3) defines the payoff function for the similarity between two matching items, as shown in equation (4):

[0048]

[0049] λ is the attenuation coefficient; in this example, λ = 10. -5 ; Calculate the payoff value 'a' between all pairs of matching items. ij The payoff value represents the reprojection deviation between two matching items; the smaller the similarity, the greater the payoff. Finally, an N×N symmetric payoff matrix A = (a ij );

[0050] Step (3) transforms the matching payoff problem into a probability estimation problem for the mixed strategy and generates the initial probability distribution of the mixed strategy;

[0051] The specific implementation method is as follows:

[0052] In game theory, the matching set enumeration O = {1, 2, 3, ..., N} is the enumeration of the pure policy set. The choice probabilities of the pure policies are represented as column vectors x, i.e., mixed policies. The matching problem is transformed into a probability estimation problem for mixed policies. The initial mixed policy x is set as a uniform probability distribution x. i =1 / N;

[0053] x is a standard type located in N-dimensional Euclidean space. In the given information, the support set σ(x) = {i∈O:x} for x∈Δ i >0} is the set of elements with non-zero probability;

[0054] Step (4): Calculate the expected and average returns of the pure strategy under the mixed strategy, and find the optimal strategy in the pure strategy set according to the Nash equilibrium condition.

[0055] The specific implementation method is as follows:

[0056] Step (4-1), the expected return of pure strategy i under the mixed strategy is shown in equation (5):

[0057]

[0058] (Ax) i Let x be the payoff when using pure strategy i under mixed strategy x, with an average payoff of x. T Ax;

[0059] In step (4-2), when using a hybrid strategy y, the average return is y. T Ax; The optimal response to the mixed policy x is the mixed policy set β(x) = {y∈Δ:y} T Ax = max z (z T If the optimal response policy of a mixed policy x is itself, then x is in Nash equilibrium, i.e. yT Ax≤x T Ax, or (Ax) i =x T Ax;

[0060] Step (4-3), let the strategy set τ + ={i∈O:(Ax)} i >x T Ax},τ - ={i∈O:(Ax)} i <x T Ax},τ + Let τ be the set of pure strategies i whose payoff is higher than the average payoff. - Let i be the set of pure strategies whose returns are lower than the average returns.

[0061] According to the Nash equilibrium condition, the optimal policy is set as M(x), as shown in Equation (6). The optimal policy is the pure policy i that maximizes the objective function value.

[0062] M(x)∈argmax{(Ax) i -x T Ax:i∈τ + (x)}∪{-(Ax) i +x T Ax:i∈τ - (x)∩σ(x)} (6)

[0063] Step (5): Based on the found optimal strategy, calculate the magnitude factor and response value of the hybrid strategy, and adjust the probability distribution of the hybrid strategy accordingly. Repeat the dynamic adjustment mechanism consisting of steps 4)-5) to update the probability distribution of the hybrid strategy.

[0064] The specific implementation method is as follows:

[0065] As shown in equation (7), y(x) is a response function. If the current policy x is not in Nash equilibrium, the system will adjust according to the optimal policy through the response function y(x) so that the policy x gradually converges to the Nash equilibrium point, that is, x moves towards e. i Closer, e i It is a standard vector where the i-th term is 1 and the rest are 0;

[0066] If M(x)∈τ + (x), the optimal policy is in the policy set with above-average returns, and the update of x will completely adopt the pure policy i; if M(x)∈τ - Given x ∩ σ(x), the optimal strategy, in a set of strategies with below-average payoffs and not yet extinct, will adjust x towards e. iIn either case, the optimal strategy will propagate, while the remaining strategies will be eliminated. For strategies not belonging to either of these two sets, the mixed strategy x will remain unchanged, where the eliminated strategy is defined by the probability distribution x. i Pure strategies with a value ≤0;

[0067]

[0068] δ y (x) is the amplitude factor, as shown in Equation (8). Its calculation is based on the distance and return comparison between the current strategy x and the response strategy y. The amplitude of the update is adjusted according to the distance between the two strategies, and the step size of x in the update process is controlled so that the evolution of x is reasonable and stable.

[0069] δ y (x) through (yx) T If A(yx) < 0, it indicates whether the change from strategy x to strategy y will lead to a decrease in returns. If the returns decrease, the adjustment range of strategy x needs to be controlled more carefully to avoid an excessive decrease in returns. If strategy y will lead to an increase in returns, a larger strategy adjustment can be made.

[0070]

[0071] Update the hybrid strategy x as shown in Equation (9), and then normalize x as shown in Equation (10), where t is the number of iterations. Repeat the dynamic adjustment mechanism of steps (4) and (5) until it reaches an evolutionary stable state. In this example, t≤5000.

[0072]

[0073] Step (6): While repeating steps (4)-(5), calculate the Nash equilibrium point of the mixed strategy, determine whether the mixed strategy has reached the evolutionary stable state, obtain the evolutionary stable strategy, retain the fine matching terms according to the probability distribution of the evolutionary stable strategy, and repeat steps (2)-(6) for the coarse matching terms to obtain more fine matching terms.

[0074] The specific implementation method is as follows:

[0075] Step (6-1) involves using a dynamic adjustment mechanism to calculate and determine the Nash equilibrium error of the optimal strategy while selecting the optimal policy and updating x. error Has the evolutionary stability condition been met? (Nash) error →0, in this example, the evolutionary stability condition is Nash. error <10 -30 The Nash equilibrium error is shown in equation (11). When the optimal strategy leads to extinction, i.e., x iWhen x gradually approaches 0, or when x reaches the Nash equilibrium condition, it is considered to be an evolutionarily stable strategy, and at this time x ends its evolutionary update.

[0076]

[0077] Step (6-2), let σ + ={i∈O:x i >μ*(maxx) i )},σ - ={i∈O:x i ≤μ*(maxx i )}, where μ is the threshold for a robust strategy; in this example, μ = 0.7, σ + Let σ be the set of exact matches. - This is the set of coarse matches;

[0078] After obtaining the evolutionarily stable policy, for i∈σ + The matching terms are retained for i∈σ - The matching terms can then be obtained by repeating steps (2)-(6) to obtain more i∈σ. + The precise matching term; where, after the first round of evolutionary game matching, if i∈σ + If the number of elements is less than m (i.e., the number of winning precise matches is less than m), or the number of iterations exceeds n, then the evolutionary game matching iteration ends. m represents the minimum threshold number of winning precise matches required after the first round of evolutionary game matching, and n represents the maximum number of iterations for evolutionary game matching. In this example, m = 20 and n = 5. In this example, if the number of winning precise matches is less than m after the first round of evolutionary game matching, it can be considered that the similarity of the matching items between the two stereo images is low, i.e., the global consistency is poor, thus terminating the evolutionary game matching. The final matching result is shown in the figure below. Figure 4 As shown.

[0079] Figure 4 This is a visualization of the matching results in this embodiment. The yellow lines represent the matching visualization lines between the source feature points in the left image and the target feature points in the right image. From the visualization of the matching results, it can be seen that in water surface images with weak textures and repeating patterns, the connected feature points are visually similar and correspond in position. The connected feature point lines are distributed regularly in the overlapping areas, and there are no irregular or chaotic feature point lines, indicating that they have high consistency and geometric relationships in space.

[0080] The above description is merely a preferred embodiment of the present invention, intended to help readers understand the implementation method of the present invention. It should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A high-precision matching method for global structural consistency image features based on game theory, characterized in that, Includes the following steps: Step 1) Use a feature detection algorithm to obtain feature points of the two images to be matched, establish feature descriptors and generate feature information, and construct an initial coarse matching set based on the similarity of feature descriptors in the two images; Step 2) Calculate the affine transformation based on the feature information between the source feature points and the target feature points in the matching items, find the reprojection deviation between the feature points of the two matching items under the same affine transformation, and then define the profit function based on the deviation to construct the profit matrix; Step 3) transforms the matching payoff problem into a probability estimation problem for the mixed strategy and generates the initial probability distribution of the mixed strategy; Step 4), calculate the expected and average returns of the pure strategy under the mixed strategy, and find the optimal strategy in the pure strategy set according to the Nash equilibrium condition; Step 5): Based on the found optimal strategy, calculate the magnitude factor and response value of the hybrid strategy, and adjust the probability distribution of the hybrid strategy accordingly. Repeat the dynamic adjustment mechanism consisting of steps 4)-5) to update the probability distribution of the hybrid strategy. Step 5) includes the following steps: As shown in equation (7), It is a response function. If the current mixed policy x is not in Nash equilibrium, the system will respond according to the optimal policy through the response function. Adjustments are made to gradually converge the mixed strategy x to the Nash equilibrium point, i.e., x moves towards... near, It is the first A standard vector with one term being 1 and the rest being 0; if In the optimal policy set where the returns are above average, the update of x will be entirely based on the pure policy. ;if In a set of strategies where the payoff is below average and the strategy has not yet become extinct, the system will adjust the x-direction. Moving; in both of the above strategies, the optimal strategy will be propagated, while the remaining strategies will be eliminated; for strategies that do not belong to either of the above two strategy sets, the mixed strategy x will remain unchanged, where the eliminated strategy is one with a probability distribution of Pure strategy; （7） As shown in Equation (8), its calculation is based on the distance and return comparison between the current mixed strategy x and the response strategy y. The magnitude of the update is adjusted according to the distance between the two strategy ratios, and the step size of x in the update process is controlled to make the evolution of x reasonable and stable. pass Determine whether the change from hybrid strategy x to response strategy y will lead to a decrease in returns. If returns decrease, the adjustment range of hybrid strategy x needs to be controlled more carefully to avoid excessive decline in returns. If response strategy y will lead to an increase in returns, then a larger strategy adjustment should be made. （8） Update the hybrid strategy x as shown in Equation (9), and then normalize x as shown in Equation (10), where t is the number of iterations. Repeat the dynamic adjustment mechanism of steps 4) and 5) until it reaches an evolutionary stable state. （9） （10）； Step 6) While repeating steps 4)-5), calculate the Nash equilibrium point of the mixed strategy, determine whether the mixed strategy has reached an evolutionary stable state, obtain the evolutionary stable strategy, and retain the fine matching terms according to the probability distribution of the evolutionary stable strategy; for the coarse matching terms, repeat steps 2)-6) to obtain more fine matching terms.

2. The high-precision matching method for global structural consistency image features based on game theory according to claim 1, characterized in that, Step 2) includes the following steps: Step (2-1): Combine the position coordinates, feature scale, and principal gradient direction information of the source and target feature points to calculate the affine transformation of the matching term. The affine transformation is shown in equation (1): （1） For matching items The source feature points, For matching items Target feature points; Source feature points The feature scale, For target feature points The feature scale, The principal gradient direction; The source feature points are obtained through affine transformation. and virtual points The affine relation is shown in equation (2): （2） Step (2-2): Under the same affine transformation, calculate the reprojection deviation between the two matching terms, as shown in equation (3): （3） set up For enumeration of matching sets, , ; Represents affine transformation Below, target feature points and virtual points True error distance With target feature points and virtual points Prediction error distance Distance deviation between; Represents affine transformation Below, target feature points and virtual points True error distance With target feature points and virtual points Prediction error distance The distance deviation between them; Equation (3) represents the distance deviation between the two matching items. and The degree of reprojection deviation between them; Step (2-3) defines the payoff function for the similarity between two matching items, as shown in equation (4): （4） The attenuation coefficient is used in this example. Calculate the payoff between each pair of all matching items. The payoff value represents the reprojection deviation between two matching items; the smaller the similarity, the greater the payoff. Finally, an N×N symmetric payoff matrix is ​​constructed. .

3. The high-precision matching method for global structural consistency image features based on game theory according to claim 2, characterized in that, Step 4) includes the following steps: Step (4-1), pure strategy under mixed strategy The expected return is shown in equation (5): （5） Using a pure policy under mixed policy x The average return when responding is ; Step (4-2), when using response strategy y, the average return is The optimal response to a mixed policy x is the mixed policy set. If the optimal response policy of a mixed policy x is itself, then x is in Nash equilibrium, i.e. , ,or , ; Step (4-3), set the strategy set , , For pure strategy A set of pure strategies whose returns are higher than the average return. For pure strategy A set of pure strategies with returns below average; According to the Nash equilibrium condition, the optimal strategy is set as follows: As shown in equation (6), the optimal strategy is the pure strategy that maximizes the objective function value. ; （6）。 4. The high-precision matching method for global structural consistency image features based on game theory according to claim 2, characterized in that, Step 6) includes the following steps: Step (6-1) involves using a dynamic adjustment mechanism to calculate and determine the Nash equilibrium error of the optimal strategy while selecting the optimal policy and updating x. Has the evolutionary stability condition been met? The Nash equilibrium error is shown in equation (11). When the optimal strategy leads to extinction, i.e. When x gradually approaches 0, or when x reaches the Nash equilibrium condition, it is considered to be an evolutionarily stable strategy, and at this time x ends its evolutionary update. （11） Step (6-2), set , , The threshold for a robust strategy For a set of exact matches, This is the set of coarse matches; After obtaining the evolutionarily stable strategy, for The matches are retained, for The matching items are then repeated from steps 2) to 6), resulting in more. The precise matching terms; among them, after the first round of evolutionary game matching, if If the number of elements is less than m, that is, the number of winning fine matches is less than m, or the number of iterations exceeds n, then the evolutionary game matching iteration ends. m represents the minimum number of winning fine matches required after the first round of evolutionary game matching, and n represents the maximum number of iterations for evolutionary game matching.

Images