Coarse error elimination method and device, equipment and storage medium
Through unsupervised learning and undirected graph construction methods, the rough errors in image matching are accurately eliminated, and the problem of insufficient image matching accuracy and robustness in the prior art is solved, and efficient matching is achieved in complex scenarios.
Patent Information
- Application Number
- CN202510492302.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-08-08
AI Technical Summary
The prior art has the problem of inaccurate removal of rough deviations in image matching, especially in high noise environments or large parallax scenarios, resulting in insufficient matching accuracy and robustness.
The image matching results are processed using unsupervised learning. By mapping local directions and local scales to attribute domains, the global directions and global scale transformation parameters are calculated, the undirected graph is constructed and the nodes are filtered to achieve accurate removal of coarse errors.
The accuracy of coarse deviation removal is improved, the image matching accuracy and robustness in high noise and large parallax scenarios are ensured, and the sensitivity effect on coarse deviation ratio is avoided.
Smart Images
Figure CN120451602A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image matching technology, and in particular to a method, device, equipment and storage medium for eliminating gross errors. Background Art
[0002] The goal of image matching is to find corresponding feature points or pixels between different images, thereby establishing geometric relationships between them for subsequent image stitching, 3D reconstruction, or image geometric correction. Due to geometric distortion, illumination variations, perspective differences, occlusions, and other issues that may exist during image data acquisition, traditional image matching algorithms often introduce a large number of erroneous matching points, or outliers, which seriously affect the accuracy and robustness of image matching.
[0003] In related technologies, the gross error elimination method includes: first, randomly sampling a number of points from the candidate matching point set, assuming that these points are correct points, and estimating the model parameters (such as homography matrix, basic matrix, etc.) through these points; then, using the model to perform consistency detection on all data and calculate the number of inliers (that is, points that conform to the model); if the number of inliers of the current model exceeds the set threshold, the optimal model is updated and this process is repeated a certain number of times (iterative process); finally, the model with the most inliers is selected, and the final model parameters are re-estimated based on all inliers to improve accuracy.
[0004] However, this method is sensitive to the proportion of gross errors. Conventional methods for eliminating gross errors assume that all points randomly sampled in the first pass are correct. In situations where there are a high proportion of gross errors (such as in high-noise environments or scenes with large parallax), the proportion of gross errors in the first pass of random sampling is high, leading to errors in the model parameters estimated based on these points. Such errors in model parameters can lead to low accuracy in the final elimination of gross errors. Summary of the Invention
[0005] The present disclosure provides a method, apparatus, device, and storage medium for removing gross errors, which can accurately remove gross errors. The technical solution includes at least the following solutions: In a first aspect, a method for removing gross errors is provided, comprising: performing image matching on a first image and a second image to obtain an image matching result, the image matching result including a plurality of matching point pairs and a local direction and a local scale of each point in each matching point pair; mapping the local direction and local scale of each matching point pair into an attribute domain to obtain a global direction and a global scale of each matching point pair; processing the matching point pairs in the attribute domain using an unsupervised learning approach to obtain global direction transformation parameters and global scale transformation parameters between images; screening a first matching point pair set from the matching point pairs in the attribute domain based on the global direction transformation parameters and the global scale transformation parameters between images; constructing an undirected graph based on the two-dimensional coordinates of each matching point pair in the first matching point pair set, the two-dimensional coordinates being used to determine a connection relationship between nodes in the undirected graph, a node in the undirected graph being a matching point pair in the first matching point pair set; screening the nodes in the undirected graph based on the connection relationship between the nodes in the undirected graph to obtain a target node set, the matching point pairs indicated by the nodes in the target node set being the matching point pairs obtained after removing gross errors from the image matching result.
[0006] Optionally, in the attribute domain, the correct matching point pairs conform to the Gaussian distribution, and the matching point pairs in the attribute domain are processed by unsupervised learning to obtain global direction transformation parameters and global scale transformation parameters between images, including: constructing a Gaussian mixture model based on the matching point pairs in the attribute domain, the Gaussian mixture model is an unsupervised learning model, and the Gaussian mixture model is used to calculate the probability density of each Gaussian distribution in the attribute domain; using the expectation maximization method to update the parameters of the Gaussian mixture model; and determining the global direction transformation parameters and the global scale transformation parameters between images based on the trained Gaussian mixture model.
[0007] Optionally, determining the global directional transformation parameters and the global scale transformation parameters between images based on the trained Gaussian mixture model includes: obtaining the probability density of each Gaussian distribution in the attribute domain based on the trained Gaussian mixture model; and determining the global directional transformation parameters and the global scale transformation parameters between images based on the mean of the Gaussian distribution with the largest probability density.
[0008] Optionally, constructing an undirected graph based on the two-dimensional coordinates of each matching point pair in the first matching point pair set includes: calculating the direction constraint error and the scale constraint error between the first node and the second node based on the two-dimensional coordinates of the first node and the second node, where the first node and the second node are any two different nodes in the undirected graph; when the direction constraint error between the first node and the second node is equal to the global direction transformation parameter, and the scale constraint error between the first node and the second node is equal to the global scale transformation parameter, connecting the first node and the second node with an edge in the undirected graph.
[0009] Optionally, based on the connection relationship between each node in the undirected graph, the nodes in the undirected graph are screened to obtain a target node set, including: sorting each node in the undirected graph according to the degree of the node, placing the node with the largest degree into a first node set, and placing other nodes in the undirected graph except the node with the largest degree into a second node set; extracting nodes from the second node set in order of degree from large to small, and determining whether the extracted nodes and each node in the first node set meet the set node connection rules; if the extracted nodes meet the node connection rules, the extracted nodes are stored in the first node set; if the extracted nodes do not meet the node connection rules, the extracted nodes are discarded; after each node in the second node set is extracted, the first node set is the target node set.
[0010] Optionally, the node connection rule is used to indicate that the extracted node is connected to all nodes in the first node set; or, the node connection rule is used to indicate that the extracted node is connected to A nodes in the first node set, where A is equal to k times B, k is a regularization coefficient, k is greater than or equal to 0.9, and B is the total number of nodes in the first node set.
[0011] In the second aspect, a gross error elimination device is also provided, including: an image matching module, used to perform image matching on the first image and the second image to obtain an image matching result, wherein the image matching result includes multiple matching point pairs and the local direction and local scale of each point in each matching point pair; a mapping module, used to map the local direction and local scale of each matching point pair to the attribute domain to obtain the global direction and global scale of each matching point pair; an unsupervised learning module, used to process the matching point pairs in the attribute domain in an unsupervised learning manner to obtain the global direction transformation parameters and the global scale transformation parameters between the images; a first screening module, used to process the matching point pairs in the attribute domain in an unsupervised learning manner to obtain the global direction transformation parameters and the global scale transformation parameters between the images based on the global direction transformation between the images. Parameters and the global scale transformation parameters are used to filter out a first matching point pair set from the matching point pairs in the attribute domain; an undirected graph construction module is used to construct an undirected graph according to the two-dimensional coordinates of each matching point pair in the first matching point pair set, and the two-dimensional coordinates are used to determine the connection relationship of the nodes in the undirected graph, and a node in the undirected graph is a matching point pair in the first matching point pair set; a second screening module is used to screen the nodes in the undirected graph based on the connection relationship between the nodes in the undirected graph to obtain a target node set, and the matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after the image matching results are eliminated of gross errors.
[0012] Optionally, in the attribute domain, the correct matching point pairs conform to the Gaussian distribution, and the unsupervised learning module is also used to construct a Gaussian mixture model based on the matching point pairs in the attribute domain. The Gaussian mixture model is an unsupervised learning model, and the Gaussian mixture model is used to calculate the probability density of each Gaussian distribution in the attribute domain; the parameters of the Gaussian mixture model are updated using the expectation maximization method; and the global direction transformation parameters and the global scale transformation parameters between images are determined based on the trained Gaussian mixture model.
[0013] Optionally, the unsupervised learning module is also used to obtain the probability density of each Gaussian distribution in the attribute domain based on the trained Gaussian mixture model; and determine the global direction transformation parameters and the global scale transformation parameters between images based on the mean of the Gaussian distribution with the largest probability density.
[0014] Optionally, the undirected graph construction module is further used to calculate the direction constraint error and the scale constraint error between the first node and the second node based on the two-dimensional coordinates of the first node and the second node, where the first node and the second node are any two different nodes in the undirected graph; when the direction constraint error between the first node and the second node is equal to the global direction transformation parameter, and the scale constraint error between the first node and the second node is equal to the global scale transformation parameter, the first node and the second node are connected by an edge in the undirected graph.
[0015] Optionally, the second screening module is also used to sort each node in the undirected graph according to the degree of the node, put the node with the largest degree into the first node set, and put other nodes in the undirected graph except the node with the largest degree into the second node set; extract nodes from the second node set in order of degree from large to small, and determine whether the extracted nodes and each node in the first node set meet the set node connection rules; if the extracted nodes meet the node connection rules, the extracted nodes are stored in the first node set; if the extracted nodes do not meet the node connection rules, the extracted nodes are discarded; after each node in the second node set is extracted, the first node set is the target node set.
[0016] Optionally, in the second screening module, the node connection rule is used to indicate that the extracted nodes are connected to all nodes in the first node set; or, the node connection rule is used to indicate that the extracted nodes are connected to A nodes in the first node set, where A is equal to k times B, k is the regularization coefficient, k is greater than or equal to 0.9, and B is the total number of nodes in the first node set.
[0017] In a third aspect, a computer device is also provided, comprising: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, thereby executing the gross error elimination method described in the above embodiment.
[0018] In a fourth aspect, a computer-readable storage medium is further provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor, thereby executing the gross error elimination method described in the above embodiment.
[0019] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the gross error elimination method described in the first aspect.
[0020] The beneficial effects of the technical solutions provided by the embodiments of the present disclosure include at least: In the embodiment of the present disclosure, the matching point pairs in the attribute domain are processed by adopting an unsupervised learning method to obtain the global direction transformation parameters and global scale transformation parameters between the images, so that the matching point pairs in the attribute domain can be preliminarily eliminated based on the calculated global direction transformation parameters and global scale transformation parameters to obtain a first set of matching point pairs. Then, an undirected graph is constructed by the two-dimensional coordinates of each matching point pair in the first set of matching point pairs, and then the target node set is obtained by screening the nodes in the undirected graph, thereby achieving further elimination of gross errors and effectively improving the accuracy of gross error elimination. Compared with the gross error elimination method in the related art, the entire process of the gross error elimination method in the embodiment of the present disclosure is a deterministic process, and there is no assumption process. Therefore, the gross error elimination method in the embodiment of the present disclosure is not affected by the gross error ratio, and the accuracy of gross error elimination is also high. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0022] Figure 1 A flow chart of a method for eliminating gross errors provided by an exemplary embodiment of the present disclosure is shown; Figure 2 A flowchart of a method for eliminating gross errors provided by another exemplary embodiment of the present disclosure is shown; Figure 3 It is a schematic diagram of the overall process of the gross error elimination method; Figure 4 It is a schematic diagram of screening the first matching point pair set; Figure 5 A schematic structural diagram of a gross error elimination device provided by an exemplary embodiment of the present disclosure is shown; Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0023] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present disclosure belongs. The words “first”, “second”, “third” and similar terms used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as “one” or “a” do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as “include” or “comprising” and similar terms mean that the elements or objects appearing before “include” or “comprising” cover the elements or objects listed after “include” or “comprising” and their equivalents, and do not exclude other elements or objects.
[0024] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.
[0025] Figure 1 A flowchart of a method for eliminating gross errors provided by an exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 1 , the method comprising: In step 101 , image matching is performed on a first image and a second image to obtain an image matching result.
[0026] The image matching results include multiple matching point pairs and the local direction and local scale of each point in each matching point pair.
[0027] Each matching point pair consists of a first matching point and a second matching point. The first matching point is a point in the first image, and the second matching point is a point in the second image. In the image matching results, the first and second matching points in the matching point pair are considered to be globally identical. In reality, image matching results can have gross errors, so the first and second matching points in the matching point pair are not necessarily globally identical.
[0028] Step 101 can be implemented using any image matching method in the related art. For example, when the first image and the second image are multimodal images, a multimodal image feature matching algorithm can be used to obtain the image matching result; when the first image and the second image are natural images, a SIFT (Scale-Invariant Feature Transform) algorithm can be used to obtain the image matching result.
[0029] In step 102, the local direction and local scale of each matching point pair are mapped into the attribute domain to obtain the global direction and global scale of each matching point pair.
[0030] Local orientation and local scale refer to orientation and scale within a local coordinate system. A local coordinate system is specific to an image; each image has a local coordinate system, and these systems are not necessarily identical. For example, the first image has a local coordinate system, and the second image also has a local coordinate system, but these systems are not necessarily identical. Global orientation and global scale refer to the entire image; there is only one global coordinate system.
[0031] The attribute domain is the domain of orientation and scale in the global coordinate system. The attribute domain includes two coordinate axes: one for global orientation and the other for global scale. Step 102 essentially converts the local orientation and local scale of each matching point pair into global orientation and global scale.
[0032] For example, formula (1) is used to convert the local direction of the matching point pair into the global direction, and formula (2) is used to convert the local scale direction of the matching point pair into the global scale.
[0033] (1) (2) In formula (1) and formula (2), are the local orientation and local scale of the two points in the i-th matching point pair, where is the local scale of the first matching point in the i-th matching point pair, is the local direction of the first matching point in the i-th matching point pair, is the local scale of the second matching point in the i-th matching point pair, is the local direction of the second matching point in the i-th matching point pair. is the global scale of the i-th matching point pair, is the global direction of the i-th matching point pair. Indicates that the direction is normalized to the range of [0,360) degrees.
[0034] Then in the attribute domain, the coordinates of the i-th matching point pair can be expressed as .
[0035] In step 103, the matching point pairs in the attribute domain are processed by unsupervised learning to obtain global direction transformation parameters and global scale transformation parameters.
[0036] In step 104, based on the global direction transformation parameters and global scale transformation parameters between the images, a first set of matching point pairs is selected from the image matching results. Through step 103 to step 104 , preliminary gross error elimination can be achieved.
[0037] In step 105 , an undirected graph is constructed according to the two-dimensional coordinates of each matching point pair in the first matching point pair set.
[0038] The two-dimensional coordinates are used to determine the connection relationship of nodes in the undirected graph, and a node in the undirected graph is a matching point pair in the first matching point pair set.
[0039] In step 106 , based on the connection relationship between each node in the undirected graph, the nodes in the undirected graph are screened to obtain a target node set.
[0040] The matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after eliminating gross errors from the image matching results.
[0041] Through step 105 to step 106 , further gross error elimination can be achieved.
[0042] In the embodiment of the present disclosure, the matching point pairs in the attribute domain are processed by adopting an unsupervised learning method to obtain global direction transformation parameters and global scale transformation parameters, so that the matching point pairs in the attribute domain can be preliminarily eliminated based on the calculated global direction transformation parameters and global scale transformation parameters to obtain a first set of matching point pairs. Then, an undirected graph is constructed by the two-dimensional coordinates of each matching point pair in the first set of matching point pairs, and then the target node set is obtained by screening the nodes in the undirected graph, thereby achieving further elimination of gross errors and effectively improving the accuracy of gross error elimination. Compared with the gross error elimination method in the related art, the entire process of the gross error elimination method in the embodiment of the present disclosure is a deterministic process, and there is no assumption process. Therefore, the gross error elimination method in the embodiment of the present disclosure is not affected by the gross error ratio, and the accuracy of gross error elimination is also high.
[0043] Figure 2 A flowchart of a method for eliminating gross errors provided by another exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 2 , the method comprising: In step 201 , image matching is performed on the first image and the second image to obtain an image matching result.
[0044] The image matching results include multiple matching point pairs and the local direction and local scale of each point in each matching point pair.
[0045] In step 202, the local direction and local scale of each matching point pair are mapped into the attribute domain to obtain the global direction and global scale of each matching point pair.
[0046] The relevant contents of step 201 to step 202 refer to the aforementioned steps 101 to step 102, and detailed description is omitted here.
[0047] Figure 3 It is a schematic diagram of the overall process of the gross error elimination method. Figure 3 Part (a) is a schematic diagram of step 201 to step 202. Figure 3 Part (b) is a schematic diagram of step 203 to step 204. Figure 3 Part (c) is a schematic diagram of step 205 to step 206.
[0048] In step 203, the matching point pairs in the attribute domain are processed by unsupervised learning to obtain global direction transformation parameters and global scale transformation parameters between images.
[0049] Correct matches in the attribute domain follow a Gaussian distribution (i.e., a normal distribution), while incorrect matches follow a uniform distribution. In this case, the goal of unsupervised learning is to find multiple Gaussian distributions in the attribute domain (i.e., correct matching point pairs) and determine the probability density of each Gaussian distribution.
[0050] Optionally, step 203 includes the following steps ac.
[0051] Step a: Construct a Gaussian mixture model based on matching point pairs in the attribute domain.
[0052] The Gaussian Mixture Model (GMM) is an unsupervised learning model that is used to calculate the probability density of each Gaussian distribution in the attribute domain.
[0053] Step b: update the parameters of the Gaussian mixture model using the expectation maximization method.
[0054] The Expectation Maximization (EM) method is an iterative algorithm commonly used to estimate the parameters of a Gaussian mixture model. Using EM, we can estimate the parameters of the Gaussian mixture model in step a, which include the mean and probability density of each Gaussian distribution.
[0055] There are many implementation methods of the Gaussian mixture model and the expectation-maximization method in related technologies, so detailed description is omitted here.
[0056] Step c: determining the global direction transformation parameters and global scale transformation parameters between images based on the trained Gaussian mixture model.
[0057] Optionally, step c includes the following two steps.
[0058] The first step is to obtain the probability density of each Gaussian distribution in the attribute domain based on the trained Gaussian mixture model.
[0059] After the Gaussian mixture model training is completed, the probability density and mean of each Gaussian distribution are known.
[0060] In the second step, the global direction transformation parameters and global scale transformation parameters between images are determined according to the mean of the Gaussian distribution with the largest probability density.
[0061] Here, the mean of each Gaussian distribution includes the mean of the global direction and the mean of the global scale between images. The global direction transformation parameter is the mean of the global direction in the mean of the Gaussian distribution with the largest probability density, and the global scale transformation parameter is the mean of the global scale in the mean of the Gaussian distribution with the largest probability density.
[0062] In step 204 , based on the global direction transformation parameters and the global scale transformation parameters between the images, a first set of matching point pairs is screened out from the matching point pairs in the attribute domain.
[0063] Optionally, step 204 can be expressed using formula (3).
[0064] (3) In formula (3), is the correctness indicator function of the i-th matching point pair, Taking 1 means that the i-th matching point pair is correct, A value of 0 indicates that the i-th matching point pair is wrong. is the global scale transformation parameter, is the global orientation transformation parameter. is the maximum deviation in the scale dimension, is the maximum deviation in the directional dimension, and is customized. For example, Take 1, Take 15. The meanings of other parameters in formula (3) are the same as those in formula (1) and formula (2), and their detailed description is omitted here.
[0065] The essence of formula (3) is that if the global scale of the i-th matching point pair is close to the global scale transformation parameter, and the global direction of the i-th matching point pair is close to the global direction transformation parameter, it means that the i-th matching point pair may be the correct matching point pair. According to formula (3), the ,all The matching point pairs with a value of 1 constitute the first matching point pair set.
[0066] Figure 4 This is a schematic diagram of screening the first matching point pair set. Figure 4 , Figure 4 There are three cases: point Des1 and point Des2 are a matching point pair; point Des1 and point Des2 are a matching point pair; point Des1 and point Des3 are a matching point pair. Take 1, Take 15, the global scale transformation parameter Take 1, global direction transformation parameter Take 90 as an example. For the case where point Des1 and point Des2 are a matching point pair, the global direction and global scale of the matching point pair are calculated according to formula (1) and formula (2): , substituting the global direction and global scale into formula (3) reveals that it does not satisfy , that is, the direction does not satisfy, which means that point Des1 and point Des2 are a matching point pair. is 0, that is, the matching point pair does not belong to the first matching point pair set.
[0067] Similarly, for the case where point Des1 and point Des2 are a matching point pair, the global direction and global scale of the matching point pair are calculated in the same way: , substitute the global direction and global scale into formula (3) to find the matching point pair is 1, that is, the matching point pair belongs to the first matching point pair set.
[0068] For the case where point Des1 and point Des3 are a matching point pair, the global direction and global scale of the matching point pair are calculated in the same way: , substituting the global direction and global scale into formula (3) reveals that it does not satisfy , that is, the scale is not satisfied, which means that point Des1 and point Des3 are a matching point pair. is 0, that is, the matching point pair does not belong to the first matching point pair set.
[0069] In step 205 , an undirected graph is constructed according to the two-dimensional coordinates of each matching point pair in the first matching point pair set.
[0070] The two-dimensional coordinates are used to determine the connection relationship of nodes in the undirected graph, and a node in the undirected graph is a matching point pair in the first matching point pair set.
[0071] A node in the undirected graph is a matching point pair, and an edge in the undirected graph is used to connect two nodes. In this case, step 205 includes the following steps de.
[0072] Step d: Calculate the direction constraint error and scale constraint error between the first node and the second node according to the two-dimensional coordinates of the first node and the second node.
[0073] The first node and the second node are any two different nodes in the undirected graph.
[0074] For example, the direction constraint error between the first node and the second node in the undirected graph is calculated using formula (4), and the scale constraint error between the first node and the second node in the undirected graph is calculated using formula (5).
[0075] (4) (5) In formula (4) and formula (5), is the direction constraint error between the first and second nodes in the undirected graph, is the scale-constrained error between the first and second nodes in the undirected graph, represents the two-dimensional coordinate vector of the first node in the undirected graph, Represents the two-dimensional coordinate vector of the second node in the undirected graph. Here, one node corresponds to a matching point pair in the first matching node set. For example, the two-dimensional coordinates of the matching point pair corresponding to the first node can be expressed as ,in is the coordinate of the first matching point in the matching point pair corresponding to the first node, is the coordinate of the second matching point in the matching point pair corresponding to the first node. Based on the two-dimensional coordinates of the matching point pair corresponding to the first node, the two-dimensional coordinate vector of the first node in the undirected graph can be calculated Similarly, based on the two-dimensional coordinates of the matching point pair corresponding to the second node, the two-dimensional coordinate vector of the second node in the undirected graph can also be calculated. Indicates finding the two-dimensional coordinate vector The angle between the coordinate axis X and Indicates finding the two-dimensional coordinate vector The angle between the coordinate axis and the X direction. Indicates normalizing the data to [0,360).
[0076] Based on the above formulas (4) and (5), the direction constraint error and scale constraint error between any two nodes in the undirected graph can be calculated.
[0077] In step e, when the direction constraint error between the first node and the second node satisfies the global direction transformation parameter and the scale constraint error between the first node and the second node satisfies the global scale transformation parameter, the first node and the second node are connected by an edge in the undirected graph.
[0078] The essence of step e is to determine whether the first and second nodes are connected in the undirected graph based on the orientation constraint error and scale constraint error between the first and second nodes. Here, the orientation constraint error is equivalent to the global orientation between the two nodes calculated based on the two-dimensional coordinates, and the scale constraint error is equivalent to the global scale between the two nodes calculated based on the two-dimensional coordinates. Only when the orientation constraint error between the first and second nodes satisfies the global orientation transformation parameters and the scale constraint error between the first and second nodes satisfies the global scale transformation parameters, does it mean that the first and second nodes conform to the global coordinate system? In this case, the first and second nodes can be connected by an edge in the undirected graph.
[0079] In step 206 , based on the connection relationship between each node in the undirected graph, the nodes in the undirected graph are screened to obtain a target node set.
[0080] The matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after eliminating gross errors from the image matching results.
[0081] Optionally, step 206 includes the following five steps.
[0082] In the first step, each node in the undirected graph is sorted according to the degree of the node, the node with the largest degree is placed in the first node set, and the other nodes in the undirected graph except the node with the largest degree are placed in the second node set.
[0083] The degree of a node is the number of edges connected to the node. The more edges connected to a node, the greater the degree of the node.
[0084] In the second step, nodes are extracted from the second node set in descending order of degree, and it is determined whether the extracted nodes and each node in the first node set meet the set node connection rules.
[0085] In the embodiment of the present disclosure, the node connection rules include the following two cases.
[0086] In the first case, the extracted nodes are all connected to the nodes in the first node set.
[0087] Sometimes, there are too few nodes that meet the first condition, resulting in fewer nodes that meet the node connection rule. Therefore, a regularization term can be introduced, which allows the extracted nodes to be disconnected from a certain proportion of nodes in the first node set. This situation is the node connection rule in the second condition.
[0088] In the second case, the extracted nodes are all connected to A nodes in the first node set, where A is equal to k times B, k is a regularization coefficient, k is greater than or equal to 0.9, and B is the total number of nodes in the first node set.
[0089] When selecting node connection rules, you can choose the first case or the second case according to your needs.
[0090] If the extracted nodes meet the node connection rules, jump to step 3; if the extracted nodes do not meet the node connection rules, jump to step 4.
[0091] The third step is to store the extracted nodes into the first node set.
[0092] The fourth step is to discard the extracted nodes.
[0093] The above steps 2 to 4 are executed in a loop until each node in the second node set is extracted, and then the process jumps to step 5.
[0094] In the fifth step, after each node in the second node set is extracted, the first node set becomes the target node set.
[0095] In the first node set determined by the first to fifth steps above, each node is fully connected (or only partially disconnected), and the connection between two nodes in the undirected graph indicates that the two nodes meet the global direction transformation parameters and the global scale transformation parameters. Therefore, the fully connected nodes in the first node set all meet the global direction transformation parameters and the global scale transformation parameters. It can be considered that the matching point pairs corresponding to these nodes are the correct matching point pairs, that is, the matching point pairs that have eliminated gross errors.
[0096] In the disclosed embodiment, rough elimination in the attribute domain (i.e., steps 203 to 204) is combined with precise elimination in the geometric domain (i.e., steps 205 to 206) to efficiently remove gross errors in the image matching results. First, the relative values of the matching point pairs in the attribute domain (direction and scale) are calculated, and the data distribution is analyzed using unsupervised learning techniques to extract the global direction transformation parameters and global scale transformation parameters between the images. Subsequently, based on the global direction transformation parameters and global scale transformation parameters between the images, gross errors that deviate significantly from the global trend are eliminated in the attribute domain, completing the first round of screening and obtaining the first set of matching point pairs. Finally, an undirected graph structure is constructed, and the matching point set is optimized based on the undirected graph structure to further accurately remove residual gross errors. This method effectively improves matching accuracy and robustness in high-noise and high-deformation matching scenarios, providing a robust solution for complex image matching tasks.
[0097] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.
[0098] Figure 5 FIG2 shows a schematic diagram of the structure of a gross error elimination device provided by an exemplary embodiment of the present disclosure. Figure 5The gross error elimination device 500 includes: an image matching module 501, a mapping module 502, an unsupervised learning module 503, a first screening module 504, an undirected graph construction module 505 and a second screening module 506.
[0099] The image matching module 501 is used to perform image matching on the first image and the second image to obtain an image matching result. The image matching result includes multiple matching point pairs and the local direction and local scale of each point in each matching point pair.
[0100] The mapping module 502 is used to map the local direction and local scale of each matching point pair into the attribute domain to obtain the global direction and global scale of each matching point pair.
[0101] The unsupervised learning module 503 is used to process the matching point pairs in the attribute domain in an unsupervised learning manner to obtain global direction transformation parameters and global scale transformation parameters between images.
[0102] The first screening module 504 is configured to screen out a first set of matching point pairs from the matching point pairs in the attribute domain based on the global direction transformation parameters and the global scale transformation parameters between the images.
[0103] The undirected graph construction module 505 is used to construct an undirected graph based on the two-dimensional coordinates of each matching point pair in the first matching point pair set. The two-dimensional coordinates are used to determine the connection relationship between the nodes in the undirected graph. A node in the undirected graph is a matching point pair in the first matching point pair set.
[0104] The second screening module 506 is used to screen the nodes in the undirected graph based on the connection relationship between each node in the undirected graph to obtain a target node set. The matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after eliminating gross errors in the image matching results.
[0105] Optionally, in the attribute domain, the correct matching point pairs conform to the Gaussian distribution, and the unsupervised learning module 503 is also used to construct a Gaussian mixture model based on the matching point pairs in the attribute domain. The Gaussian mixture model is an unsupervised learning model. The Gaussian mixture model is used to calculate the probability density of each Gaussian distribution in the attribute domain; the parameters of the Gaussian mixture model are updated using the expectation maximization method; and the global direction transformation parameters and global scale transformation parameters between images are determined based on the trained Gaussian mixture model.
[0106] Optionally, the unsupervised learning module 503 is further used to obtain the probability density of each Gaussian distribution in the attribute domain based on the trained Gaussian mixture model; and determine the global direction transformation parameters and global scale transformation parameters between images based on the mean of the Gaussian distribution with the largest probability density.
[0107] Optionally, the undirected graph construction module 505 is also used to calculate the direction constraint error and the scale constraint error between the first node and the second node based on the two-dimensional coordinates of the first node and the second node, where the first node and the second node are any two different nodes in the undirected graph; when the direction constraint error between the first node and the second node is equal to the global direction transformation parameter, and the scale constraint error between the first node and the second node is equal to the global scale transformation parameter, the first node and the second node are connected by an edge in the undirected graph.
[0108] Optionally, the second screening module 506 is also used to sort each node in the undirected graph according to the degree of the node, put the node with the largest degree into the first node set, and put other nodes in the undirected graph except the node with the largest degree into the second node set; extract nodes from the second node set in order of degree from large to small, and determine whether the extracted nodes and each node in the first node set meet the set node connection rules; if the extracted nodes meet the node connection rules, the extracted nodes are stored in the first node set; if the extracted nodes do not meet the node connection rules, the extracted nodes are discarded; after each node in the second node set is extracted, the first node set is the target node set.
[0109] Optionally, in the second screening module 506, the node connection rule is used to indicate that the extracted nodes are connected to all nodes in the first node set; or, the node connection rule is used to indicate that the extracted nodes are connected to A nodes in the first node set, where A is equal to k times B, k is the regularization coefficient, k is greater than or equal to 0.9, and B is the total number of nodes in the first node set.
[0110] It should be noted that the gross error elimination device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate gross error elimination. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the gross error elimination device provided in the above embodiment and the gross error elimination method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0111] The division of modules in the embodiments of the present disclosure is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present disclosure may be integrated into a single processor, exist physically as separate modules, or be integrated into a single module. The integrated modules may be implemented in either hardware or software functional modules.
[0112] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a personal computer, mobile phone, or communication device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.
[0113] Figure 6 Schematic diagram of the structure of the computer device provided by the embodiment of the present disclosure. Figure 6 As shown, the computer device 600 includes a processor 601 and a memory 602 .
[0114] Processor 601 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 601 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0115] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is executed by the processor 601 to implement the gross error elimination method provided in the embodiments of the present disclosure.
[0116] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or fewer components than shown in the figure, or combine some components, or adopt a different arrangement of components.
[0117] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a computer device, the computer device is enabled to execute the gross error elimination method provided in the embodiments of the present disclosure.
[0118] The embodiments of the present disclosure further provide a computer program product, including a computer program / instruction, which implements the gross error elimination method provided in the embodiments of the present disclosure when the computer program / instruction is executed by a processor.
[0119] The above description is merely an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.
Claims
1. A method for eliminating gross errors, characterized in that: The method comprises: Performing image matching on the first image and the second image to obtain an image matching result, wherein the image matching result includes a plurality of matching point pairs and a local direction and a local scale of each point in each matching point pair; Mapping the local direction and local scale of each matching point pair into an attribute domain to obtain the global direction and global scale of each matching point pair; The matching point pairs in the attribute domain are processed by unsupervised learning to obtain the global direction transformation parameters and global scale transformation parameters between images. Based on the global direction transformation parameter and the global scale transformation parameter between images, a first matching point pair set is selected from the matching point pairs in the attribute domain; constructing an undirected graph based on the two-dimensional coordinates of each matching point pair in the first matching point pair set, wherein the two-dimensional coordinates are used to determine a connection relationship between nodes in the undirected graph, and a node in the undirected graph is a matching point pair in the first matching point pair set; Based on the connection relationship between each node in the undirected graph, the nodes in the undirected graph are screened to obtain a target node set, and the matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after eliminating gross errors from the image matching results.
2. The method according to claim 1, characterized in that In the attribute domain, the correct matching point pairs conform to the Gaussian distribution. The method of using unsupervised learning to process the matching point pairs in the attribute domain to obtain global direction transformation parameters and global scale transformation parameters includes: Constructing a Gaussian mixture model based on matching point pairs in the attribute domain, wherein the Gaussian mixture model is an unsupervised learning model and is used to calculate the probability density of each Gaussian distribution in the attribute domain; Updating the parameters of the Gaussian mixture model using the expectation maximization method; The global direction transformation parameters and the global scale transformation parameters between images are determined based on the trained Gaussian mixture model.
3. The method according to claim 2, characterized in that The determining the global direction transformation parameters and the global scale transformation parameters between images based on the trained Gaussian mixture model includes: Obtaining the probability density of each Gaussian distribution in the attribute domain according to the trained Gaussian mixture model; The global direction transformation parameter and the global scale transformation parameter between images are determined according to the mean value of the Gaussian distribution with the largest probability density.
4. The method according to any one of claims 1 to 3, characterized in that The step of constructing an undirected graph according to the two-dimensional coordinates of each matching point pair in the first matching point pair set includes: Calculating a direction constraint error and a scale constraint error between a first node and a second node according to two-dimensional coordinates of the first node and the second node, where the first node and the second node are any two different nodes in the undirected graph; When a direction constraint error between the first node and the second node is equal to the global direction transformation parameter, and a scale constraint error between the first node and the second node is equal to the global scale transformation parameter, the first node and the second node are connected by an edge in the undirected graph.
5. The method according to any one of claims 1 to 3, characterized in that The step of screening the nodes in the undirected graph based on the connection relationship between the nodes in the undirected graph to obtain a target node set includes: Sort each node in the undirected graph according to the node's degree, put the node with the largest degree into a first node set, and put the other nodes in the undirected graph except the node with the largest degree into a second node set; Extracting nodes from the second node set in descending order of degree, and determining whether the extracted nodes and each node in the first node set satisfy a set node connection rule; If the extracted node meets the node connection rule, the extracted node is stored in the first node set; If the extracted nodes do not meet the node connection rules, the extracted nodes will be discarded; After each node in the second node set is extracted, the first node set becomes the target node set.
6. The method according to claim 5, characterized in that The node connection rule is used to indicate that the extracted nodes are all connected to the nodes in the first node set; or, The node connection rule is used to indicate that the extracted node is connected to A nodes in the first node set, where A is equal to k times B, k is a regularization coefficient, k is greater than or equal to 0.9, and B is the total number of nodes in the first node set.
7. A gross error elimination device, characterized in that: The device comprises: An image matching module is used to perform image matching on the first image and the second image to obtain an image matching result, wherein the image matching result includes a plurality of matching point pairs and a local direction and a local scale of each point in each matching point pair; A mapping module, configured to map the local direction and local scale of each matching point pair into an attribute domain to obtain the global direction and global scale of each matching point pair; An unsupervised learning module is used to process matching point pairs in the attribute domain using an unsupervised learning method to obtain global direction transformation parameters and global scale transformation parameters; A first screening module is configured to screen out a first set of matching point pairs from the matching point pairs in the attribute domain based on the global direction transformation parameter and the global scale transformation parameter between images; an undirected graph construction module, configured to construct an undirected graph based on the two-dimensional coordinates of each matching point pair in the first matching point pair set, wherein the two-dimensional coordinates are used to determine a connection relationship between nodes in the undirected graph, and a node in the undirected graph is a matching point pair in the first matching point pair set; The second screening module is used to screen the nodes in the undirected graph based on the connection relationship between each node in the undirected graph to obtain a target node set, and the matching point pairs indicated by the nodes in the target node set are the matching point pairs obtained after eliminating gross errors in the image matching results.
8. A computer device, characterized in that: The computer device includes: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.