Mismatching mark point correction method and system based on improved RANSAC (Random Sample Consensus) algorithm

Abstract

The invention belongs to the technical field of computer vision and three-dimensional image processing, and particularly discloses a mismatching mark point correction method and system based on an improved RANSAC algorithm. According to the method, congruent triangle criteria are introduced into the RANSAC algorithm, and potential matching point pairs conforming to geometric consistency are screened. Only when the potential matching point pairs meet the geometric consistency, the transformation matrixes R and T are solved, that is, mismatching points existing in the potential matching point pairs are effectively eliminated, so that on the premise that the point cloud splicing precision and the success rate are not changed, wrong calculation is reduced, the point cloud registration efficiency and robustness are improved, and the point cloud registration accuracy is improved. The method is suitable for various application scenes such as industrial part measurement, medical modeling, cultural relic digital protection and virtual reality.

Description

Technical Field

[0001] This application belongs to the field of computer vision and 3D image processing technology, and more specifically, relates to a method and system for correcting mismatched markers based on an improved RANSAC algorithm. Background Technology

[0002] 3D reconstruction technology can acquire the complete 3D shape of an object's surface with high precision and efficiency, and has been widely used in various fields such as cultural relic protection, industrial component measurement, medical examination, and virtual reality. However, in practical applications, due to the limited field of view of hardware measurement equipment and the problem of occlusion between objects, a single measurement often cannot obtain complete point cloud data of the measured object. Therefore, it is necessary to stitch point cloud data with overlapping areas from multiple perspectives into the same coordinate system. During the point cloud stitching process, due to the limitations of the object's material and the measurement perspective, it is often necessary to introduce a marker-based method for auxiliary stitching. This method relies on the Euclidean distance invariance between marker points. By finding geometrically consistent marker point pairs between different perspectives, the calculated transformation matrix allows the point clouds from different perspectives to be stitched into the same coordinate system.

[0003] Lei Yuzhen et al., based on constructing potential matching point pairs using the Euclidean distance matrix, introduced the random sample consensus (RANSAC) algorithm to randomly select three matching point pairs from the potential matching point pairs for transformation matrix calculation. This method can effectively remove mismatches caused by noise and equipment measurement accuracy, and can improve the accuracy and stability of point cloud stitching to a certain extent.

[0004] Although the RANSAC algorithm can effectively remove incorrect matching point pairs, it may fail to find the correct matching point pair before the algorithm ends when the number of iterations is small. This is especially true when measuring objects with large areas, where the number of potential matching point pairs increases due to the increased number of marker points, and a small number of iterations may lead to splicing failure. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application aims to provide a method and system for correcting mismatched markers based on an improved RANSAC algorithm, which addresses the problem of splicing failure caused by an increase in potential matching point pairs and a limited number of iterations.

[0006] To achieve the above objectives, in a first aspect, this application provides a method for correcting mismatched markers based on an improved RANSAC algorithm, comprising: (1) In the potential matching point pairs set under the two perspectives A and B, three pairs of potential matching points are randomly selected respectively. The three potential matching points from the same set form a triangle. The geometric criterion of congruent triangles is used to check the geometric consistency of the two triangles. If the check is successful, proceed to step (2). Otherwise, skip the current iteration and proceed to step (1) to start the next round of iteration. (2) Calculate the rotation and translation matrices between the point clouds from viewpoint A and viewpoint B; (3) Use rotation and translation matrices to transform the point cloud from view B to view A. Calculate the nearest neighbor Euclidean distance between each point in the transformed point cloud and the point cloud from view A. If the nearest neighbor Euclidean distance is less than the set distance error threshold, the point is considered an interior point; otherwise, it is an exterior point. (4) Count the number of interior points in this iteration; (5) Repeat steps (1)-(4) until the maximum number of iterations is reached; (6) Determine the maximum number of interior points in all iterations, and recalculate the rotation matrix and translation matrix based on the corresponding set of interior points.

[0007] Preferably, in step (1), the geometric support of each potential matching point pair is first determined, and then weighted random sampling is performed based on the geometric support. The higher the geometric support of the potential matching point pair, the higher its probability of being selected.

[0008] Preferably, the geometric support of a potential matching point pair is the number of other point pairs in the matching point pair set that can together form a congruent triangle criterion, and the sampling probability of a point pair is the ratio of its geometric support to the sum of the geometric supports of all point pairs.

[0009] Preferably, in step (1), the geometric criterion of congruent triangles is used to perform geometric consistency verification on the two triangles, as follows: if the length difference of the corresponding sides of the two triangles is less than the distance error threshold, the verification is successful; otherwise, the verification is unsuccessful.

[0010] Preferably, the distance error threshold ranges from [0.1mm, 0.3mm].

[0011] Preferably, a K-dimensional spatial tree is used to calculate the transformed point cloud. Each point in the cloud and the point cloud from viewpoint A The nearest neighbor Euclidean distance between them.

[0012] Preferably, it is used for multi-view point cloud data stitching in the 3D reconstruction process.

[0013] Secondly, this application provides a mismatch marker correction system based on an improved RANSAC algorithm, including a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions; the one or more processors call the computer instructions to cause the system to perform the method as described in the first aspect.

[0014] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect.

[0015] It is understood that the beneficial effects of the second and third aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0016] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application provides a method for correcting mismatched marker points based on an improved RANSAC algorithm. The method introduces a congruent triangle criterion into the RANSAC algorithm to screen potential matching point pairs that meet geometric consistency. Only when a potential matching point pair satisfies geometric consistency will the transformation matrices R and T be solved. This effectively eliminates mismatched points from the potential matching point pairs, thereby reducing erroneous calculations and improving the efficiency and robustness of point cloud registration while maintaining the accuracy and success rate of point cloud stitching. This method is applicable to various scenarios such as industrial component measurement, medical modeling, digital preservation of cultural relics, and virtual reality. Attached Figure Description

[0017] Figure 1 This is a flowchart of a mismatch marker correction method based on an improved RANSAC algorithm provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the experimental measurement system provided in the embodiments of this application.

[0019] Figure 3 This is a schematic diagram of the Venus physical model provided in the embodiments of this application.

[0020] Figure 4 These are schematic diagrams of point cloud data from different perspectives provided in the embodiments of this application, wherein (a) corresponds to point cloud data from perspective A and (b) corresponds to point cloud data from perspective B.

[0021] Figure 5These are schematic diagrams of point cloud registration results provided by different methods in the embodiments of this application, wherein (a) corresponds to the distance matrix method, (b) corresponds to the original RANSAC method, and (c) corresponds to the method proposed in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0024] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0025] In this application, the term "electrical connection" can refer to a direct circuit connection or a signal transmission via a communication protocol.

[0026] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0028] For ease of understanding, the English abbreviations and related technical terms used in the embodiments of this application will be explained and described below.

[0029] The basic principle of point cloud-assisted stitching based on marker points is described below.

[0030] First, distinctive markers need to be affixed to the surface of the object to be measured. The spatial positions of these markers under different measurement viewpoints conform to the Euclidean distance invariance principle. Based on this principle, assume that viewpoints A and B are a pair of point cloud viewpoints to be stitched together with overlapping markers, and the set of three-dimensional coordinates of the markers under viewpoint A is as follows: , , , For set The three-dimensional coordinates of the marker point. The set of three-dimensional coordinates of the marker point from view B is as follows: , , , For set The three-dimensional coordinates of the marker point.

[0031] To solve for the transformation matrix between point clouds from viewpoint A and viewpoint B , At least three pairs of non-collinear markers that correspond in two different viewpoints are required; otherwise, the transformation matrix cannot be uniquely determined. , .gather The marker points in the model are not one-to-one potential matching point pairs. Based on the Euclidean distance invariance of marker points in 3D space, and considering the systematic measurement errors in marker point coordinates during 3D reconstruction, it is generally stipulated that: when the Euclidean distance error between marker points is less than a certain threshold... ( When the distances are equal, they can be considered as equal. Based on this, the three-dimensional coordinates of the marker points from the point cloud perspectives A and B are obtained. According to the Euclidean distance calculation formula:

[0032] It can be obtained The Euclidean distance between any two marker points in the matrix is ​​obtained by calculating the Euclidean distance matrix. :

[0033] Similarly, it can also be obtained The Euclidean distance between any two points in the matrix is ​​obtained by calculating the Euclidean distance matrix. .

[0034] After obtaining the Euclidean distance matrix, Each row in Each row in the table is compared, provided the difference in Euclidean distance is not greater than the error threshold. Therefore, it is assumed that Euclidean distances are equal. For example, consider... The first row distance value and The distance values ​​in the fifth row are compared. Within the allowable error threshold, if the number of equal distance values ​​is greater than or equal to 3 (considering the self-point), it is considered... and This is a potential pair of matching points. By repeating the above steps, we can filter out a set of potential matching point pairs from the 3D set of marker points viewed from both A and B perspectives. and ,in, , They are a subset of and It is a potential matching point pair.

[0035] Analysis reveals two aspects of mismatching in the marker mismatching process based on the RANSAC algorithm. When obtaining potential matching point pairs using the Euclidean distance matrix method, mismatches occur: Firstly, if markers are arranged manually and are equidistant or otherwise irrationally distributed on the object surface, it may be difficult to effectively distinguish between different markers, leading to incorrect matching. Secondly, the calculation of the three-dimensional coordinates of markers may be biased due to factors such as system noise and ambient light interference from the surface structured light measurement equipment, resulting in errors in the calculation of the Euclidean distance matrix and causing mismatches. The existence of these mismatched points reduces the accuracy of solving the transformation matrices R and T, ultimately affecting the overall quality of point cloud stitching.

[0036] The embodiments of this application are described below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, this application provides a method for correcting mismatched markers based on an improved RANSAC algorithm, including: Step (1) The set of potential matching point pairs from the perspectives of A and B and In the process, three pairs of potential matching points are randomly selected. The three potential matching points from the same set form a triangle. The geometric criterion of congruent triangles is used to check the geometric consistency of the two triangles. If the check is successful, proceed to step (2). Otherwise, skip the current iteration and proceed to step (1) to start the next round of iteration.

[0038] Preferably, in step (1), the geometric support of each potential matching point pair is first determined, and then weighted random sampling is performed based on the geometric support. The higher the geometric support of the potential matching point pair, the higher its probability of being selected.

[0039] Preferably, the geometric support of a potential matching point pair is the number of other point pairs in the matching point pair set that can together form a congruent triangle criterion, and the sampling probability of a point pair is the ratio of its geometric support to the sum of the geometric supports of all point pairs.

[0040] It should be noted that geometric support refers to the number of congruent triangles that a potential matching point pair can form with other point pairs. This metric reflects the degree of consistency of the point pairs within the global geometric topology. Specifically, for a potential matching point pair... Iterate through all other potential matching pairs. If two pairs of points can form triangles satisfying the congruence criterion from two different perspectives, then the geometric support is incremented by 1. Based on the geometric support, a weighted sampling probability is defined; specifically, for point pairs... The sampling probability is the ratio of its geometric support to the sum of the geometric supports of all point pairs. By using weighted sampling, geometrically more consistent and reliable point pairs have a higher probability of being selected, thereby reducing the probability of sampling incorrect point pair combinations and reducing the number of iterations required to achieve successful registration.

[0041] Preferably, the geometric criterion of congruent triangles is used to perform geometric consistency verification on the two triangles, as follows: if the length difference of the corresponding sides of the two triangles is less than the distance error threshold, the verification is successful; otherwise, the verification is unsuccessful and the current iteration is skipped.

[0042] Preferably, the distance error threshold ranges from [0.1mm, 0.3mm], and can be adjusted according to actual conditions.

[0043] Step (2) Calculate the rotation matrix and translation matrix between the point clouds from view A and view B.

[0044]

[0045] That is, satisfying: Point cloud from view A Point cloud as the target point cloud, point cloud from view B. For source point cloud, For rotation matrix, It is a translation matrix.

[0046] This application uses the unit quaternion method to solve for rotation and translation matrices, but also supports other methods for solving them.

[0047] Step (3) Use rotation and translation matrices to transform the point cloud from view B. Transform the view to A and calculate the transformed point cloud. Each point in the cloud and the point cloud from viewpoint A The nearest neighbor Euclidean distance between points is used to determine if a point is an interior point if the nearest neighbor Euclidean distance is less than a set distance error threshold; otherwise, it is an exterior point.

[0048]

[0049] Preferably, a K-dimensional spatial tree is used to calculate the transformed point cloud. Each point in the cloud and the point cloud from viewpoint A The nearest neighbor Euclidean distance between them.

[0050] The implementation of the K-nearest neighbor algorithm requires considering how to quickly search for the K nearest neighbors. A K-dimensional tree (KD-tree) is a data structure that facilitates fast retrieval of data in a K-dimensional space. The specific method is as follows: First, select a point cloud... As a reference point cloud, a KD tree is constructed based on it. Secondly, for the point cloud... Each point in Initialize a variable `best_dist` to infinity to store the closest distance found so far, and also initialize a variable `best_point` to... A point in the tree is used to store the currently found nearest neighbor. Then, the search process is executed: 1) Starting from the root node: Starting from the root node of the KD-tree, search downwards along the tree, and at each node, based on the current point... 1) Based on the coordinate relationship between the split point and the current axis, select either the left or right subtree to continue the search; 2) Update the nearest point: when a leaf node is reached, calculate the current point. The Euclidean distance to the midpoint of the leaf node; if this distance is less than best_dist, then update best_dist and best_point; 3) Backtracking check: During backtracking, check if the other subtree of the current node might contain a closer point. Specifically, if... If the distance to the current node's splitting hyperplane is less than best_dist, then the search needs to proceed to the other subtree; 4) Calculate the point and points The Euclidean distance. Then, for the point cloud. Each point in Repeat the nearest neighbor search process described above to finally obtain the point. In point cloud The nearest neighbor and its distance.

[0051] Step (4) Count the number of interior points in this iteration.

[0052] Step (5) Repeat steps (1)-(4) until the maximum number of iterations is reached.

[0053] Step (6) Determine the maximum number of interior points in all iterations, and recalculate the rotation and translation matrices based on the corresponding set of interior points to optimize the accuracy of point cloud stitching.

[0054] Preferably, the above-mentioned mismatched marker correction method based on the improved RANSAC algorithm is applied to multi-view point cloud data stitching in the 3D reconstruction process.

[0055] Example To verify the effectiveness of the method in this application, this embodiment constructs a hardware system for three-dimensional imaging of surface structured light based on binocular vision, and uses this system to collect experimental data.

[0056] like Figure 2 As shown, the experimental system consists of two industrial CMOS cameras, an industrial projector, a customized bracket, and a data transmission cable. The single measurement error of the system is about 0.03 mm, the single measurement area is about 600 mm × 400 mm, and the distance between the experimental system and the object being measured is about 0.49 m.

[0057] like Figure 3 As shown, the experimental subject is a Venus model with affixed markers. These markers are circular dots with a black background and gray surface that are manually pasted onto the object's surface. The markers should be placed on a relatively flat area of ​​the object being tested.

[0058] First, use Figure 2 The experimental system shown is for Figure 3 Point cloud data was collected from the front of the Venus model with marked points, i.e., viewpoint A. Then, the Venus model was rotated by a certain angle, and point cloud data was collected from viewpoint B, resulting in point cloud data from views A and B, as shown below. Figure 4 As shown.

[0059] To verify the advantages of the proposed method in terms of stitching success rate and error reduction, this application collected point cloud data from the Venus model across the entire field of view and quantitatively compared it with two existing methods, the distance matrix method and RANSAC, through experiments. The experimental results are as follows: Figure 5 As shown.

[0060] As shown in Table 1, this embodiment compares the performance of different algorithms in terms of stitching success rate, stitching time, and MSE (Mean Squared Error). While the distance matrix method requires a very short stitching time, its stitching success rate is poor. However, for RANSAC and the method described in this application, it can be seen that under the same number of iterations, the stitching success rate and MSE are the same for both methods. The average stitching time for the RANSAC method is 1.2053s, while the method described in this application only requires 0.1474s, a reduction of approximately 88%. That is, in the full-view point cloud stitching experiment on the Venus model, the stability and stitching efficiency of this method in a real-world measurement environment are verified.

[0061] Table 1 Comparative Experiments of Venus Model

[0062] Overall, this application provides a stable and efficient solution to the point cloud stitching problem. Furthermore, due to the improved efficiency, the application of this algorithm in real-time point cloud stitching and large-scene 3D reconstruction can be further explored in the future.

[0063] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0064] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions stored in the memory to execute the methods in the above embodiments.

[0065] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0066] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0067] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0068] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0069] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0070] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0071] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0072] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for correcting mis-matching landmark points based on an improved RANSAC algorithm, characterized in that, The method comprises the following steps: (1) randomly selecting 3 pairs of potential matching points from the set of potential matching point pairs under two views of A and B, 3 potential matching points from the same set forming a triangle, and performing geometric consistency checking on the two triangles by using the geometric criterion of congruent triangles, if the checking is successful, going to step (2), otherwise, skipping the current iteration and going to step (1) to start the next iteration; (2) calculating the rotation matrix and the translation matrix between the point clouds under the views of A and B; (3) using the rotation matrix and the translation matrix to transform the point cloud under the view of B to the view of A, calculating the nearest neighbor Euclidean distance between each point in the transformed point cloud and the point cloud under the view of A, if the nearest neighbor Euclidean distance is less than a set distance error threshold, considering that the point is an inner point, otherwise, it is an outer point; (4) counting the number of inner points in the current iteration; (5) repeating steps (1)-(4) until the maximum number of iterations is reached; (6) determining the maximum number of inner points in all iterations, and recalculating the rotation matrix and the translation matrix according to the inner point set corresponding thereto.

2. The false matching landmark correction method of claim 1, wherein, In step (1), the geometric support degree of each potential matching point pair is first determined, and then weighted random sampling is performed based on the geometric support degree, and the higher the geometric support degree of the potential matching point pair, the higher the probability of being selected.

3. The false matching landmark correction method of claim 2, wherein, The geometric support degree of a potential matching point pair is the number of other point pairs that can form a triangle satisfying the criterion of congruent triangles together with the potential matching point pair, and the sampling probability of a point pair is the ratio of its geometric support degree to the sum of the geometric support degrees of all point pairs.

4. The false matching landmark correction method of claim 1, wherein, In step (1), the geometric consistency checking of the two triangles by using the geometric criterion of congruent triangles is as follows: if the length difference of the corresponding edges of the two triangles is less than a distance error threshold, the checking is successful, otherwise, the checking is unsuccessful.

5. The false matching landmark correction method of claim 4, wherein, The distance error threshold has a value range of [0.1mm, 0.3mm].

6. The false matching landmark correction method of claim 1, wherein, A K-dimensional space tree is adopted to calculate the nearest neighbor Euclidean distance between each point and the point cloud under the A view angle. between each point and the point cloud under the A view angle.​ 7. The mis-matched landmark correction method according to any one of claims 1 to 6, wherein, The method is applied to multi-view point cloud data splicing in the process of three-dimensional reconstruction.

8. A false matching landmark point correction system based on an improved RANSAC algorithm, characterized in that, The system comprises a memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, the computer program codes comprising computer instructions; The one or more processors invoke the computer instructions to enable the system to perform the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The instructions, when executed on an electronic device, enable the electronic device to perform the method of any one of claims 1-7.

Images