Method and System for Geometric Modeling and Error Evaluation of Part Digital Twin Based on Point Cloud

Abstract

The present invention discloses a method and system for digital twin geometric modeling and error evaluation of parts based on point cloud. A KDTree data structure of the point cloud data Pc of the part to be measured is established, and after noise reduction processing, the point cloud Po2 is obtained; the KDTree search method is used to process the point cloud Po2; the measured wireframe point cloud Po3 and the model wireframe point cloud Pm1 are obtained by using the point cloud feature extraction method; the coordinate transformation matrix To3→m from the wireframe point cloud to the three-dimensional model point cloud is obtained by using the ICP method; the original point cloud is subjected to coordinate transformation and registered again to obtain the original point cloud Pof; according to the three-dimensional model, the parametric equation of the feature surface of the part to be measured and the three-dimensional coordinates of the point cloud are obtained, and based on the original point cloud Pof, the distance between each point in the point cloud and the feature surface is judged to segment the feature surface to be measured; the segmented point cloud is fitted by the RANSAC algorithm to obtain the parametric expression of the part feature, and the manufacturing error of the surface to be measured is obtained, realizing error evaluation. The present invention improves the accuracy and efficiency of error detection and is applicable to error detection in the part processing stage and precision analysis in the product assembly stage.

Description

Technical Field

[0001] The present invention belongs to the technical field of advanced manufacturing and automation, and particularly relates to a method and system for digital twin geometric modeling and error evaluation of parts based on point cloud. Background Art

[0002] Part error evaluation is a key link to ensure product quality and the safe operation of equipment, and is widely used in the product assembly process. With the continuous improvement of product complexity, the impact of assembly on product quality is gradually increasing. Therefore, it will be increasingly important to accurately control the assembly quality during the assembly process. An important prerequisite for regulating the assembly quality of products is to evaluate the actual errors of parts. Based on this, various means can be used to control and improve the assembly quality of products. Traditional error evaluation methods such as coordinate measuring machines based on manual operation and template gauges generally detect common geometric errors such as flatness, coaxiality, and perpendicularity, and it is difficult to meet the diversity and timeliness requirements in complex production and assembly scenarios. Moreover, the inspection of parts is often independent of the processing and assembly links, and it is impossible to perform on-demand inspection, resulting in a time lag. Therefore, an instant and comprehensive error evaluation method is needed to provide a guiding basis for assembly quality control.

[0003] Analyzing part manufacturing errors using digital inspection technology mainly involves obtaining the point cloud of the part surface and then using a certain analysis method to analyze various errors of the part. Currently, the main methods for manufacturing error analysis are three-dimensional reconstruction method and model registration method. There will be reconstruction errors in three-dimensional reconstruction, so it is not suitable for error analysis. General model registration methods can only evaluate the distance between the overall point cloud and the standard model or the distance from a certain point to the standard model, and cannot reflect the shape and position errors of the part. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for digital twin geometric modeling and error evaluation of parts based on point cloud to solve the technical problems of low efficiency and accuracy of digital inspection.

[0005] The present invention adopts the following technical solutions:

[0006] A method and system for digital twin geometric modeling and error evaluation of parts based on point cloud, comprising the following steps:

[0007] S1. Obtain comprehensive point cloud data P of the part to be measured c ;

[0008] S2. Establish the KDTree data structure of the point cloud data P obtained in step S1 c , and use KDTree to traverse each point in the point cloud data P c to obtain the point cloud P o2 after noise reduction processing;

[0009] S3. Use the KDTree search method in the rasterized point cloud space to process the point cloud P obtained in step S2 o2 ;

[0010] S4. Use the point cloud feature extraction method to obtain the measured wireframe point cloud P o2 by extracting the edges of the point cloud P after being processed in step S3 o3 and the model wireframe point cloud P m1 ;

[0011] S5. Use the ICP method to register the measured wireframe point cloud P o3 and the model wireframe point cloud P m1 obtained in step S4 to obtain the coordinate transformation matrix T o3→m from the wireframe point cloud to the 3D model point cloud; perform coordinate transformation on the original point cloud according to the coordinate transformation matrix and register it with the 3D model point cloud again to obtain the registered original point cloud P of ;

[0012] S6. Obtain the parametric equation of the feature surface of the part to be measured and the 3D coordinates of the point cloud based on the 3D model. Based on the original point cloud P of obtained in step S5, judge the distance between each point in the point cloud and the feature surface, and segment the feature surface to be measured; fit the segmented point cloud through the RANSAC algorithm to obtain the parametric expression of the part feature and obtain the manufacturing error of the surface to be measured, realizing error evaluation.

[0013] Specifically, in step S1, for the same type of part to be measured, each time point cloud acquisition is performed, the pose of the part is relatively fixed with respect to the position of the coordinate system of the point cloud acquisition device itself. The part to be detected is grabbed by the manipulator, and the point cloud data P c of the part to be measured is obtained by rotating and moving different poses.

[0014] Specifically, in step S2, using KDTree to traverse each point specifically means:

[0015] S201. For a point p i in the point cloud, determine the position where the point p i is located through the established KDTree structure, and search for all points among its adjacent root nodes and child nodes to establish the nearest neighbor point set H;

[0016] S202. Calculate the Euclidean distance between all points in the nearest neighbor point set H obtained in step S201 and the point p i , and find the K points closest to the point p i to establish the K-neighborhood of the point p i .

[0017] Specifically, in step S3, the KDTree search method for rasterizing the point cloud space is as follows:

[0018] S301. Calculate the envelope range of the point cloud T in space to obtain min x , max x , min y , max y , min z , max z , and the length, width, and height: abs x , abs y , abs z ;

[0019] S302. Expand the envelope range of the point cloud T in space with an expansion factor of scale. Then, evenly divide each dimension of the expanded point cloud space into 10 parts and calculate the length, width, and height of each small grid: grid x , grid y , grid z ;

[0020] S303. Calculate the reference points for each dimension and determine which grid the point is located in according to the coordinates of the point;

[0021] S304. Traverse the point cloud T and rasterize the point cloud according to the coordinates of each point;

[0022] S305. Build a KDTree search structure for the point cloud in each grid. When searching for the corresponding point later, traverse the KDTree structure at the specified grid according to the coordinates of the corresponding point.

[0023] Furthermore, in step S303, the reference points for each dimension are specifically:

[0024] benchmark x =(max x +min x -abs x ×scale) / 2

[0025] benchmark y =(max y +min y -abs y ×scale) / 2

[0026] benchmark z =(max z +min z -abs z ×scale) / 2.

[0027] Specifically, step S4 is as follows:

[0028] S401. Construct the KDTree of point cloud P o2 ;

[0029] S402. Traverse point cloud P o2 , and search for the nearest 4 points;

[0030] S403. Determine the parameters of the plane based on the last three points obtained in step S402, and calculate the distance from the search point to this plane;

[0031] S404. If the distance obtained in step S403 is greater than the given threshold, it is considered a wireframe point. The set of wireframe points searched forms the measured wireframe point cloud P o3 and the model wireframe point cloud P m1 .

[0032] Specifically, in step S5, using the ICP method to register the measured wireframe point cloud P o3 obtained in step S4 and the model wireframe point cloud P m1 is as follows:

[0033] S501. Select P and Q as the initial point clouds for iterative registration;

[0034] S502. Traverse each point in P, and use the KDTree structure to find the point with the closest Euclidean distance in Q as the corresponding point of this point, forming a set of corresponding points;

[0035] S503. Use the singular value decomposition method through the set of corresponding points to find a coordinate transformation matrix R and T to minimize the objective function;

[0036] S504. According to the calculated R and T, perform a coordinate transformation on the source point cloud, and determine a new set of corresponding points based on the closest Euclidean distance between the transformed point cloud and the target point cloud;

[0037] S505. Repeat steps S502 to S504 until the error function is less than the given threshold or the number of iterations reaches the specified upper limit.

[0038] Furthermore, in step S503, the objective function f(R,T) is:

[0039]

[0040] where k is the number of measured point clouds, q i is the discrete point of the measured point cloud, R is the rotation transformation matrix from the measured point cloud to the model point cloud, p i is the discrete point of the model point cloud, and T is the translation transformation matrix from the measured point cloud to the model point cloud.

[0041] Specifically, in step S6, the RANSAC fitting algorithm is used to fit the segmented point cloud, and the parametric expression form of the part error is specifically as follows:

[0042] S601. Randomly select n points from the point cloud set P to be fitted to form a subset S of P, and initialize the model M using the subset S, where n is the minimum number of samples required for the feature M to be fitted;

[0043] S602. Compare the data in the remaining samples with the error of the model M one by one, and take the samples within the allowable range as valid data. The valid data forms a consensus set S*;

[0044] S603. When the number of data in the consensus set S* obtained in step S602 is greater than or equal to n, obtain the correct model parameters, and then recalculate the model parameters using methods such as least squares;

[0045] S604. Repeat steps S601 to S603. After a certain number of samplings are completed and the consensus set S* is determined, select the parameters obtained from the largest consensus set after sampling as the parameter description error evaluation for describing the feature point cloud.

[0046] In a second aspect, an embodiment of the present invention provides a part digital twin geometric modeling and error evaluation system based on point cloud, including:

[0047] A data module that obtains comprehensive point cloud data P of the part to be measured c ;

[0048] A noise reduction module that builds a KDTree data structure for the point cloud data P obtained by the data module c and traverses each point in the point cloud data P using the KDTree. After noise reduction processing, the point cloud P is obtained c ; o2 ;

[0049] A processing module that processes the point cloud P obtained by the noise reduction module using the KDTree search method for rasterizing the point cloud space o2 ;

[0050] An extraction module that uses a point cloud feature extraction method to obtain the measured wireframe point cloud P o2 by extracting the edge of the point cloud P after being processed by the processing module o3 and the model wireframe point cloud P m1 ;

[0051] A registration module that uses the ICP method to register the measured wireframe point cloud P o3 and the model wireframe point cloud P m1 obtained by the extraction module to obtain the coordinate transformation matrix T from the wireframe point cloud to the three-dimensional model point cloud o3→m; Coordinate-transform the original point cloud according to the coordinate transformation matrix, and register it with the 3D model point cloud again to obtain the registered original point cloud P of ;

[0052] An evaluation module, which obtains the parametric equation of the feature surface of the part to be measured and the 3D coordinates of the point cloud based on the 3D model, and based on the original point cloud P obtained by the registration module of Judge the distance between each point in the point cloud and the feature surface, and segment the feature surface to be measured; Fit the segmented point cloud through the RANSAC algorithm to obtain a parametric expression of the part feature, and obtain the manufacturing error of the surface to be measured to realize error evaluation.

[0053] Compared with the prior art, the present invention has at least the following beneficial effects:

[0054] The method and system for digital twin geometric modeling and error evaluation of parts based on point cloud of the present invention uses point cloud scanning to obtain part error information, which can provide a more comprehensive, efficient and accurate data basis for subsequent data processing; Based on the implementation scenario of the present invention, environmental noise is quickly and accurately removed by judging noise according to the Euclidean distance from discrete points to the model; Considering the large amount of point cloud data obtained and the need for a large amount of neighborhood information of points in subsequent data processing, the KDTree search method for rasterizing the point cloud space is used, thereby improving the calculation efficiency; In order to obtain a more accurate registration result, two registrations are carried out successively, from coarse to fine; Finally, the RANSAC algorithm is used to separate part features, reducing the influence of edge noise.

[0055] Furthermore, error evaluation based on point cloud can obtain more comprehensive data information compared with other methods. Massive point cloud data can describe the part morphology, and 3D scanning has higher efficiency and the accuracy can also meet the requirements.

[0056] Furthermore, the present invention uses a manipulator-assisted method to obtain point cloud data. Therefore, the rough pose of the part in the camera coordinate system can be obtained. Based on this, homogeneous transformation can be performed on the scanned data, and then the KDTree search method is used to judge the Euclidean distance from each point to the model point cloud, and accordingly judge whether it is noise. The time complexity of this method is O(n), and it can effectively filter environmental noise.

[0057] Furthermore, since the amount of point cloud data obtained is about one million, and the subsequent registration and segmentation processing both use the search for the nearest point extensively, the KDTree search method is improved, and the point cloud space is rasterized to improve the search efficiency.

[0058] Furthermore, many mechanical components are rotating bodies and have many small features such as holes and grooves. Direct registration will not be able to identify these features, thus affecting the registration effect. Therefore, feature extraction is first performed on the point cloud model to increase the proportion of these features in the whole, and then registration is carried out to obtain a more accurate pose; finally, based on this, the original point cloud is registered, and accurate matching can be performed through a large amount of data in the original point cloud to obtain an accurate registration result.

[0059] Furthermore, in order to increase the proportion of the small features of the part in the point cloud, feature extraction needs to be performed on it. Here, according to the distribution characteristics of the points on the wireframe, it is judged by determining the distance from the search point to the local plane of the surrounding neighborhood. If it is a wireframe point, the distance is relatively large, and if it is a plane or surface point, the distance approaches 0. Therefore, the wireframe point cloud is obtained.

[0060] Furthermore, ICP is an effective method for accurate registration. For the high-precision requirements needed for part error evaluation, the present invention performs registration based on the ICP method.

[0061] Furthermore, the ICP registration takes the minimum Euclidean distance between the corresponding points of the target point cloud and the source point cloud as the objective function, and does not perform registration based on the high-dimensional features of the point cloud data. Therefore, the complete measurement data can be retained, and thus an accurate registration result can be obtained.

[0062] Furthermore, when performing point cloud feature segmentation, due to the existence of manufacturing errors, if the segmentation threshold is too small, over-segmentation will occur, and if it is too large, there will be more noise. Therefore, a larger threshold is first used for segmentation to obtain complete data, and then the RANSAC method is used for secondary fitting. This method can distinguish inliers and outliers (noise), so the fitting effect is not affected by noise, and a more accurate result can be obtained.

[0063] It can be understood that the beneficial effects of the second aspect above can be referred to the relevant descriptions in the first aspect above, and will not be elaborated here.

[0064] In summary, the method of the present invention has higher efficiency and accuracy; the nearest point search efficiency is improved through the grid-based point cloud search strategy; the proportion of the small features of the part in the point cloud is increased through the wireframe point cloud feature extraction method, and a relatively accurate pose is obtained based on registration, and then the original point cloud is registered to obtain a more accurate result; according to the spatial pose of the part features after registration, rough segmentation is directly performed, with high efficiency, and then the RANSAC scheme is used for fitting to avoid the influence of noise, so as to obtain an accurate error evaluation result.

[0065] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0066] Figure 1 is the workflow diagram of the present invention

[0067] Figure 2 is the schematic diagram of parametric segmentation of cylindrical features;

[0068] Figure 3 is the schematic diagram of parametric segmentation of conical features;

[0069] Figure 4 is the schematic diagram of the instance object;

[0070] Figure 5 is the schematic diagram of the physical point cloud;

[0071] Figure 6 is the schematic diagram of the initial pose;

[0072] Figure 7 is the schematic diagram of the standard ICP registration result;

[0073] Figure 8 is the schematic diagram of the improved KDTree registration result;

[0074] Figure 9 is the schematic diagram of the point cloud reduction + improved KDTree registration result;

[0075] Figure 10 is the schematic diagram of the point cloud segmentation result;

[0076] Figure 11 is the schematic diagram of the actual manufacturing error of the part measured by the coordinate measuring machine;

[0077] Figure 12 is the schematic diagram of the experimental result;

[0078] Figure 13 is the layout diagram of the device;

[0079] Figure 14 is the schematic diagram of the industrial robot. Specific embodiments

[0080] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0081] In the description of the present invention, it should be understood that the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0082] It should also be understood that the terms used in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the description of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0083] It should be further understood that the term " / and" used in the description of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. For example, A and / or B can represent: the case where A exists alone, the case where A and B exist simultaneously, and the case where B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0084] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present invention to describe preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0085] Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detected (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)".

[0086] Schematic diagrams of various structures according to the disclosed embodiments of the present invention are shown in the drawings. These figures are not drawn to scale, where for the purpose of clear expression, some details are enlarged and some details may be omitted. The shapes of the various regions and layers shown in the figures and their relative sizes and positional relationships are only exemplary, and may actually deviate due to manufacturing tolerances or technical limitations. Those skilled in the art can additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0087] The present invention provides a method and system for digital twin geometric modeling and error assessment of parts based on point cloud. By automatically acquiring part point cloud data and establishing a three-dimensional model in a specified coordinate system, it provides a basis for subsequent point cloud preprocessing and registration. A wireframe-based point cloud reduction method is proposed to obtain the wireframe features of the point cloud, thereby improving the efficiency and accuracy of registration. A KDTree search method for rasterized point cloud space is proposed to improve the speed of finding corresponding points in ICP registration. The measured point cloud and model point cloud of the part are registered to obtain point cloud data in a unified coordinate system. Based on the known three-dimensional model coordinate system, the point cloud of the specified feature surface of the part is accurately segmented. The segmented point cloud is fitted to obtain a parametric expression of the part error.

[0088] Please refer to Figure 1 , a method and system for digital twin geometric modeling and error assessment of parts based on point cloud according to the present invention, comprising the following steps:

[0089] S1. Standardize the way of obtaining point cloud. The manipulator grabs the part to be detected, rotates and moves in different poses to obtain the comprehensive point cloud data P of the part to be detected c ;

[0090] For the same batch of identical parts, the pose of the manipulator when grabbing the part each time is fixed. Therefore, the spatial coordinates of the point cloud data taken by the laser scanner for the first time are basically the same. For the subsequent point cloud images taken at different poses of the part, they are automatically stitched by the point cloud stitching function of the 3D scanner with the first point cloud data as the reference. Therefore, the poses of the obtained point cloud data in space are basically the same. Then, through the camera coordinate CC of the laser scanner, the coordinate MEC of the end of the manipulator, and the origin coordinate POC of the part to be detected, the pose relationship of the origin coordinate of the part relative to the camera coordinate is calculated, and the point cloud data P c is transformed to the origin of the part three-dimensional model through coordinate transformation, thereby simplifying the complexity of subsequent registration and segmentation. The coordinate transformation equation is:

[0091] T c→o =T c→m ·T m→o (1-1)

[0092] wherein, T p→o is the coordinate transformation matrix from the camera coordinate system to the origin coordinate of the part three-dimensional model, T c→m is the coordinate transformation matrix from the camera coordinate system to the end coordinate system of the manipulator, and T m→o is the coordinate transformation matrix from the end coordinate system of the manipulator to the origin coordinate system of the part.

[0093] P o =P c ·T c→o (1-2)

[0094] Among them, P c is the point cloud data in the original camera coordinate system, and P o is the point cloud data in the part origin coordinate system.

[0095] S2. Establish the KDTree data structure for the point cloud data P c obtained in step S1, and use the KDTree to traverse each point in the point cloud data P c After noise reduction processing, the point cloud P o2 is obtained;

[0096] KDTree is a data structure used to divide the k-dimensional data space. Essentially, it is a binary tree and is widely used as the basis for dividing the three-dimensional data space. For three-dimensional point cloud data, each point contains (x, y, z) coordinate information. The dimension with the largest variance among these three dimensions is used as the specified dimension for data segmentation to construct the KDTree structure.

[0097] By using the KDTree to establish a clear topological relationship for the point cloud, the neighborhood search for the point cloud is no longer carried out for all points and comparing the Euclidean distance, but by finding the root node and child nodes, quickly finding K nearest points to form the K-neighborhood of the point, which greatly improves the point cloud search efficiency. The steps to search for the point cloud neighborhood using the KDTree are as follows:

[0098] S201. For a point p i in the point cloud, determine the position where the point p i is located through the established KDTree structure, and search for all points among its adjacent root nodes and child nodes to establish the nearest neighbor point set H;

[0099] S202. Calculate the Euclidean distance between all points in the nearest neighbor point set H and the point p i , and find the K nearest points from them to establish the K-neighborhood of the point p i . i

[0100] By using the KDTree to judge the distance from each point in P o to the surface of the three-dimensional model, the environmental noise points in the source point cloud can be initially filtered out, such as the robotic arm, the ground, the wall, etc. However, due to the spatial pose of P o and P m being only roughly the same, not all noise points can be accurately filtered out. Therefore, after the above operations, the preliminarily noise-reduced point cloud P o1 is obtained, and the point cloud P o1 is basically distributed near the point cloud of the three-dimensional model.

[0101] Since most of the noise points have been filtered out, the number of point clouds has decreased significantly. Then, the filtering method is used to accurately remove the outliers in P o1 to obtain a relatively intact point cloud P o2 . The steps of using the filtering method to remove outliers are as follows: First, for each point, form a point set consisting of the point and its N nearest points; assume that this point set conforms to a normal distribution, and calculate the mean and standard deviation of the point set; if the point is within the pre-set standard deviation range, for example, within one standard deviation, then retain the point, otherwise remove it.

[0102] S3. Propose a KDTree search method for rasterizing the point cloud space to process the point cloud P o2 obtained in step S2 to improve the speed of finding corresponding points in ICP registration;

[0103] When facing a large number of point clouds, ICP needs to find the nearest point in the KDTree structure constructed in the entire point cloud space during each iteration. In fact, most of the time of the ICP algorithm is spent on finding corresponding points. For the above-obtained source point cloud S and target point cloud T, the feature is that the poses of S and T in space are relatively close. Therefore, consider a method to improve the search efficiency of corresponding points, specifically as follows:

[0104] S301. Calculate the envelope range of the point cloud T in space, that is, obtain min x , max x , min y , max y , min z , max z , and its length, width, and height: abs x , abs y , abs z ;

[0105] S302. Expand the envelope range of the point cloud T in space, with an expansion factor of scale. Then, evenly divide each dimension of the expanded point cloud space into 10 parts, and calculate the length, width, and height of each small grid: grid x , grid y , grid z , where the purpose of the expansion factor is to make the search range of the point cloud space include the source point cloud S, and generally take 1.5 to 2;

[0106] S303. Calculate the reference points for each dimension to determine which grid the point is located in according to the coordinates of the point;

[0107] Specifically as follows:

[0108]

[0109] S304. Traverse the point cloud T and rasterize the point cloud according to the coordinates of each point;

[0110] The specific method of allocating points to the corresponding grids is as follows:

[0111]

[0112] where: grid i , grid j , grid k represents the index of the three-dimensional grid space of this point, p x , p y , p z is the coordinate of this point, and [] represents rounding the result.

[0113] S305. Construct a KDTree search structure for the point cloud in each grid. When searching for the corresponding point later, traverse the KDTree structure at this place according to the point coordinate to the specified grid.

[0114] S4. Propose a point cloud feature extraction method to reduce the number of point clouds and improve the ICP registration speed and accuracy by extracting the edges of the point cloud P o2 processed in step S3;

[0115] Generally, the point cloud registration process first performs rough registration of the point cloud, aiming to obtain a better initial pose for subsequent fine registration of the point cloud using the iterative closest point (ICP) method. However, in the technical solution of the present invention, P o2 obtained through the above steps already has a better initial pose, so the step of rough registration of the point cloud can be omitted.

[0116] The fine registration uses the iterative closest point algorithm (ICP algorithm). The ICP algorithm calculates the distance between the corresponding points of the source point cloud and the target point cloud, constructs a rotation and translation matrix, transforms the source point cloud, and calculates the mean square error after the transformation. If the mean square error meets the threshold condition, the algorithm ends. Otherwise, it continues to repeat the iteration until the error meets the threshold condition or the iteration times terminate. Therefore, the ICP algorithm has the following characteristics: the registration result has a relatively high accuracy; it has strict requirements for the initial matrix, and a poor initial matrix seriously affects the algorithm performance and may even cause the situation of local optimum.

[0117] The ICP algorithm is essentially an overall optimal algorithm. However, in a large number of dense point clouds, the tiny features of the measured part cannot be effectively recognized, and the features of the part are submerged in the overall optimum. Therefore, the present invention proposes a point cloud feature extraction method to reduce the number of point clouds by extracting the edges of the point cloud, increasing the proportion of the feature point cloud of the measured part in the whole, thereby improving the ICP registration speed and accuracy. The specific approach is as follows:

[0118] S401. Construct the KDTree of point cloud P o2 for nearest neighbor search;

[0119] S402. Traverse the point cloud to search for the nearest 4 points (the first point is the search point, so 4 nearest points need to be found);

[0120] S403. Determine the parameters of a plane based on the last three points and calculate the distance from the search point to this plane;

[0121] S404. If the distance obtained in step S403 is greater than the given threshold, it is considered a wireframe point, and the set of wireframe points obtained from the search constitutes the measured wireframe point cloud P o3 and the model wireframe point cloud P m1 .

[0122] S5. Use the ICP algorithm to register the wireframe point cloud P o3 and the point cloud P m1 obtained from the above steps, and obtain the coordinate transformation matrix T o3 from the wireframe point cloud P m1 to the 3D model point cloud P o3→m . According to the coordinate transformation matrix, perform coordinate transformation on the original point cloud, and register it with the 3D model point cloud again to obtain the registered original point cloud P of ;

[0123] For the source point cloud P and the target point cloud Q, the purpose of the ICP algorithm is to find a coordinate transformation matrix so that the source point cloud coincides with the target point cloud to the greatest extent in space after coordinate transformation. The evaluation criterion for the coincidence degree is the average of the Euclidean distances between the corresponding points of the two point clouds. The method for the ICP to determine the corresponding points of the two point clouds is to traverse all the points in the source point cloud and find the point with the closest Euclidean distance in the target point cloud as the corresponding point of this point. After determining the corresponding points of the two point clouds, a coordinate transformation matrix needs to be found to minimize the average distance between the corresponding points of the source point cloud after transformation and the target point cloud. The target error function is expressed by the formula as follows:

[0124]

[0125] where k is the number of measured point clouds, q i is the discrete point of the measured point cloud, R is the rotation transformation matrix from the measured point cloud to the model point cloud, p i is the discrete point of the model point cloud, and T is the translation transformation matrix from the measured point cloud to the model point cloud.

[0126] For the source point cloud P and the target point cloud Q, the process of using the classical ICP algorithm for registration is as follows:

[0127] S501. Select P and Q as the initial point clouds for iterative registration;

[0128] S502. Traverse each point in P, and use the KDTree structure to find the point with the closest Euclidean distance in Q as the corresponding point of this point, forming a set of corresponding points;

[0129] S503. Use the method of singular value decomposition (SVD) through the set of corresponding points to find a coordinate transformation matrix R and T to minimize the objective function (5-1);

[0130] S504. According to the calculated R and T, perform a coordinate transformation on the source point cloud, and determine a new set of corresponding points based on the closest Euclidean distance between the transformed point cloud and the target point cloud;

[0131] S505. Repeat steps S502 to S504 until the error function is less than a given threshold or the number of iterations reaches the specified upper limit.

[0132] According to the coordinate transformation matrix obtained above, perform a coordinate transformation on the point cloud P o2 and register it again with the 3D model point cloud P m Registration using the wireframe point cloud can maximize the coincidence of the features of the two point clouds and is not affected by other irrelevant point clouds. However, due to laser scanning errors and manufacturing errors of the parts, the accuracy of the wireframe features extracted from the point cloud is still insufficient. Therefore, it is necessary to register the original dense point cloud. Since a relatively accurate pose has been obtained through the registration of the refined point cloud, performing ICP registration on the dense point cloud again to obtain the point cloud P of can avoid falling into a local optimal solution, and the number of iterations is greatly reduced, ensuring both the registration efficiency and accuracy.

[0133] S6. According to the 3D model, obtain the parametric equation of the feature surface of the part to be measured and the 3D coordinates of the point cloud. Based on the original point cloud P obtained in step S5 of judge the distance between each point in the point cloud and the feature surface, and then judge whether the point belongs to the plane to be segmented. Fit and segment the point cloud through the RANSAC algorithm to obtain a parametric expression of the part feature, and obtain the manufacturing error of the surface to be measured, realizing error evaluation.

[0134] When establishing the 3D model point cloud, the information of its coordinate origin has been obtained. Therefore, the parametric equation of the part feature surface can be directly obtained. After the registration of the laser scanning point cloud is completed, its spatial pose is consistent with the 3D model point cloud. Therefore, the specified feature surface can be accurately segmented according to its distribution in space. The parametric equations and distance calculation methods of several common features are as follows:

[0135] (1) For any plane feature in space, its known parametric equation is as follows:

[0136] Ax + By + Cz + D = 0 (7 - 1)

[0137] The distance sum and constraint condition from the discrete points of the laser scanned point cloud to the plane are as follows:

[0138]

[0139] (2) For any cylindrical feature in space, such as Figure 2 as shown, given the centers O1(x1, y1, z1) and O2(x2, y2, z2) of the upper and lower circular surfaces of the cylindrical surface and the radius R, the distance judgment formula from any point p(x0, y0, z0) in space to the cylindrical surface is:

[0140]

[0141] Among them, the point c is the perpendicular point coordinate of the point p to the central axis of the cylinder, specifically:

[0142]

[0143] Among them:

[0144]

[0145] (3) For any conical surface feature in space, such as Figure 3 as shown, given the cone vertex A, the center O of the cone bottom surface, the cone bottom radius R, and a point p in space, the plane a1 can be determined through the three points A, O, and p; the plane a2 can be determined by passing through A and O and the plane a1 being perpendicular; the plane a3 can be determined by being perpendicular to a1 and having an angle θ with the plane a2 passing through the point A; then the distance from the point p to the conical surface is the distance from the point p to the plane a3.

[0146]

[0147] After point cloud segmentation, a series of points distributed on the part feature surface are obtained. In order to obtain its parametric representation form, it is necessary to fit the segmented point cloud. The RANSAC algorithm is different from the least squares algorithm. The least squares fits all data and finds a suitable result from it. However, when there are many abnormal data (noises) in the data, the fitting effect is not good; while the RANSAC algorithm fits most data. It divides the data into valid data and invalid data. The data that deviates from most data is invalid data, and finally only looks for the fitting result in the valid data. Even after certain processing, the point cloud obtained by laser scanning will contain a certain number of noise points. Therefore, the RANSAC algorithm has a better fitting effect. The process of the RANSAC algorithm is as follows:

[0148] S601. Randomly select n points (n is the minimum number of samples required for the feature M to be fitted) from the point cloud set P to be fitted to form a subset S of P, and initialize the model M.

[0149] S602. Compare the error between the data in the remaining samples and the model M one by one. If it is within the allowable range, it is considered valid data, and the valid data constitutes a consistency set S*.

[0150] S603. If the number of data in S* is greater than or equal to n, it is considered that the correct model parameters are obtained, and then the model parameters are recalculated using methods such as least squares.

[0151] S604. Repeat steps S601 to S603. After a certain number of samplings, if S* is not found, the algorithm fails; otherwise, the parameters obtained from the largest consistency set obtained after sampling are selected as the parameter description error evaluation for describing the feature point cloud.

[0152] Please refer to Figure 13 , a point cloud scanning device is designed, including a 3D scanner, an industrial robot, a zero component to be measured, etc.

[0153] There are generally two types of 3D scanners: handheld and fixed. For handheld scanners, usually the part to be measured is fixed, and the operator holds the 3D scanner and scans the part to be measured from various angles to obtain its point cloud data; for fixed scanners, usually the scanning instrument itself is fixed, and the part to be measured rotates on a turntable.

[0154] Please refer to Figure 14 , most industrial parts have the characteristics of few features and high symmetry, which makes point cloud registration often fall into local optimal solutions. To solve this problem, the present invention combines an industrial robot to standardize the point cloud scanning method and obtain the initial pose of the point cloud that meets the registration requirements. As Figure 11 shown in the device layout plan, this device is designed with the service goal of assembly accuracy analysis. Among them, when the 3D scanner is working, the fixed body does not move. The grasping robot grabs the part from the station of the part to be measured, then performs 3D point cloud measurement at the 3D measurement station, and then transfers the part to the assembly station to cooperate with the assembly robot to complete product assembly.

[0155] Specifically, during the 3D detection process, the grasping robot needs to rotate and translate different poses to obtain the point cloud data of the part from different perspectives, and finally form a complete point cloud model through the stitching of multiple point clouds.

[0156] In another embodiment of the present invention, a point cloud-based digital twin geometric modeling and error assessment system for parts is provided. This system can be used to implement the above-mentioned point cloud-based digital twin geometric modeling and error assessment method and system. Specifically, the point cloud-based digital twin geometric modeling and error assessment system for parts includes a data module, a noise reduction module, a processing module, an extraction module, a registration module, and an assessment module.

[0157] Among them, the data module obtains the comprehensive point cloud data P of the part to be measured c ;

[0158] The noise reduction module builds the KDTree data structure of the point cloud data P obtained by the data module c and traverses each point in the point cloud data P using the KDTree. After noise reduction processing, the point cloud P is obtained c ; o2 ;

[0159] The processing module processes the point cloud P obtained by the noise reduction module using the KDTree search method for rasterizing the point cloud space o2 ;

[0160] The extraction module uses the point cloud feature extraction method to obtain the measured wireframe point cloud P o2 and the model wireframe point cloud P o3 by extracting the edges of the point cloud P after being processed by the processing module m1 ;

[0161] The registration module uses the ICP method to register the measured wireframe point cloud P o3 and the model wireframe point cloud P m1 obtained by the extraction module to obtain the coordinate transformation matrix T from the wireframe point cloud to the three-dimensional model point cloud o3→m ; According to the coordinate transformation matrix, the coordinates of the original point cloud are transformed, and then registered with the three-dimensional model point cloud again to obtain the registered original point cloud P of ;

[0162] The assessment module obtains the parametric equation of the feature surface of the part to be measured and the three-dimensional coordinates of the point cloud based on the three-dimensional model. Based on the original point cloud P of obtained by the registration module, it judges the distance between each point in the point cloud and the feature surface, and segments the feature surface to be measured; the segmented point cloud is fitted by the RANSAC algorithm to obtain the parametric expression of the part feature, and the manufacturing error of the surface to be measured is obtained to achieve error assessment.

[0163] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. Components in the descriptions and illustrations of the embodiments of the present invention herein can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0164] Taking the nozzle fixing body part as an example object, as Figure 4 and Figure 5 shown. First, the surface point cloud data is collected by a 3D scanner. After preprocessing, a point cloud model is obtained, as Figure 6 shown; different registration methods are adopted, and the results are as Figure 7 , Figure 8 and Figure 9 shown. The registration accuracy and time consumption are shown in the following table. It can be seen that the method proposed by the present invention has obvious improvements in both registration accuracy and efficiency.

[0165] Table 1 Comparison of registration effects of different methods

[0166]

[0167] The registered model is segmented, and the result is as Figure 10 shown, and two key assembly features are segmented. The parametric expression form of the plane feature is:

[0168] Ax + By + Cz + D = 0

[0169] The fitting result of the two planes is:

[0170] (-0.001968 -0.999991 0.003627 0.0304)

[0171] (1.909e-7 -0.999991 0.003632 57.8493)

[0172] The nominal value of the upper surface of the fixing body is 58 mm. Therefore, the position error of the upper surface of the fixing body is -0.181 mm. The actual error of the part is detected by a coordinate measuring instrument, and the result is as Figure 12 shown. The experimental result error is -0.197 mm, and the calculation error of the present invention is 8.12%.

[0173] In summary, for the method and system for digital twin geometric modeling and error evaluation of parts based on point cloud of the present invention, the concept of digital twin is introduced. Based on the measured point cloud data of parts, a method and system for digital twin geometric modeling and error evaluation of parts based on point cloud are proposed. By means of point cloud model registration and segmentation, a digital twin geometric model of key features of parts with deviations is constructed, improving the accuracy and efficiency of error detection, and serving error detection in the part machining stage and precision analysis in the product assembly stage.

[0174] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more of the processes or multiple processes and / or blocks Figure 1 one or more of the blocks or multiple blocks.

[0176] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one or more of the processes or multiple processes and / or blocks Figure 1 one or more of the blocks or multiple blocks.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one or more of the processes or multiple processes and / or blocks Figure 1 one or more of the blocks or multiple blocks.

[0178] The above content is only for explaining the technical idea of the present invention and shall not be used to limit the protection scope of the present invention. Any modification made on the basis of the technical solution according to the technical idea proposed by the present invention falls within the protection scope of the claims of the present invention.

Claims

1. A method for digital twin geometric modeling and error evaluation of parts based on point cloud, characterized in that, Including the following steps: S1. Obtain the comprehensive point cloud data of the part to be measured ; S2. Establish the point cloud data obtained in step S1 to form a KDTree data structure, and use the KDTree to traverse each point in the point cloud data . After noise reduction processing, a point cloud is obtained ; S3. Process the point cloud obtained in step S2 using the KDTree search method for the rasterized point cloud space. The KDTree search method for the rasterized point cloud space is specifically as follows: S301. Calculate the envelope range of the point cloud T in space to obtain , , and the length, width, and height ; S302. Expand the envelope range of the extended point cloud T in space with an expansion ratio of scale. Then, evenly divide each dimension of the expanded point cloud space into 10 parts, and calculate the length, width, and height of each small grid: ; S303. Calculate the reference points of each dimension, and determine which grid the point is located in according to the coordinates of the point; S304. Traverse the point cloud T, and rasterize the point cloud according to the coordinates of each point; S305. Construct a KDTree search structure for the point cloud in each grid. When searching for the corresponding point subsequently, traverse the KDTree structure at the specified grid according to the coordinates of the corresponding point; S4. Adopt a point cloud feature extraction method to obtain the measured wireframe point cloud by extracting the edges of the point cloud processed in step S3 and the model wireframe point cloud ; ; S5. Use the ICP method to process the measured wireframe point cloud obtained in step S4 and the model wireframe point cloud for registration. Use the singular value decomposition method through the corresponding point sets to find a coordinate transformation matrix R and T to minimize the objective function. The objective function is as follows: Among them, is the number of measured point clouds, is the discrete points of the measured point cloud, is the rotation transformation matrix from the measured point cloud to the model point cloud, is the discrete points of the model point cloud, is the translation transformation matrix from the measured point cloud to the model point cloud, and the coordinate transformation matrix from the wireframe point cloud to the 3D model point cloud is obtained ; the coordinate transformation of the original point cloud is performed according to the coordinate transformation matrix, and it is registered with the 3D model point cloud again to obtain the registered original point cloud ; S6. Obtain the parametric equation of the feature surface of the part to be measured and the three-dimensional coordinates of the point cloud based on the three-dimensional model, and based on the original point cloud obtained in step S5 Judge the distance between each point in the point cloud and the feature surface, and segment the feature surface to be measured; Fit the segmented point cloud by the RANSAC algorithm to obtain the parametric expression of the part feature, obtain the manufacturing error of the surface to be measured, and realize error evaluation.

2. The method for digital twin geometric modeling and error assessment of parts based on point cloud according to claim 1, wherein In step S1, for the same part to be measured, each time point cloud acquisition is performed, the pose of the part is relatively fixed with respect to the position of the coordinate system of the point cloud acquisition device itself. The part to be detected is grasped by the manipulator, and different poses are rotated and moved to obtain the point cloud data of the part to be measured .

3. The method for digital twin geometric modeling and error evaluation of parts based on point cloud according to claim 1, wherein, In step S2, using the KDTree to traverse each point specifically is as follows: S201. For a point in the point cloud , determine the position of the point through the established KDTree structure, and search for all points among its adjacent root nodes and child nodes to establish a nearest neighbor point set ; S202. Calculate the Euclidean distances between all the points in the nearest neighbor point set obtained in step S201 and point , and find the points that are closest to point to establish the - neighborhood of point . Neighborhood.

4. The method for digital twin geometric modeling and error evaluation of parts based on point cloud according to claim 1, wherein, In step S303, the reference points of each dimension specifically are: 。 5. The method for digital twin geometric modeling and error evaluation of parts based on point cloud according to claim 1, wherein Step S4 specifically is: S401. Construct a point cloud KDTree; S402. Traverse the point cloud , and search for the 4 nearest points; S403. Determine the parameters of the plane according to the last three points obtained in step S402, and calculate the distance from the search point to the plane; S404. If the distance obtained in step S403 is greater than a given threshold, it is considered a wireframe point, and the set of wireframe points obtained by searching constitutes the measured wireframe point cloud and the model wireframe point cloud .

6. The method for digital twin geometric modeling and error evaluation of parts based on point cloud according to claim 1, characterized in that, In step S5, the measured wireframe point cloud obtained in step S4 and the model wireframe point cloud are registered specifically as follows: S501. Select P and Q as the initial point clouds for iterative registration; S502. Traverse each point in P, and use the KDTree structure to find the point with the closest Euclidean distance in Q as the corresponding point of this point to form a corresponding point set; S503. Use the singular value decomposition method through the corresponding point set to find a coordinate transformation matrix R and T to minimize the objective function; S504. According to the calculated R and T, perform a coordinate transformation on the source point cloud, and determine a new corresponding point set based on the closest Euclidean distance between the transformed point cloud and the target point cloud; S505. Repeat steps S502 to S504 until the error function is less than the given threshold or the number of iterations reaches the specified upper limit.

7. The method for geometric modeling and error evaluation of part digital twin based on point cloud according to claim 1, characterized in that In step S6, using the RANSAC fitting algorithm to fit the segmented point cloud, and obtaining the parametric expression form of the part error specifically is: S601. Randomly select n points from the point cloud set P to be fitted to form a subset S of P, and initialize the model M using the subset S, where n is the minimum number of samples required for the feature M to be fitted; S602. Compare the errors between the data in the remaining samples and the model M one by one, and regard the samples within the allowable range as valid data, and the valid data constitute a consistency set S*; S603. When the number of data in the consistency set S* obtained in step S602 is greater than or equal to n, obtain the correct model parameters, and then recalculate the model parameters using methods such as least squares; S604. Repeat steps S601 to S603. After a certain number of samplings are completed and the consistency set S* is determined, select the parameters obtained from the largest consistency set after sampling as the parameter description error evaluation for describing the feature point cloud.

8. A part digital twin geometric modeling and error evaluation system based on point cloud, characterized in that, Including: Data module, obtaining comprehensive point cloud data of the part to be measured ; Noise reduction module, establishing the point cloud data obtained by the data module of the KDTree data structure, using the KDTree to traverse the point cloud data in each point, and obtaining the point cloud after noise reduction processing ; The processing module processes the point cloud obtained by the noise reduction module by using the KDTree search method for rasterizing the point cloud space The KDTree search method for rasterizing the point cloud space is specifically as follows: Calculate the envelope range of the point cloud T in space , , and the length, width, and height ; Expand the envelope range of the point cloud T in space with an expansion ratio of scale, and then evenly divide each dimension of the expanded point cloud space into 10 parts to calculate the length, width, and height of each small grid: ; Calculate the reference points of each dimension, and determine which grid the point is located in according to the coordinates of the point; Traverse the point cloud T, and rasterize the point cloud according to the coordinates of each point; Build a KDTree search structure for the point cloud in each grid. When searching for the corresponding point later, traverse the KDTree structure at the specified grid according to the coordinates of the corresponding point; The extraction module uses a point cloud feature extraction method to obtain the measured wireframe point cloud by extracting the edges of the point cloud processed by the extraction and processing module and the model wireframe point cloud ; ; The registration module uses the ICP method to perform registration on the measured wireframe point cloud obtained by the extraction module and the model wireframe point cloud to find a coordinate transformation matrix R and T through the corresponding point set using the singular value decomposition method to minimize the objective function. The objective function is as follows: Among them, is the number of measured point clouds,[ is the discrete points of the measured point cloud,[ is the rotation transformation matrix from the measured point cloud to the model point cloud,[ is the discrete points of the model point cloud,[ is the translation transformation matrix from the measured point cloud to the model point cloud, and obtain the coordinate transformation matrix from the wireframe point cloud to the 3D model point cloud[ ; perform coordinate transformation on the original point cloud according to the coordinate transformation matrix, and register it with the 3D model point cloud again to obtain the registered original point cloud[ ; An evaluation module, which obtains the parametric equation of the feature surface of the part to be measured and the three-dimensional coordinates of the point cloud based on the three-dimensional model, and based on the original point cloud obtained by the registration module Judges the distance between each point in the point cloud and the feature surface, and segments the feature surface to be measured; fits the segmented point cloud through the RANSAC algorithm to obtain a parametric expression of the part feature, obtains the manufacturing error of the surface to be measured, and realizes error evaluation.

Images