Part virtual assembly method based on three-dimensional point cloud

Abstract

The invention provides a part virtual assembly method based on a three-dimensional point cloud. Comprising the following steps: 1, acquiring surface point cloud data of a measured piece; 2, registering the point cloud to a design model of the part; 3, selecting a section according to the position of the design model, extracting contour points of the point cloud on the section, and calculating part manufacturing deviation according to contour information; 4, planning an optimal assembly sequence with the minimum overall deviation according to the manufacturing deviation of each part; and 5, for the part point cloud under the optimal assembly sequence, extracting a section contour vertical to an assembly surface, calculating the distance between part boundary points, and outputting the assembly clearance of the part. According to the method, the overall and local features of the part surface point cloud are mainly utilized to realize part virtual assembly and deviation detection, the automation degree is high, the application range is wide, good stability and repeatability are achieved, the part assembly result can be predicted, assembly can be guided, and the assembly quality of actual parts is effectively improved.

Description

Technical Field

[0001] The present invention relates to a method for detecting the quality of parts production, and specifically to the field of three-dimensional point cloud data acquisition and processing, and in particular to grating fringe three-dimensional scanning, point cloud registration, point cloud segmentation, feature fitting, and point cloud resampling. Background Art

[0002] During the parts production and processing process, due to the limitations of the manufacturing process, the actual parts produced will always have certain errors relative to the design model. When the manufacturing accuracy requirements of the parts are high, or the parts need to be further assembled according to the design model, it is necessary to measure and model the actual parts to provide size and position information for detecting the manufacturing accuracy of the workpiece or for subsequent processes.

[0003] The current virtual assembly technology mainly focuses on the design process of parts, using the CAD design model of the parts to plan the assembly sequence and path, as well as the prediction of the assembly results, which is not completely consistent with the actual assembly process of the parts; the technology of virtual assembly using the 3D digital model of the physical parts mostly performs large-area feature fitting on the surface of the parts, losing the detailed information of the surface of the parts, resulting in rough planning of the assembly process and prediction of the assembly results. These virtual assembly technologies do not simulate the actual assembly process of parts to a high degree. Summary of the invention

[0004] In order to solve the problem that the virtual assembly has a low degree of simulation of the actual parts assembly process, the present invention discloses a parts virtual assembly method based on three-dimensional point cloud, which performs grating stripe three-dimensional scanning on the actual parts to obtain a dense point cloud of the part surface, and realizes virtual assembly and assembly quality inspection through registration, resampling and other methods, including the following steps:

[0005] (1) Perform three-dimensional measurement to obtain surface point cloud data of the measured object.

[0006] The projector of the 3D measurement system projects multi-frequency phase-shifted sinusoidal grating stripes at multiple brightnesses onto the surface of the part to be measured. Multiple stripes of different frequencies are selected at each brightness, and N vertical stripes with a phase difference of 2π / N are projected at each frequency. The left and right cameras of the 3D measurement system are triggered to take pictures each time the projection is performed. After the continuous phase map of the projected part surface is obtained through phase solution and phase expansion, the same-name point matching and 3D reconstruction are performed according to the polar line constraint relationship between the two cameras. After 3D measurement of multiple different positions on the part surface to obtain multiple pieces of point cloud data, the complete point cloud data of the surface of the measured part is obtained after point cloud splicing, fusion and filtering.

[0007] (2) Align the measured point cloud of the part to the design model.

[0008] The CAD design model is sampled to obtain the model point cloud data, and then the directed bounding boxes of the model point cloud and the measured point cloud are obtained through the three eigenvectors of the covariance matrix. The two directed bounding boxes are registered according to up to 24 different registration methods, and the cosine similarity under each registration relationship is calculated.

[0009]

[0010] Where R m is the rotation matrix, and is the unit vector sum of two point clouds, satisfying:

[0011]

[0012] In the formula and are the centroid coordinates of each point in the two point clouds. Cosine similarity can represent the similarity of point clouds in spatial distribution direction. The closer the spatial distribution directions of two point clouds are, the closer the cosine similarity is to 1. Select a registration method with the largest cosine similarity to obtain a rough registration result, and then use the point-to-surface iterative closest point ICP method to achieve precise registration of the measured point cloud and the model point cloud.

[0013] (3) Select the cross section according to the position of the design model, extract the contour points of the point cloud on the cross section, and calculate the part manufacturing deviation based on the contour information.

[0014] After the alignment of the measured point cloud and the design model is achieved, the section is selected according to the position of the design model, and the neighboring point set whose distance to the section is less than the threshold in the measured point cloud is extracted, and a kd-tree is established among them. For each point in the neighboring point set, the five points with the closest Euclidean distance to it are found, and these six points are used to perform local quadratic surface fitting based on the least squares method. On the intersection of the fitting surface and the section, the point with the same x-coordinate as the x-coordinate of the centroid of the six points is taken as a contour point of the point cloud on the section, and the root mean square error of the distance from all contour points to the theoretical contour is calculated to determine the size of the manufacturing deviation of the part.

[0015] (4) Plan the optimal assembly sequence with the smallest overall deviation based on the manufacturing deviation of each part.

[0016] Considering that two parts with large manufacturing deviations will generate large stress after being assembled together, affecting the stability and reliability of the overall structure, the assembly sequence that makes the manufacturing errors of adjacent parts very close is selected as the assembly sequence with the minimum stress after the overall structure is assembled. Introduce the error function:

[0017]

[0018] where δ iThe manufacturing error is the difference between the manufacturing errors of two adjacent parts on the cross-section near their respective assembly positions during assembly. The manufacturing error is the root mean square error of the distance from the cross-section contour point to the theoretical contour. All assembly sequences of the parts are traversed and the one with the smallest error function is selected as the optimal assembly sequence. The virtual assembly is realized by aligning the measured point cloud to the design model position in (2).

[0019] (5) For the part point cloud under the optimal assembly sequence, extract the cross-sectional contour perpendicular to the assembly surface, calculate the distance between the part boundary points, and output the assembly clearance of the parts.

[0020] The contour points of the part point cloud are extracted on the section perpendicular to the assembly surface. The five points closest to the assembly surface are extracted according to the design model. The center of gravity of these five points is calculated as the boundary point coordinates of the part. The size of the assembly gap or overlap is calculated based on the boundary point coordinates of the two parts.

[0021] The present invention mainly utilizes the overall and local features of the part surface point cloud to realize part virtual assembly and deviation detection, has a high degree of automation, a wide range of applications, good stability and repeatability, can predict part assembly results and guide assembly, and effectively improve the assembly quality of actual parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a virtual assembly method based on three-dimensional point cloud is provided in an embodiment.

[0023] Figure 2 A structural diagram of the bottom of a rocket tank provided in an embodiment.

[0024] Figure 3 A schematic diagram of the structure of a three-dimensional measurement system provided in an embodiment.

[0025] Figure 4 A schematic diagram of point cloud data of the rocket tank bottom melon slice parts provided in the embodiment.

[0026] Figure 5 A schematic diagram of a directed bounding box of a melon segment part point cloud provided in an embodiment.

[0027] Figure 6 A rough registration result diagram of the measured point cloud and the model point cloud of the melon slice part provided in the embodiment.

[0028] Figure 7 This is a diagram of the precise registration results of the measured point cloud and the model point cloud of the melon slice part provided in the embodiment.

[0029] Figure 8 A schematic diagram of the longitudinal section selection position provided for the embodiment.

[0030] Fig. 9A schematic diagram of the segmentation result of the neighboring points of the longitudinal section provided in the embodiment, wherein the highlighted part is the segmentation result.

[0031] Fig.10 A schematic diagram of the longitudinal cross-section contour point resampling results provided in the embodiment. DETAILED DESCRIPTION

[0032] In order to better understand the present invention, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0033] Embodiment, for the rocket tank bottom melon slice parts based on three-dimensional point cloud virtual assembly technology, the process refers to the attached Figure 1 The bottom of the rocket tank is made of 6 identical melon slice parts, each melon slice part is 900mm high. The structure is as shown in the attached Figure 2 The embodiment comprises the following steps:

[0034] (1) Perform three-dimensional measurement to obtain surface point cloud data of the measured object.

[0035] Refer to the attached Figure 3 A three-dimensional measurement system including a fringe projector and two cameras was built, in which the dual cameras had been calibrated and the internal and external parameters were known. The fringe projector projects three-frequency four-step fringe stripes at different brightnesses onto the surface of the melon slice part, that is, three groups of fringe stripes with projected fringe periods of 15, 16, and 17 pixels at each brightness, and each group of fringe stripes includes 4 fringe patterns with phase differences of π / 2. Each time the fringe stripes are projected, the dual cameras of the three-dimensional measurement system are synchronously triggered to take pictures, and images of different grayscales are obtained by setting a variety of different exposure times. All images of different grayscales are used to synthesize a grating fringe image with the highest contrast but not overexposed for each pixel, and the phase is solved and unfolded according to the fringe pattern to obtain the true phase distribution of the object surface, and then each pair of same-name points in the pictures taken by the left and right cameras is uniquely determined according to the epipolar geometry relationship of the binocular camera. Based on the matching relationship of the same-name points, the three-dimensional point cloud of a single field of view on the surface of the melon slice part can be reconstructed in three dimensions using the triangulation method. Then, by splicing, fusing and filtering the three-dimensional point clouds of each part of the surface of the melon slice part, the complete point cloud data of the surface of the melon slice part can be obtained. The schematic diagram of the melon slice point cloud is shown in the attached figure. Figure 4 .

[0036] (2) Align the measured point cloud of the melon segment part to the design model.

[0037] First, the model point cloud is obtained on the CAD design model of the melon peel by sampling. Then, the covariance matrix of the model point cloud and the measured point cloud is calculated respectively. According to the boundary of the point cloud in the direction of the three eigenvectors of the covariance matrix, the directed bounding box of the point cloud is determined. The results are shown in the attached figure. Figure 5. Rough registration is achieved by registering two directed bounding boxes. However, the directed bounding box of the melon slice is a flat cuboid. There are 8 different registration methods for registering two directed bounding boxes, and only one registration method is correct. Calculate the cosine similarity under each registration method

[0038]

[0039] Where R m is the rotation matrix, and is the unit vector sum of two point clouds, satisfying:

[0040]

[0041] In the formula and are the centroid coordinates of each point in the two point clouds. Cosine similarity can represent the similarity of point clouds in spatial distribution direction. The closer the spatial distribution directions of two point clouds are, the closer the cosine similarity is to 1. Selecting a registration method with the largest cosine similarity can obtain the correct rough registration result. The rough registration result can be found in the attached figure. Figure 6 Then, the point-to-surface iterative closest point ICP method is used to achieve precise alignment between the measured point cloud and the model point cloud. The precise alignment results are shown in the attached Figure 7 .

[0042] (3) Select the cross section according to the position of the design model, extract the contour points of the point cloud on the cross section, and calculate the part manufacturing deviation based on the contour information.

[0043] After registering the measured point cloud of the melon slice part to its design model, according to the position of the design model, the surface passing through the symmetry axis of the entire tank bottom and with an angle of 2° with the melon slice assembly surface is selected as the cross section (refer to the attached Figure 8 ). The contour points on this section can well reflect the curvature deviation of the melon slice part and provide information for subsequent planning of the optimal assembly sequence. Then, according to the section position, a point cloud subset with a distance less than 2 mm from the section is segmented. The segmentation results are as follows: Fig. 9 The highlighted part in the segmented point cloud subset is established. For each point, the 5 points with the closest Euclidean distance are found. These 6 points are used to perform local quadratic surface fitting based on the least squares method. On the intersection of the fitting surface and the cross section, the point with the same x coordinate as the x coordinate of the centroid of the 6 points is taken as a contour point of the point cloud on the cross section. The results of all contour point extraction refer to the attached Fig.10 . Calculate the root mean square error of the distance from all contour points to the theoretical contour to determine the size of the manufacturing deviation of the part.

[0044] (4) Plan the optimal assembly sequence with the smallest overall deviation based on the manufacturing deviation of each part.

[0045] For the tank bottom, the design model of the six melon slice parts is the same. In theory, all the melon slice parts can be assembled into a tank bottom in any order. However, in reality, the manufacturing deviation of each melon slice part is different. Different assembly sequences may cause different internal stresses of the tank bottom after assembly, affecting the stability and reliability of the structure. In order to minimize the overall internal stress of the tank bottom after the melon slices are assembled, the assembly sequence with the closest manufacturing deviations of adjacent melon slice parts is selected as the optimal assembly sequence. Introduce the error function:

[0046]

[0047] where δ i It is the cross section of two adjacent parts near their respective assembly positions during assembly (refer to the attached Figure 8 ), where the manufacturing error is the root mean square error of the distance from the cross-sectional contour point to the theoretical contour. Traverse all assembly sequences of the part, select the one with the smallest error function as the best assembly sequence, and implement virtual assembly by aligning the measured point cloud to the design model position in step (2).

[0048] (5) For the part point cloud under the optimal assembly sequence, extract the cross-sectional contour perpendicular to the assembly surface, calculate the distance between the part boundary points, and output the assembly clearance of the parts.

[0049] After completing the virtual assembly of the melon slice parts according to the optimal assembly sequence, take the sections of the entire tank bottom at 50, 150, 250, 350, 450, 550, 650, 750, and 850 mm from the bottom surface, extract the contour points on the sections, and for each melon slice part, select the center of gravity of the three points with the largest and smallest angles according to the angle of the contour point to the center of gravity of the tank as the boundary points of the melon slice part on the section, that is, the corresponding points on the assembly surface. Calculate the distance between the corresponding assembly points of adjacent melon slice parts to determine the assembly interference clearance of the melon slice parts.

Claims

1. A method for virtual assembly of parts based on three-dimensional point cloud, characterized in that Calculate assembly deviations and provide assembly instructions before actual assembly of parts, including the following steps: (1) Perform three-dimensional measurement to obtain surface point cloud data of the measured object; (2) Align the measured point cloud of the part to the design model; (3) Select a cross section based on the position of the design model, extract the contour points of the point cloud on the cross section, and calculate the part manufacturing deviation based on the contour information; (4) Plan the optimal assembly sequence with the smallest overall deviation based on the manufacturing deviation of each part; (5) For the part point cloud under the optimal assembly sequence, extract the cross-sectional contour perpendicular to the assembly surface, calculate the distance between the part boundary points, and output the assembly clearance of the parts.

2. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 1, characterized in that: The (1) method uses a projector of a three-dimensional measurement system to project multi-frequency phase-shifted sinusoidal grating stripes at multiple brightnesses onto the surface of the part to be measured, selects stripes of different frequencies at each brightness, and projects N vertical stripe patterns with a phase difference of 2π / N at each frequency. Each time a projection is made, the left and right cameras of the three-dimensional measurement system are triggered to take pictures. After a continuous phase map of the projected part surface is obtained through phase solution and phase expansion, the same-name point matching and three-dimensional reconstruction are performed according to the epipolar constraint relationship between the two cameras. After point cloud stitching, fusion, and filtering, the complete point cloud data of the surface of the part to be measured is obtained.

3. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 1, characterized in that: The method (2) first samples the CAD design model of the part to obtain a model point cloud, calculates the directed bounding boxes of the model point cloud and the measured point cloud respectively according to the eigenvectors of the covariance matrix, calculates the cosine similarity of the directed bounding boxes under different registration methods, selects the registration method with the smallest cosine similarity for registration, realizes the rough registration of the measured point cloud and the model point cloud, and then uses the point-to-surface iterative closest point ICP algorithm to realize the precise registration of the measured point cloud and the model point cloud.

4. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 3, characterized in that: After the directed bounding box of the point cloud is calculated in (2), a rough registration is achieved by aligning the directed bounding box of the measured point cloud with the directed bounding box of the target point cloud. A maximum of 24 different registration methods of the two rectangular bounding boxes are traversed to calculate the cosine similarity of the two point clouds. The calculation formula of the cosine similarity is as follows: Where R m is the rotation matrix, and is the unit vector sum of two point clouds, satisfying: In the formula and For the barycentric coordinates of each point in the two point clouds, select the cosine similarity λ m The measured point cloud is rotated by the smallest rotation matrix, and then the measured point cloud is translated according to the center of gravity connection line of the two point clouds to complete the rough alignment.

5. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 1, characterized in that: In the method (3), after selecting a cross section, a set of neighboring points whose distance to the cross section is less than a threshold is extracted, and a kd-tree is established therein. For each point in the set of neighboring points, the five points whose Euclidean distance is closest to the point are found, and these six points are used to perform local quadratic surface fitting based on the least squares method. On the intersection of the fitting surface and the cross section, a point whose x coordinate is the same as the x coordinate of the centroid of the six points is taken as a contour point of the point cloud on the cross section, and the root mean square error of the distance from all contour points to the theoretical contour is calculated to determine the size of the manufacturing deviation of the part.

6. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 1, characterized in that: The (4) described above introduces the error function: where δ i It is the difference in manufacturing errors of two adjacent parts on the cross sections near their respective assembly positions during assembly. The manufacturing error is the root mean square error of the distance from the cross-section contour point to the theoretical contour. All assembly sequences of the parts are traversed, and the one with the smallest error function is selected as the optimal assembly sequence. The virtual assembly is realized by aligning the measured point cloud to the design model position.

7. The method for virtual assembly of parts based on three-dimensional point cloud according to claim 1, characterized in that: The (5) described above extracts the contour points of the part point cloud on the cross section perpendicular to the assembly surface, extracts several points of the contour points closest to the assembly surface according to the design model, calculates the center of gravity of these points as the boundary point coordinates of the part, and calculates the size of the assembly gap or overlap according to the boundary point coordinates of the two parts.

Images