Method and system for profile measurement based on 3D scanning

Abstract

The present invention provides a contour measurement method and system based on 3D scanning, wherein the measurement method includes the following steps: (a) acquiring 3D point cloud data of the surface to be measured based on depth coordinates; (b) reconstructing a three-dimensional model surface corresponding to the surface to be measured based on the acquired 3D point cloud data; and (c) calculating the contour of the surface to be measured based on the three-dimensional model surface and the theoretical point cloud model surface of the surface to be measured.

Description

Technical Field

[0001] This invention relates to the field of machine vision measurement, and more particularly to a contour measurement method and system based on 3D scanning. Background Technology

[0002] In the production of automotive parts, the application of curved surface design is becoming increasingly widespread, such as in car doors and body shells. However, due to factors such as process parameters and the complexity of curved surface design, the accuracy of workpiece surface dimensions can be easily affected. Therefore, it is necessary to measure the surface profile of the workpiece. Surface profile reflects the variation of the measured actual profile relative to the ideal profile, and its main descriptive index is profile error. The measurement of workpiece surface profile is of great significance to the accuracy and sealing performance of vehicle assembly.

[0003] The main methods for measuring surface profile error include: contouring device measurement, cross-sectional profile template measurement, optical tracking profile measuring instrument measurement, and coordinate measuring machine (CMM) measurement. CMM measurement can be further divided into CMM and structured light methods. Except for CMM measurement, the other three methods require a theoretical profile template. The actual profile surface of the workpiece is compared with the profile surface of the theoretical template to determine the surface profile error value. These three methods are only suitable for inspection during mass production and have relatively low measurement accuracy.

[0004] While coordinate measuring machines (CMMs) offer high precision, their slow single-point scanning speed and low efficiency make them unsuitable for inspecting the surface contours of large areas on production lines. Structured light methods, on the other hand, are suitable for measuring large, easily deformable cover parts. Summary of the Invention

[0005] A major advantage of this invention is that it provides a contour measurement method and system based on 3D scanning. The 3D scanning method has a fast sampling frequency, which can acquire a large amount of position information for three-dimensional reconstruction in a short time, making it fast and efficient.

[0006] Another advantage of the present invention is that it provides a contour measurement method and system based on 3D scanning, wherein the measurement method is based on the high accuracy of 3D camera scanning, which can meet the high-precision measurement requirements of precision product manufacturing.

[0007] Another advantage of the present invention is that it provides a contour measurement method and system based on 3D scanning, wherein the measurement method adopts non-contact sampling, and uses a 3D camera to scan the workpiece surface to be measured along the XY coordinate axis, avoiding damage to the workpiece surface caused by contact sampling, and has a fast sampling frequency and high accuracy.

[0008] Another advantage of this invention is that it provides a contour measurement method and system based on 3D scanning. The measurement method is based on the principles of dynamic programming and interior point method to plan the optimal scanning path and obtain the scanning scheme with the shortest total scanning path length and the fewest turning times. This method achieves a large amount of effective data acquisition per unit time and is highly efficient.

[0009] Another advantage of the present invention is that it provides a contour measurement method and system based on 3D scanning, wherein the measurement method reconstructs the surface to be measured in three dimensions, making the geometric information of the surface to be measured more complete and specific, making up for the problem of missing points, and the contour calculation accuracy is higher.

[0010] Another advantage of the present invention is that it provides a contour measurement method and system based on 3D scanning, wherein the measurement method obtains a 3D point cloud model of the surface to be measured by stitching together the depth information of the surface to be measured by 3D scanning, and calculates the contour of the surface to be measured based on the 3D point cloud model, which helps to improve the measurement accuracy and precision.

[0011] According to one aspect of the present invention, a contour measurement method based on 3D scanning, which can achieve the foregoing and other objectives and advantages, includes the following steps:

[0012] (a) Acquire 3D point cloud data of the surface under test based on depth coordinates;

[0013] (b) Reconstructing a three-dimensional model surface corresponding to the surface under test based on the acquired 3D point cloud data; and

[0014] (c) Calculate the profile of the surface to be measured based on the three-dimensional model surface and the theoretical point cloud model surface of the surface to be measured.

[0015] According to an embodiment of the present invention, step (a) of the measurement method includes:

[0016] (a.1) The surface to be measured is scanned line by line along the set scanning path using a depth information acquisition device to obtain a depth image corresponding to the surface to be measured; and

[0017] (a.2) Analyze the depth image of the surface to be tested and obtain the 3D coordinate information corresponding to the depth image to obtain 3D point cloud data corresponding to the surface to be tested.

[0018] According to an embodiment of the present invention, step (a.1) of the measurement method further includes the following steps:

[0019] (a.1) Scan the surface to be measured along a predetermined scanning path to obtain a depth image corresponding to a sub-region of the surface to be measured; and

[0020] (a.2) Switch the scanning path of the surface to be tested and scan the surface to be tested along the scanning path to obtain a depth image corresponding to another sub-region of the surface to be tested, so as to completely scan the surface to be tested.

[0021] According to an embodiment of the present invention, the measurement method further includes the following steps prior to step (a):

[0022] (a0) Based on the principle of scan path optimization: Obtain the optimal scan path, where △y i This represents the area scanned along the Y-axis during the Nth scan.

[0023] According to an embodiment of the present invention, step (a0) of the measurement method further includes the following step:

[0024] (a0.1) Divide the entire surface to be tested into several small regions; and

[0025] (a0.2) Identify the small regions of invalid scans, merge these small regions and delete them as a whole to determine the optimal scan path.

[0026] According to an embodiment of the present invention, step (b) of the measurement method further includes the following steps:

[0027] (b.1) Establish a reference coordinate system corresponding to the 3D point cloud data; and

[0028] (b.2) Based on the reference coordinate system and the 3D point cloud data, a surface reconstruction algorithm using mesh growth is used to obtain the surface of the three-dimensional model.

[0029] According to an embodiment of the present invention, step (b.2) of the measurement method further includes:

[0030] Perform local triangulation on the point cloud data; and

[0031] Find more points to connect the mesh according to the surface neighborhood standard, and let the mesh grow until all possible points are connected to complete the reconstruction of the surface of the workpiece 3D model.

[0032] According to an embodiment of the present invention, step (a) of the measurement method further includes:

[0033] (a.3) The 3D point cloud data units are spliced ​​together to form a complete 3D point cloud data group corresponding to the surface to be tested.

[0034] According to an embodiment of the present invention, step (a.3) of the measurement method further includes the following steps:

[0035] (a.3.1) Move other 3D point cloud data units to their theoretical positions based on the position of one 3D point cloud data unit;

[0036] (a.3.2) Calculate the offset, height difference and rotation angle between any two adjacent 3D point cloud data units, and adjust the relative position between the adjacent 3D point cloud data units based on the offset, height difference and rotation angle to obtain a 3D point cloud data set corresponding to the surface to be measured.

[0037] According to an embodiment of the present invention, step (a.3.2) of the measurement method further includes the following steps:

[0038] (a.3.2.1) Obtain the overlapping region of two adjacent 3D point cloud data units, and denote subgraph A and subgraph B; and

[0039] (a.3.2.2) Calculate the offset, height difference, and rotation angle of two adjacent 3D point cloud data units based on subgraph A and subgraph B.

[0040] According to an embodiment of the present invention, step (a.3.2.2) of the measurement method further includes:

[0041] Set at least 3 mark points in each overlapping area; and

[0042] Based on the correspondence between the three Mark points in the coordinate system of two adjacent 3D point cloud data units, calculate the offset or rigid body transformation matrix of one of the 3D point cloud data units.

[0043] According to an embodiment of the present invention, in step (c) of the measurement method, the distance from each point on the three-dimensional model surface to the corresponding point on the theoretical model surface is calculated, and twice the maximum value among all distances is determined as the profile error.

[0044] According to an embodiment of the present invention, step (c) further includes the following steps:

[0045] (c.1) Create a planar model corresponding to the surface to be measured;

[0046] (c.2) Obtain the common part of the 3D model surface and the planar model at the corresponding position of the surface to be measured, and extract the contour line at the specified position; and

[0047] (c.3) Align the extracted contour lines with the theoretical contour lines and find their maximum deviation to obtain the line profile.

[0048] According to another aspect of this application, this application further provides a contour measurement system based on 3D scanning, comprising:

[0049] A point cloud processing module is used to acquire 3D point cloud data corresponding to the surface to be measured.

[0050] A 3D reconstruction module, wherein the 3D reconstruction module reconstructs a 3D model surface based on the 3D point cloud data; and

[0051] A contour calculation module, wherein the contour calculation module obtains the contour of the surface to be measured by comparing the surface of the three-dimensional model with the surface of the theoretical point cloud model.

[0052] According to one embodiment of the present invention, the device further includes a depth information acquisition module, which scans the surface of the workpiece to be tested line by line along a set scanning route to obtain 3D coordinate information under the depth image.

[0053] According to an embodiment of the present invention, the point cloud processing module includes an image analysis unit and a point cloud stitching unit, wherein the image analysis unit is used to analyze the depth image of the surface to be measured to obtain 3D point cloud data corresponding to the surface to be measured, and the point cloud stitching unit stitches the 3D point cloud data units corresponding to each sub-region of the area to be measured into a complete 3D point cloud data group.

[0054] According to one embodiment of the present invention, a path planning module is further included, wherein the path planning module is connected to the depth information acquisition device, and the path planning module formulates a suitable scanning path for the depth information acquisition device based on the surface to be measured.

[0055] The further objects and advantages of the invention will become fully apparent from the following description and accompanying drawings.

[0056] These and other objects, features and advantages of the present invention will become fully apparent from the following detailed description and accompanying drawings. Attached Figure Description

[0057] Figure 1 This is a flowchart of a contour measurement method based on 3D scanning according to a first preferred embodiment of the present invention.

[0058] Figure 2 This is a flowchart illustrating the steps of the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention.

[0059] Figure 3A and Figure 3BThis is a schematic diagram of the scanning path of the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram of stitching point cloud data according to the contour measurement method based on 3D scanning of the present invention, as described in the first preferred embodiment above.

[0061] Figure 5A This is a schematic diagram of the overlapping area of ​​two adjacent sub-regions during the point cloud data stitching process of the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention.

[0062] Figure 5B This is a schematic diagram of the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention with respect to the transformation matrix.

[0063] Figure 6 This is a schematic diagram of triangular meshing for reconstructing a three-dimensional surface model in the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention.

[0064] Figure 7 This is a schematic diagram of the contour measurement method based on 3D scanning according to the first preferred embodiment of the present invention, regarding contour calculation.

[0065] Figure 8 This is a schematic diagram of the system framework of a contour measurement system based on 3D scanning according to a second preferred embodiment of the present invention. Detailed Implementation

[0066] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0067] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0068] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0069] Referring to the accompanying drawings of this invention Figures 1 to 8 As shown, a contour measurement method and system based on 3D scanning according to a first preferred embodiment of the present invention will be described in the following description, hereinafter referred to as the contour measurement method based on 3D scanning. The measurement method mainly includes: planning the optimal scanning path; using a 3D camera along the XY coordinate axis to record the coordinate information of the workpiece surface to be measured in a combination of motion and scanning; stitching together the sub-regions scanned multiple times to form a complete surface and reconstructing a three-dimensional model; and calculating the surface contour by comparing the theoretical contour with the actual contour. It is understood that the measurement method based on 3D scanning has a fast sampling frequency, can acquire a large amount of positional information for three-dimensional reconstruction in a short time, and is fast and efficient. The measurement system has high scanning accuracy, which can meet the high-precision measurement requirements of precision product (workpiece) production. It is worth mentioning that in this preferred embodiment of the present application, the measurement method can achieve rapid and accurate measurement of the workpiece surface dimensions.

[0070] It is worth mentioning that the measurement method employs non-contact sampling, using a 3D camera to scan the workpiece surface along the XY coordinate axes, avoiding damage to the workpiece surface caused by contact sampling. This method offers high sampling frequency and accuracy. Furthermore, based on dynamic programming and interior point methods, the method plans the optimal scanning path, obtaining the scanning scheme with the shortest total path length and fewest turning points. This results in a large amount of effective data acquisition per unit time and high efficiency.

[0071] like Figure 1 and Figure 2 As shown, the contour measurement method based on 3D scanning in this preferred embodiment of the present invention includes the following steps:

[0072] (a) Acquire 3D point cloud data of the surface under test based on depth coordinates;

[0073] (b) Reconstructing a three-dimensional model surface corresponding to the surface under test based on the acquired 3D point cloud data; and

[0074] (c) Calculate the profile of the surface to be measured based on the three-dimensional model surface and the theoretical point cloud model surface of the surface to be measured.

[0075] In step (a) of the measurement method of the preferred embodiment of the present invention, a depth information acquisition device is used to sequentially scan the surface to be measured along a set scanning path to obtain 3D coordinate information under the depth image. By adjusting the relative position of the depth information acquisition device and the surface to be measured, the scanning path direction is adjusted, and 3D coordinate information under another sub-region can be obtained using the above method. The above steps are repeated until the entire workpiece surface is completely scanned.

[0076] Specifically, in step (a) of the measurement method, the surface to be measured is scanned line by line along a predetermined scanning path to obtain 3D point cloud data corresponding to the surface to be measured. It is worth noting that step (a) of the measurement method in this preferred embodiment of the present application includes:

[0077] (a.1) The surface to be measured is scanned line by line along the set scanning path using a depth information acquisition device to obtain a depth image corresponding to the surface to be measured; and

[0078] (a.2) Analyze the depth image of the surface to be tested and obtain the 3D coordinate information corresponding to the depth image to obtain 3D point cloud data corresponding to the surface to be tested.

[0079] Specifically, step (a.1) of the measurement method further includes the following steps:

[0080] (a.1) Scan the surface to be measured along a predetermined scanning path to obtain a depth image corresponding to a sub-region of the surface to be measured; and

[0081] (a.2) Switch the scanning path of the surface to be tested and scan the surface to be tested along the scanning path to obtain a depth image corresponding to another sub-region of the surface to be tested, so as to completely scan the surface to be tested.

[0082] It should be noted that, in this preferred embodiment of the present application, the surface to be measured is divided into several sub-regions, and the depth information acquisition device scans each sub-region line by line. Therefore, after a single scan of each sub-region of the surface to be measured, the depth information acquisition device obtains a depth image corresponding to that sub-region, and then parses the depth image of the sub-region into corresponding 3D point cloud data units. In other words, the 3D point cloud data corresponding to the surface to be measured in this application is composed of 3D point cloud data units corresponding to several sub-regions.

[0083] It is worth mentioning that, in this preferred embodiment of the present application, the depth information acquisition device may be, but is not limited to, a structured light imaging device, a 3D camera, etc.

[0084] The measurement method of the present invention further includes the following steps prior to step (a):

[0085] (a0) Based on the principle of scan path optimization: Obtain the optimal scan path, where △y i This represents the area scanned along the Y-axis during the Nth scan.

[0086] The measurement method of the present invention further includes the following steps in step (a0):

[0087] (a0.1) Divide the entire surface to be tested into several small regions; and

[0088] (a0.2) Identify the small regions of invalid scans, merge these small regions and delete them as a whole to determine the optimal scan path.

[0089] Those skilled in the art will understand that the measurement method of this preferred embodiment is based on the principles of dynamic programming and interior point method, plans the optimal scanning path, and obtains the scanning scheme with the shortest total scanning path length and the fewest turning times. This method achieves a large amount of effective data acquisition per unit time and is highly efficient.

[0090] like Figure 3A and Figure 3B As shown, the 3D camera scans the surface to be scanned during its movement. After each scan, the 3D camera moves along a direction perpendicular to the scan direction, switching the scan path. For example, moving along the y-direction increases the scanned area, while moving along the x-direction only changes the position, increasing the number of scans.

[0091] Step (a) of the measurement method in the preferred embodiment of the present invention involves scanning the surface to be measured to obtain the depth coordinate information corresponding to the surface. Directly selecting discrete points to calculate the profile can lead to imbalanced sample sets and low accuracy in the calculation results. Therefore, three-dimensional reconstruction of the point cloud data is necessary. This makes the geometric information of the workpiece surface more complete and specific, compensating for the problem of missing points, and thus improving the accuracy of the profile calculation.

[0092] Step (b) of the measurement method in the preferred embodiment of the present invention is to reconstruct the three-dimensional model surface based on the acquired 3D point cloud data. First, a reference coordinate system is established for the point cloud. Then, a surface reconstruction algorithm based on mesh growth is used. This algorithm first performs local triangulation on the point cloud data, and then finds more points to connect the mesh according to the surface neighborhood standard, so that the mesh grows until all possible points are connected, thereby completing the reconstruction of the three-dimensional surface model of the workpiece.

[0093] like Figure 6As shown, in step (a) of the measurement method in the preferred embodiment described above, the 3D point cloud data obtained includes data points D1, D2, D3, D4, D5, D6…, where any three points in the 3D point cloud data can form a plane, and the plane formed by the three closest points is closest to the structure of the actual plane. Therefore, after locally triangulating the three closest data points D1, D2, and D3 using a mesh growth surface reconstruction algorithm, a plane (a mesh) formed by data points D1, D2, and D3 is obtained. Following the proximity principle, D2, D3, and D4 are then used to form two interconnected meshes in the same way. Thus, D1, D2, D3, and D4 form two interconnected planes. More points are searched to connect the meshes according to the surface neighborhood standard, allowing the mesh to grow until all possible data points are connected, so that all the 3D point cloud data are connected into a three-dimensional model surface that approximates the surface to be measured.

[0094] Step (b) of the measurement method of the preferred embodiment of the present invention further includes the following steps:

[0095] (b.1) Establish a reference coordinate system corresponding to the 3D point cloud data; and

[0096] (b.2) Based on the reference coordinate system and the 3D point cloud data, a surface reconstruction algorithm using mesh growth is used to obtain the surface of the three-dimensional model.

[0097] Step (b.2) of the measurement method of the preferred embodiment of the present invention further includes:

[0098] Perform local triangulation on the point cloud data; and

[0099] Find more points to connect the mesh according to the surface neighborhood standard, and let the mesh grow until all possible points are connected to complete the reconstruction of the surface of the workpiece 3D model.

[0100] It should be noted that for some workpieces with large surfaces, such as car doors and bodies, a single scan cannot obtain a depth image corresponding to the entire surface. By scanning multiple times and then stitching the images together, a complete depth image and 3D point cloud data corresponding to the surface to be measured can be obtained.

[0101] Therefore, step (a) of the measurement method in the preferred embodiment of the present invention further includes:

[0102] (a.3) The 3D point cloud data units are spliced ​​together to form a complete 3D point cloud data group corresponding to the surface to be tested.

[0103] In other words, in step (a.3) of the measurement method of the preferred embodiment of the present invention, multiple 3D point cloud data units obtained in step (a) are stitched together to form 3D point cloud data corresponding to the entire surface to be measured by feature-based point cloud stitching, so as to reconstruct the three-dimensional model surface.

[0104] In step (a.3) of the measurement method of the preferred embodiment of the present invention, the starting point pixel coordinates (x1, y1) of each sub-region are organized, and these coordinates are translated to the theoretical position of the overall image to obtain the overlapping areas of adjacent regions, denoted as sub-images A and B. Translation matching and height difference calculation are performed on sub-images A and B to calculate the offset. x ,offset y and height difference offset z Add at least 3 Mark points in each overlapping region. Utilize the correspondence of these Mark points in different coordinate systems to calculate the offset or rigid body transformation matrix. Based on the principle of coordinate system transformation, convert the depth image coordinate information into point cloud coordinate information. Based on the acquired offset or rigid body transformation, finally obtain the 3D point cloud data of the workpiece.

[0105] In detail, in this preferred embodiment of the present application, any 3D point cloud data unit can be used as a reference data unit. By adjusting the positional relationship of the remaining 3D point cloud data units, the 3D point cloud data units are stitched together to form a complete 3D point cloud data group corresponding to the surface under test. As an example, in this preferred embodiment of the present application, the first point cloud data unit is used as the reference data unit, and the relative positions of the subsequently acquired 3D point cloud data units are adjusted sequentially to form the 3D point cloud data group corresponding to the surface under test.

[0106] like Figure 5AAs shown, as an example, the starting pixel coordinates of the first sub-region of the surface under test are (x1, y1), and the starting pixel coordinates of the second sub-region of the surface under test are (x2, y2). The second 3D point cloud data unit is translated as a whole to the theoretical position of the image (i.e., the second 3D point cloud data unit is moved according to the moving step size of the scanning chamber). The first and second 3D point cloud data units have an overlapping area after the movement, that is, the adjacent edge areas of the first and second 3D point cloud data units overlap. Let the overlapping area of ​​the first 3D point cloud data unit be sub-image A, and the overlapping area of ​​the second 3D point cloud data unit be sub-image B. At this time, sub-image A and sub-image B may not completely overlap, that is, there may be a height difference, angle difference, or positional offset between sub-image A and sub-image B. Therefore, the relative positions of sub-images A and B need to be adjusted so that they completely overlap. This completes the stitching of the first and second 3D point cloud data sets, allowing for accurate stitching of individual 3D point cloud data units. Translation matching and height difference calculations are performed on sub-images A and B, and the offset is calculated. x ,offset y and height difference offset z .

[0107] Accordingly, step (a.3) of the measurement method described in this application further includes the following steps:

[0108] (a.3.1) Move other 3D point cloud data units to their theoretical positions based on the position of one 3D point cloud data unit;

[0109] (a.3.2) Calculate the offset, height difference and rotation angle between any two adjacent 3D point cloud data units, and adjust the relative position between the adjacent 3D point cloud data units based on the offset, height difference and rotation angle to obtain a 3D point cloud data set corresponding to the surface to be measured.

[0110] It is worth mentioning that in step (a.3.1) of the measurement method described in this application, each of the 3D point cloud data units is moved to the theoretical position based on the coordinate values ​​of the starting point pixels of each sub-region in the region to be measured.

[0111] The measurement method described in this application further includes the following steps in step (a.3.2):

[0112] (a.3.2.1) Obtain the overlapping region of two adjacent 3D point cloud data units, and denote subgraph A and subgraph B; and

[0113] (a.3.2.2) Calculate the offset, height difference, and rotation angle of two adjacent 3D point cloud data units based on subgraph A and subgraph B.

[0114] Furthermore, step (a.3.2.2) of the measurement method described in this application further includes:

[0115] Set at least 3 mark points in each overlapping area; and

[0116] Based on the correspondence between the three Mark points in the coordinate system of two adjacent 3D point cloud data units, calculate the offset or rigid body transformation matrix of one of the 3D point cloud data units.

[0117] It is worth mentioning that, in this preferred embodiment of the application, the three Mark points in the overlapping area are set as (M1, M2, M3), wherein the coordinates of the Mark points in the coordinate system of one 3D point cloud data unit are marked as (x1, y1, z1), and the coordinates of the adjacent 3D point cloud data unit are marked as (x2, y2, z2). Based on the above coordinates, the offset, height difference and rotation angle between the two adjacent 3D point cloud data units are calculated.

[0118] In step (a.3.2.2) of the measurement method of this application: based on the principle of coordinate system transformation, the depth image coordinate information is converted into point cloud coordinate information, and the position of the 3D point cloud data unit is adjusted based on the calculated offset or rigid body transformation to obtain a 3D point cloud data group corresponding to the surface to be measured.

[0119] Step (c) of the measurement method described in this application is to calculate the profile of the surface to be measured. For calculating the profile of the surface to be measured, a method is adopted that compares the actual point cloud model surface with the theoretical point cloud model surface. The distance from each point in the actual point cloud model to the corresponding point in the theoretical point cloud model is calculated, and twice the maximum value among all distances is determined as the profile error. Optionally, in another optional embodiment of this application, the profile is a directed distance, that is, the minimum positive value minus the minimum negative value is used as the profile, that is, the maximum deviation values ​​on both sides are taken and added together to obtain the corresponding profile.

[0120] Therefore, in step (c) of the measurement method of this application, the distance from each point on the three-dimensional model surface to the corresponding point on the theoretical model surface is calculated, and twice the maximum value among all distances is determined as the profile error.

[0121] Those skilled in the art will understand that the measurement method can also be used to measure the line profile of the surface under test. For cases requiring line profile calculation, a planar model can be created first, and then the common portion of the 3D model and the planar model at the corresponding position of the workpiece can be obtained. The profile line at the specified position can then be further extracted. After the profile line coordinates are extracted, a polynomial fitting is performed, and then the maximum deviation is calculated by aligning it with the theoretical profile line, which is the line profile.

[0122] Therefore, step (c) of the measurement method of this application further includes the following steps:

[0123] (c.1) Create a planar model corresponding to the surface to be measured;

[0124] (c.2) Obtain the common part of the 3D model surface and the planar model at the corresponding position of the surface to be measured, and extract the contour line at the specified position; and

[0125] (c.3) Align the extracted contour lines with the theoretical contour lines and find their maximum deviation to obtain the line profile.

[0126] Refer to the accompanying drawings in this application specification. Figure 8 As shown, the contour measurement system based on 3D scanning according to the second preferred embodiment of the present invention will be described in the following description, hereinafter referred to as the measurement system. The measurement system plans the optimal scanning path; uses a 3D camera along the XY coordinate axis to record the coordinate information of the workpiece surface to be measured in a combination of motion and scanning; stitches together the sub-regions scanned multiple times to form a complete surface and reconstructs a three-dimensional model; and calculates the surface contour by comparing the theoretical contour with the actual contour. It is understood that the measurement system based on 3D scanning has a fast sampling frequency, can acquire a large amount of positional information for three-dimensional reconstruction in a short time, and is fast and efficient. The measurement system has high scanning accuracy, which can meet the high-precision measurement requirements of precision product (workpiece) production. It is worth mentioning that in this preferred embodiment of the present application, the measurement system can achieve rapid and accurate measurement of the workpiece surface dimensions.

[0127] It is worth mentioning that the measurement system adopts a non-contact sampling method, using a 3D camera to scan the workpiece surface to be measured along the XY coordinate axis, avoiding damage to the workpiece surface caused by contact sampling, and offering high sampling frequency and accuracy. Furthermore, based on dynamic programming and interior point methods, the measurement system plans the optimal scanning path, obtaining the scanning scheme with the shortest total path length and the fewest turning times. This system achieves a large amount of effective data acquisition per unit time, resulting in high efficiency.

[0128] The measurement system includes a depth information acquisition device 10, a point cloud processing module 20, a 3D reconstruction module 30, and a contour calculation module 40. The depth information acquisition device 10 is used to acquire depth information of the surface to be measured to obtain depth image information corresponding to the surface to be measured. The point cloud processing module 20 obtains 3D point cloud data corresponding to the surface to be measured based on the depth image acquired by the depth information acquisition device. The 3D reconstruction module reconstructs the 3D surface according to the 3D point cloud data obtained by the point cloud processing module to obtain a 3D model surface corresponding to the surface to be measured. The contour calculation module 40 obtains the contour information of the surface to be measured by comparing the 3D model surface and the theoretical point cloud model surface.

[0129] In detail, the depth information acquisition device 10 can be, but is not limited to, a 3D camera or a structured light camera. The depth information acquisition device 10 scans the surface of the workpiece under test line by line along a predetermined scanning path to obtain 3D coordinate information under the depth image. Then, by changing the running direction of the 3D camera, 3D coordinate information under another sub-region can be obtained using the above method. The above steps are repeated until the entire workpiece surface is completely scanned.

[0130] The depth information acquisition device 10 sequentially scans the surface to be measured along a set scanning path to obtain 3D coordinate information under the depth image. By adjusting the relative position of the depth information acquisition device 10 and the surface to be measured to adjust the scanning path direction, 3D coordinate information under another sub-region can be obtained using the above method. The above steps are repeated until the entire workpiece surface is completely scanned.

[0131] The depth information acquisition device 10 scans the surface to be measured line by line along a predetermined scanning path to obtain 3D point cloud data corresponding to the surface to be measured. It is worth mentioning that the depth information acquisition device 10 scans the surface to be measured line by line along the predetermined scanning path to obtain a depth image corresponding to the surface to be measured. The depth image information is transmitted to the point cloud processing module 20, which parses the depth image into 3D point cloud data.

[0132] The point cloud processing module 20 includes an image parsing unit 21 and a point cloud stitching unit 22. The image parsing unit 21 is used to parse the depth image of the surface to be measured and obtain the 3D coordinate information corresponding to the depth image to obtain 3D point cloud data corresponding to the surface to be measured.

[0133] In this preferred embodiment of the present invention, the depth information acquisition device 10 scans the surface to be tested along a set scanning path to obtain a depth image corresponding to a sub-region of the surface to be tested; when the depth information acquisition device 10 switches the scanning path of the surface to be tested and scans the surface to be tested along the scanning path to obtain a depth image corresponding to another sub-region of the surface to be tested, the surface to be tested is completely scanned.

[0134] Therefore, in this preferred embodiment of the application, the depth information acquisition device 10 scans line by line along a set scanning path to obtain multiple depth image units corresponding to the surface to be measured. These depth image units are then parsed into corresponding 3D point cloud data units by the image parsing unit 21. The point cloud stitching unit 22 stitches the 3D point cloud data units corresponding to each sub-region of the area to be measured into a complete 3D point cloud data set, so that the 3D reconstruction module 30 can reconstruct a 3D model surface based on the 3D point cloud data set.

[0135] The point cloud stitching unit 22 stitches together the various 3D point cloud data units to form a complete 3D point cloud data group corresponding to the surface under test. In other words, in the measurement system of the preferred embodiment of the present invention, the point cloud stitching unit 22 stitches together multiple acquired 3D point cloud data units into a 3D point cloud data group corresponding to the entire surface under test through feature-based point cloud stitching, in order to reconstruct the three-dimensional model surface.

[0136] The point cloud stitching unit 22 organizes the starting pixel coordinates (x1, y1) of each sub-region, translates these coordinates to the theoretical position of the overall image, and obtains the overlapping areas of adjacent regions, denoted as sub-images A and B. Translation matching and height difference calculation are performed on sub-images A and B to calculate the offset. x ,offset y and height difference offset z Add at least 3 Mark points in each overlapping region. Utilize the correspondence of these Mark points in different coordinate systems to calculate the offset or rigid body transformation matrix. Based on the principle of coordinate system transformation, convert the depth image coordinate information into point cloud coordinate information. Based on the acquired offset or rigid body transformation, finally obtain the 3D point cloud data of the workpiece.

[0137] The point cloud stitching unit 22 can use any 3D point cloud data unit as a reference data unit, and adjust the positional relationship of the remaining 3D point cloud data units to stitch them together into a complete 3D point cloud data group corresponding to the surface under test. As an example, in this preferred embodiment of the application, the point cloud stitching unit 22 uses the first point cloud data unit as a reference data unit, and sequentially adjusts the relative positions of subsequently acquired 3D point cloud data units to stitch them together into the 3D point cloud data group corresponding to the surface under test.

[0138] As an example, the point cloud stitching unit 22 sets the starting pixel coordinates of the first sub-region of the surface under test to (x1, y1) and the starting pixel coordinates of the second sub-region of the surface under test to (x2, y2), and translates the second 3D point cloud data unit as a whole to the theoretical position of the image (i.e., moves the second 3D point cloud data unit according to the moving step size of the scanning chamber). The first and second 3D point cloud data units have an overlapping area after the movement, that is, the adjacent edge areas of the first and second 3D point cloud data units overlap. Let the overlapping area of ​​the first 3D point cloud data unit be sub-image A, and the overlapping area of ​​the second 3D point cloud data unit be sub-image B. At this time, sub-image A and sub-image B may not completely overlap, that is, there may be a height difference, angle difference, or positional offset between sub-image A and sub-image B. Therefore, it is necessary to adjust the relative positions of sub-images A and B so that they completely overlap. Only then can the first and second 3D point cloud data be stitched together, allowing for accurate stitching of each 3D point cloud data unit. The point cloud stitching unit 22 performs translation matching and height difference calculation on sub-images A and B, calculating the offset. x ,offset y and height difference offset z .

[0139] The point cloud stitching unit 22 moves other 3D point cloud data units to their theoretical positions based on the position of one 3D point cloud data unit, and calculates the offset, height difference, and rotation angle between any two adjacent 3D point cloud data units. Based on the offset, height difference, and rotation angle, it adjusts the relative positions between adjacent 3D point cloud data units to obtain a 3D point cloud data set corresponding to the surface to be measured.

[0140] It is worth mentioning that the point cloud stitching unit 22 moves each of the 3D point cloud data units to the theoretical position based on the coordinate values ​​of the starting point pixels of each sub-region in the area to be tested.

[0141] The point cloud stitching unit 22 acquires the overlapping area of ​​two adjacent 3D point cloud data units and records sub-image A and sub-image B. Based on sub-image A and sub-image B, it calculates the offset, height difference and rotation angle of the two adjacent 3D point cloud data units.

[0142] The point cloud stitching unit 22 sets at least 3 Mark points in each overlapping area. Based on the correspondence of the 3 Mark points in the coordinate system of two adjacent 3D point cloud data units, the offset or rigid body transformation matrix of one of the 3D point cloud data units is calculated.

[0143] It is worth mentioning that, in this preferred embodiment of the application, the three Mark points in the overlapping area are set as (M1, M2, M3), wherein the coordinates of the Mark points in the coordinate system of one 3D point cloud data unit are marked as (x1, y1, z1), and the coordinates of the adjacent 3D point cloud data unit are marked as (x2, y2, z2). Based on the above coordinates, the offset, height difference and rotation angle between the two adjacent 3D point cloud data units are calculated.

[0144] The point cloud stitching unit 22 converts the depth image coordinate information into point cloud coordinate information based on the coordinate system transformation principle, and adjusts the position of the 3D point cloud data unit based on the calculated offset or rigid body transformation to obtain a 3D point cloud data group corresponding to the surface to be measured.

[0145] Directly selecting discrete points to calculate contour accuracy can lead to imbalanced sample sets and low calculation accuracy. Therefore, 3D reconstruction of the point cloud data is necessary. This results in more complete and detailed workpiece geometry, compensates for missing points, and improves the accuracy of contour accuracy calculation.

[0146] Therefore, in this preferred embodiment of the application, the 3D reconstruction module 30 reconstructs a 3D model surface corresponding to the actual surface to be measured based on the 3D point cloud data group processed by the point cloud processing module 20. Specifically, the 3D reconstruction module 30 reconstructs the 3D model surface based on the acquired 3D point cloud data. The 3D reconstruction module 30 establishes a reference coordinate system for the point cloud, and then uses a surface reconstruction algorithm based on mesh growth. This algorithm first performs local triangulation of the point cloud data, and then finds more points to connect the mesh according to the surface neighborhood standard, so that the mesh grows until all possible points are connected, thereby completing the reconstruction of the 3D surface model of the workpiece.

[0147] The 3D reconstruction module 30 establishes a reference coordinate system corresponding to the 3D point cloud data, and uses a surface reconstruction algorithm based on the reference coordinate system and the 3D point cloud data to obtain the surface of the 3D model.

[0148] The 3D reconstruction module 30 performs local triangulation of the point cloud data, searches for more points to connect the mesh according to the surface neighborhood standard, and allows the mesh to grow until all possible points are connected to complete the reconstruction of the surface of the 3D model of the workpiece.

[0149] It should be noted that for some workpieces with large surfaces, such as car doors and bodies, a single scan cannot obtain a depth image corresponding to the entire surface. By scanning multiple times and then stitching the images together, a complete depth image and 3D point cloud data corresponding to the surface to be measured can be obtained.

[0150] The contour calculation module 40 compares the reconstructed 3D model surface with the theoretical point cloud model surface to obtain the contour information of the surface to be measured. Specifically, the contour calculation module 40 calculates the contour of the workpiece surface by comparing the actual point cloud model surface with the theoretical point cloud model surface, calculating the distance from each point in the actual point cloud model to the corresponding point in the theoretical point cloud model, and determining twice the maximum value among all distances as the contour error.

[0151] For cases requiring line profile calculation, the profile calculation module 40 can first create a planar model, then obtain the common part of the 3D model and the planar model at the corresponding position of the workpiece, and further extract the profile line at the specified position. After the profile line coordinates are extracted, polynomial fitting is performed, and then the maximum deviation is calculated by aligning it with the theoretical profile line, i.e., the line profile degree.

[0152] like Figure 8 As shown, the measurement system further includes a path planning module 50, which is connected to the depth information acquisition device 10. The path planning module 50 formulates a suitable scanning path for the depth information acquisition device 10 based on the surface to be measured. In other words, in this preferred embodiment of the application, the measurement system adopts non-contact point acquisition, using a 3D camera to scan the workpiece surface to be measured along the XY coordinate axis, avoiding damage to the workpiece surface caused by contact point acquisition, and achieving high sampling frequency and high accuracy. Based on the principles of dynamic programming and interior point method, the optimal scanning path is planned to obtain the scanning scheme with the shortest total scanning path length and the fewest turning times. This method achieves a large amount of effective data acquisition per unit time and high efficiency.

[0153] The path planning module 50 is based on the principle of scan path optimization. Where △y i The area represented by the Nth scan along the Y-axis is divided into several small regions. Invalid scan regions are identified, merged, and then deleted as a whole. Finally, the optimal scan path is determined.

[0154] The measurement system described in the preferred embodiment of the present invention obtains the depth coordinate information corresponding to the surface under test by scanning the surface under test. Directly selecting discrete points to calculate the profile can lead to problems such as an unbalanced sample set and low accuracy of the calculation results. Therefore, it is necessary to perform three-dimensional reconstruction of the point cloud data. This makes the geometric information of the workpiece surface more complete and specific, compensating for the problem of missing points, and thus improving the accuracy of the profile calculation.

[0155] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

Claims

1. A 3D scan based profile measurement method, characterized in that, The measurement method comprises the following steps: (a) obtaining 3D point cloud data of the surface to be measured based on depth coordinates; (b) based on the obtained 3D point cloud data, reconstructing a three-dimensional model surface corresponding to the surface to be measured using a mesh growing surface reconstruction algorithm, wherein the mesh growing surface reconstruction algorithm comprises locally triangulating the point cloud data and connecting points according to a surface neighborhood criterion to generate a continuous surface; and (c) calculating the profile of the surface to be measured according to the three-dimensional model surface and a theoretical point cloud model surface of the surface to be measured, calculating the distance between each point of the three-dimensional model surface and the corresponding point of the theoretical model surface, and determining the maximum value of all distances as twice the profile error; step (c) further comprises: (c.1) creating a plane model corresponding to the surface to be measured; (c.2) obtaining the common part of the three-dimensional model surface and the plane model at the corresponding position, and extracting the profile line at the specified position; (c.3) aligning the extracted profile line with the theoretical profile line and calculating the maximum deviation to obtain the line profile.

2. The measurement method according to claim 1, wherein step (a) of the measurement method comprises: (a.1) scanning the surface to be measured row by row along a set scanning path by a depth information acquisition device to obtain a depth image corresponding to the surface to be measured; and (a.2) analyzing the depth image of the surface to be measured to obtain 3D coordinate information corresponding to the depth image to obtain 3D point cloud data corresponding to the surface to be measured.

3. The measurement method according to claim 2, wherein step (a.1) of the measurement method further comprises the following steps: scanning the surface to be measured along a set scanning path to obtain a depth image corresponding to a sub-region of the surface to be measured; and switching the scanning path of the surface to be measured and scanning the surface to be measured along the scanning path to obtain a depth image corresponding to another sub-region of the surface to be measured until the surface to be measured is completely scanned.

4. The measurement method according to claim 2, wherein the measurement method further comprises the following step before step (a):

5. The measurement method according to claim 4, wherein step (a0) of the measurement method further comprises the following steps: (a0) Based on the scanning path optimization principle: Obtaining the optimal scanning path, wherein represents the scanning area along the Y direction of the coordinate axis for the Nth time. (a0.1) dividing the surface to be measured into a plurality of small regions; and (a0.2) determining the small regions with invalid scanning, merging these small regions and deleting them as a whole to determine the optimal scanning path.

6. The measurement method according to any one of claims 2 to 5, wherein step (a) of the measurement method further comprises: (a.3) splicing the 3D point cloud data units to obtain a complete 3D point cloud data set corresponding to the surface to be measured.

7. The measurement method according to claim 6, wherein step (a.3) of the measurement method further comprises the following step: (a.3.1) moving other 3D point cloud data units to their theoretical positions based on the position of a piece of 3D point cloud data unit. ​ (a.3.2) calculating the offset, height difference and rotation angle between any two adjacent 3D point cloud data units, and adjusting the relative position between the adjacent 3D point cloud data units based on the offset, height difference and rotation angle to obtain a 3D point cloud data set corresponding to the surface to be measured.

8. The measurement method according to claim 7, wherein the step (a.3.2) of the measurement method further comprises the following steps: (a.3.2.1) obtaining the overlapping region of two adjacent 3D point cloud data units, and recording the subgraph A and subgraph B; and (a.3.2.2) calculating the offset, height difference and rotation angle of the two adjacent 3D point cloud data units based on the subgraph A and subgraph B.

9. The measurement method according to claim 8, wherein the step (a.3.2.2) of the measurement method further comprises: setting at least 3 Mark points in each overlapping region; and calculating the offset or rigid transformation matrix of one of the 3D point cloud data units based on the corresponding relationship of the 3 Mark points in the coordinate system of the two adjacent 3D point cloud data units.

10. A profile measurement system based on 3D scanning, characterized in that, comprises: a point cloud processing module for obtaining 3D point cloud data corresponding to a surface to be measured; a three-dimensional reconstruction module for reconstructing a three-dimensional model surface based on the 3D point cloud data using a mesh growing surface reconstruction algorithm, wherein the mesh growing surface reconstruction algorithm comprises locally triangulating the point cloud data and connecting points according to a surface neighborhood criterion to generate a continuous surface; and a profile calculation module, wherein the profile calculation module obtains the profile of the surface to be measured by comparing the three-dimensional model surface with a theoretical model surface, calculates the distance from each point of the three-dimensional model surface to the corresponding point of the theoretical model surface, and determines that twice the maximum value of all distances is the profile error; the profile calculation module can also create a plane model corresponding to the surface to be measured, obtain the common part of the three-dimensional model and the plane model at the corresponding position of the surface to be measured, extract the profile line at the specified position, align the extracted profile line with the theoretical profile line, and calculate the maximum deviation to obtain the line profile.

11. The measurement system according to claim 10, further comprising a depth information acquisition module for scanning the surface to be measured row by row along a set scanning route to obtain 3D coordinate information under a depth image.

12. The measurement system according to claim 11, wherein the point cloud processing module comprises an image analysis unit and a point cloud splicing unit, wherein the image analysis unit is used to analyze the depth image of the surface to be measured to obtain 3D point cloud data corresponding to the surface to be measured, and the point cloud splicing unit splices the 3D point cloud data units corresponding to each subregion of the surface to be measured into a complete 3D point cloud data set.

13. The measurement system of claim 12, further comprising a path planning module, wherein the path planning module is connected to the depth information acquisition module, and a scanning path is made by the depth information acquisition module based on the surface to be measured by the path planning module.

Images