Multi-angle three-dimensional measurement device and point cloud denoising method

Abstract

The invention discloses a multi-angle three-dimensional measurement device and a point cloud denoising method. The device comprises a projection device, an image acquisition device, a connecting piece for fixedly connecting the projection device and the image acquisition device, a two-dimensional displacement platform group for adjusting a measurement distance and an electric rotating platform for rotating a measured object, all parts are linked by a controller, and one-button automatic calibration and multi-angle measurement are realized. The method is based on the device, and noise is effectively filtered out from multi-view point clouds collected by rotating around the same axis through nearest neighbor search and statistical judgment. Through combination of displacement and the rotating platform, multi-angle data are automatically obtained, and metal reflection interference is effectively inhibited; through an innovative rotation consistency denoising algorithm, the point cloud quality is remarkably improved, and the method is suitable for multi-angle three-dimensional measurement including high-reflection workpieces.

Description

Technical Field

[0001] This invention belongs to the field of optical three-dimensional measurement technology, and specifically relates to a multi-angle three-dimensional measurement device and a point cloud denoising method. Background Technology

[0002] Structured light 3D measurement technology, with its non-contact and high-precision characteristics, has important applications in manufacturing, including dimensional measurement of high-precision parts, defect detection, and reverse engineering. This is especially true in the automotive, aerospace, and electronics manufacturing industries, where stringent dimensional accuracy requirements for components are high, the number of parts is large, and manual inspection is very costly.

[0003] There are already many structured light 3D measurement devices on the market, but traditional structured light 3D measurement devices usually only measure one surface, or only the object being measured is rotated, resulting in insufficient measurement angles, or multi-angle measurements require additional equipment and manual scanning, thus increasing costs. Because the original images acquired by the structured light 3D measurement method contain noise or cannot well identify the edges between the object being measured and the background, the final point cloud data will contain many invalid points that do not belong to the object being measured. In manufacturing-related fields, many high-reflectivity metal parts need to be fully 3D measured, but the overexposure of the images caused by metal reflection seriously affects the 3D measurement results. Therefore, there are three prominent pain points in the existing technology: (1) lack of multi-angle measurement devices that can adapt to different object sizes and shapes; (2) for highly reflective objects, single measurement is prone to point cloud loss due to overexposure; (3) traditional point cloud denoising methods (such as statistical filtering) cannot effectively distinguish between real surface points and fixed noise caused by reflection, speckle, etc. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-angle three-dimensional measurement device and a point cloud denoising method, which realizes multi-angle measurement by linking a displacement platform and a rotation platform, and uses the rotation consistency of the point cloud for denoising.

[0005] The technical solution for achieving the objective of this invention is as follows: Firstly, this invention provides a multi-angle three-dimensional measurement device, comprising:

[0006] A projection device used to project coded structured light onto the object being measured;

[0007] Image acquisition device, used to acquire structured light images modulated by the object under test;

[0008] A rigid connector is used to fix the projection device and the image acquisition device together, and to make their optical centers at the same height. The connector allows the projection device and the image acquisition device to change their horizontal viewing angle and to adjust their pitch angle as a whole.

[0009] The two-dimensional displacement platform assembly consists of two mutually perpendicular electric displacement platforms, used to jointly adjust the horizontal and vertical distances of the connecting member and its projection device and image acquisition device relative to the object being measured.

[0010] An electric rotary platform is used to carry and drive the object being measured to rotate about a single axis.

[0011] The controller is electrically connected to the projection device, image acquisition device, two-dimensional displacement platform group and electric rotary platform, and is used to control the coordinated work of each component to realize automatic calibration and automatic multi-angle measurement.

[0012] Furthermore, the projection device and the image acquisition device are connected by a trigger line, so that the image acquisition device is triggered to acquire an image every time the projection device projects an image.

[0013] Furthermore, the controller is configured to: upon receiving a signal from the image acquisition device that it has completed the acquisition of an image sequence, control the electric rotating platform to rotate by a predetermined angle, and then start the next measurement cycle.

[0014] Furthermore, the rigid connector includes a cover plate, a base plate, and a projection device bracket and an image acquisition device bracket that are movably clamped between the two; by tightening the screws and nuts between the base plate and the cover plate, the friction force on the bracket can be adjusted to lock and release its position.

[0015] Furthermore, the horizontal and vertical movement ranges of the two-dimensional displacement platform assembly are both no less than 20 cm, and the rotation angle range of the electric rotating platform is 0 to 360 degrees.

[0016] Secondly, the present invention provides a point cloud denoising method applicable to point cloud results obtained by the three-dimensional measuring device described in the first aspect, the method comprising:

[0017] (1) Input the target point cloud C to be denoised and M control point clouds that are from the same object as the target point cloud, collected around the same rotation axis but at different angles and have overlapping parts;

[0018] (2) Establish KD-tree data structures for the target point cloud and each of the reference point clouds respectively;

[0019] (3) For the point cloud that needs to be filtered, traverse the points P in it;

[0020] (4) Find the unique nearest neighbor among its several neighboring point clouds;

[0021] (5) Calculate the Euclidean distance d between point P and each of the nearest neighbors found;

[0022] (6) Count the number of Euclidean distances d less than the preset threshold D, and record it as the number of successful matches k;

[0023] (7) If the number of successful matches k is greater than or equal to the preset effective number threshold K, then the judgment point P is a valid point and is retained; otherwise, the judgment point P is a noise point and is removed.

[0024] (8) Output all valid points that are retained to form a denoised point cloud.

[0025] Furthermore, the preset threshold D is adaptively set according to the single measurement accuracy of the three-dimensional measuring device, and its value ranges from 1 to 5 times the average spacing of the point cloud.

[0026] Furthermore, the effective number threshold K has a range of 1 ≤ K ≤ M, and the value of K is positively correlated with the strictness of denoising.

[0027] Furthermore, the input target point cloud C and all reference point clouds have been unified to the same coordinate system through coordinate transformation based on the rotation axis and angle information obtained from the calibration.

[0028] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the second aspect.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] (1) The device of the present invention uses a displacement platform and connectors to adjust the position and orientation of the camera and projector and adjust the appropriate lens focal length to adapt to the test objects of different sizes.

[0031] (2) The device of the present invention can realize one-click automatic calibration of the device and one-click multi-angle three-dimensional reconstruction of the object under test by using the control unit to operate the device.

[0032] (3) The results from multiple angles can form a contrast and complement, which has a good effect on solving the problem of overexposure caused by metal reflection in 3D reconstruction, resulting in the loss of overexposed parts of the 3D point cloud results.

[0033] (4) The method of the present invention utilizes the characteristic that noise does not rotate with the object, and can filter out high-density point clouds that cannot be processed by traditional point cloud statistical filtering methods in multi-angle point clouds.

[0034] (5) The method of the present invention has high adaptability and can filter out various noise point clouds without prior noise model of point cloud.

[0035] (6) The method of the present invention is applicable to the three-dimensional point cloud results obtained by the device of the present invention, and realizes the automation of the acquisition and filtering of multi-angle three-dimensional measurement point cloud data. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the device of the present invention.

[0037] Figure 2 This is a physical diagram of the device of the present invention.

[0038] Figure 3 This is a diagram showing the device connections.

[0039] Figure 4 This is a schematic diagram of the connection between the camera and the projector.

[0040] Figure 5 Flowchart of point cloud denoising method.

[0041] Figure 6 This refers to the metal block in the specific embodiment.

[0042] Figure 7 This is a diagram of the stone steps in a specific embodiment.

[0043] Figure 8 This is a single-view point cloud image of a metal block.

[0044] Figure 9 This is a multi-view point cloud image of a metal block.

[0045] Figure 10 Point cloud image before noise removal of stone steps.

[0046] Figure 11 Point cloud image after denoising the stone steps. Detailed Implementation

[0047] This invention proposes a multi-angle structured light three-dimensional measurement device, which consists of a camera and a projector and their connecting bracket, two electric displacement platforms and an electric rotation platform.

[0048] The multi-angle three-dimensional measurement method used in the device of this invention includes:

[0049] Adjust the position of the camera projector to fit the current size of the object being measured.

[0050] Place the calibration plate on a rotating platform and project the phase-shifted image sequence onto the calibration plate.

[0051] The camera acquires grayscale and phase-shift modulated images of the calibration board, and then searches for the coordinates of feature points on the checkerboard calibration board in the grayscale images.

[0052] Phase resolution is performed on the phase-shift modulated image, and the coordinates of the feature points under the projector's viewpoint are calculated using the phase value and the coordinates of the feature points.

[0053] Rotate the calibration board by a certain angle and repeat the above process until you have obtained several feature point results of the calibration board (ensuring that the rotation axis remains unchanged during the process).

[0054] The intrinsic and extrinsic parameters of the camera and projector are calibrated using the found feature points.

[0055] At this point, we have obtained several sets of feature point coordinates, their absolute phase information, and the intrinsic and extrinsic parameters of the camera and projector. Treating the projector as another camera, we use a structured light algorithm to perform 3D reconstruction on multiple sets of feature points, obtaining multiple sets of scattered points. We then use a random sampling consensus algorithm to fit and calibrate the rotation axis of these scattered points.

[0056] Replace the calibration plate with the object being tested.

[0057] The phase-shifted image is projected onto the object under test using a projector, and a grayscale image is captured using a camera.

[0058] Phase is calculated from the acquired phase-shifted images, and point cloud data is obtained by 3D reconstruction using the internal and external parameters of the equipment obtained during calibration.

[0059] The object under test is rotated, and the above operation is repeated to obtain point cloud data at multiple angles. The angle and direction of each rotation are recorded. The number of rotations and the angles are determined by the requirements of the 3D measurement.

[0060] By fitting the calibrated rotation axis and the rotation angle and direction recorded in the previous step during the calibration process, coordinate transformation is performed on each group of point clouds, and they are rotated around the rotation axis to the correct position. Then, point cloud filtering and point cloud merging are performed to obtain the three-dimensional measurement results of the measured object from multiple angles.

[0061] This invention includes a point cloud denoising method that utilizes the point cloud results as the rotation result of the measured object. During measurement, noise does not rotate with the object. An algorithm is used to determine whether a point has a corresponding point after rotation. If a corresponding point is found, it can be determined that this point is likely not a noise point.

[0062] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0063] Example

[0064] like Figure 1 First, a measuring device needs to be set up, which includes: a projector 1, an industrial camera 2, a connector 3, two electrically controlled displacement platforms 4 and 5, and an electrically controlled rotary platform 6. Connector 3 consists of four parts, such as... Figure 4The system includes a cover plate 301, a base plate 302, a projector bracket 303, and a camera bracket 304. The camera is a monochrome camera with a resolution of 2448×2048, and the projector is a DLP projector with a resolution of 1920×1080. Both the camera and projector are connected to the computer via trigger cables. The electrically controlled displacement platform and electrically controlled rotation platform are connected to the computer through stepper motor controllers. Figure 3 As shown. The final physical device is as follows. Figure 2 A computer-controlled projector projects a sequence of phase-shifted images. For each projected image, a trigger signal is sent to the camera to initiate the acquisition process, and the image is transmitted to the computer. The computer saves the acquired images and determines if the phase calculation requirements are met. It waits until a complete sequence of images is acquired, performs image processing, and then determines if the set rotation requirements have been met. If further rotation is needed, a signal is sent to the stepper motor controller to control the electrically controlled rotating platform. After rotation is complete, a signal is sent back to the computer to repeat the above operations for automatic measurement.

[0065] I. Basic Principles of Phase-Shifting 3D Reconstruction:

[0066] The measuring device uses a four-step phase-shift method, and the acquired phase-shift modulation image is as follows:

[0067] I i (x, y)=I′(x, y)+I″(x, y)cos[φ(x, y)+δ i (1)

[0068] (1) In the formula I i (x, y) is the light intensity of the i-th phase-shifted image acquired by the camera, which is a known quantity; I′(x, y) is the background light intensity of the image; I″(x, y) is the modulated light intensity of the image, where φ(x, y) is the phase change caused by the three-dimensional shape of the object being measured, which is the value we need to calculate. i This is an added phase shift; this device uses a four-step phase shift, therefore:

[0069]

[0070] (2) N in the formula is the number of phase shift steps, and four phase shifts are N = 4.

[0071] Use the following formula to calculate the wrap phase:

[0072]

[0073] (3) In the formula, φ(x, y) is the wrapping phase value at image (x, y), I i (i = 0, 1, 2, 3) correspond to the four phase-shifted images in the four-step phase-shifting method.

[0074] The absolute phase value is obtained by unwrapping the phase using the multi-frequency heterodyne method. This absolute phase value can then be used to map the pixel coordinates from the camera's viewpoint to the projector's viewpoint. This method allows for projector calibration and the use of triangulation to perform 3D reconstruction of the object being measured.

[0075] II. Calibration of Rotation Axis and Coordinate Transformation

[0076] During the camera and projector calibration process, the coordinates of the calibration board's feature points, as well as the intrinsic and extrinsic parameters of the camera and projector, are obtained from the camera and projector's perspectives, respectively. This allows for 3D reconstruction of the calibration board's feature points. Since the captured images of the calibration board are rotated around the same axis, a series of 3D coordinate points around this axis can be obtained.

[0077] In this embodiment, the rotation axis can be calibrated as follows: First, these points are layered according to different heights. Each group of points obtained from the layering is distributed on a concentric circle perpendicular to the rotation axis, with the rotation axis located at its center. Therefore, to find the rotation axis, the center of the circle can be fitted to the points of each layer; the center of the circle is a point on the rotation axis. Several layers of points will result in several points on the rotation axis. Using these points to fit a straight line, the spatial equation of the rotation axis can be obtained. In this example, the random sample consensus algorithm is used for the fitting operation.

[0078] Knowing the rotation axis and rotation angle, we can perform coordinate transformations on a 3D point cloud, rotating it to the correct position. A rotation matrix is ​​constructed using the rotation axis and rotation angle. Multiplying the original coordinates by this rotation matrix (left-left) yields the coordinates rotated around the rotation axis by a specific angle. In this embodiment, the constructed rotation matrix is:

[0079]

[0080] Where K = 1 - cos(α), M = n x x0+n y y0+n z z0, n x ,n y ,n z Let x, y, and z be the x, y, and z direction vectors of the rotation axis, respectively, where x0, y0, and z0 are any points on the axis, and α is the rotation angle.

[0081]

[0082] Where (x, y, z) T Let (x′, y′, z′) be the original coordinates. T These are the coordinates after rotation.

[0083] III. Noise Filtering Methods Based on Rotational Characteristics

[0084] During the measurement process, due to image noise and the inability to completely extract the area where the measured object is located, the final 3D reconstruction result contains a considerable number of invalid 3D coordinate points. Traditional point cloud denoising uses relevant algorithms to filter out isolated outliers or points in specific regions. However, because the device of this invention requires rotating the measured object during measurement, the final point cloud results have a certain correlation, and the point cloud noise caused by the surrounding environment and random point cloud noise do not rotate with the rotation of the measured object. Utilizing this characteristic, an algorithm can be designed to filter out this point cloud noise that does not rotate with the rotation of the measured object.

[0085] Combination Figure 5 H angle measurements were performed on the object to obtain H rotated point cloud data sets, denoted as C. A KD-tree was created for each point cloud set to sort the points and quickly obtain the nearest neighbor search result. The i-th point in point cloud C is denoted as C[i]. Then another point cloud C' is taken... n Using nearest neighbor search in C' n Find the nearest neighbor of point C[i]. Set the judgment distance D, threshold condition K, and judgment number k. The distance between the two points is d. Then the judgment number can be obtained according to the following formula:

[0086]

[0087] Let the initial value of k be 0, for point P ij We should iterate through the M point clouds that overlap with it to obtain the final judgment number k. Compare k with K (1≤K≤M):

[0088]

[0089] Based on the judgment results, valid points are retained, and invalid points are deleted. Using the above method to filter each point in point cloud C yields a noise-filtered point cloud. The judgment distance D can be adaptively adjusted according to the point cloud resolution, and the threshold condition K is manually set; a larger K value results in stricter judgment of valid points.

[0090] IV. Actual Measurement Data

[0091] A measurement example is the measurement of a 70mm long metal block using the apparatus and method of this embodiment. Figure 6 As shown. Point cloud results obtained from single-sided measurements and multi-angle overlays on the same surface are compared, as shown... Figure 8 and Figure 9 As shown, the point cloud obtained after multi-angle measurements can compensate for overexposed areas, thus reducing the impact of overexposure. Multi-angle measurements were performed on a stone step model with complex textures, such as... Figure 7 As shown, the point cloud is processed using the denoising method of this invention. The point cloud results before and after denoising are as follows. Figure 10 and Figure 11 Before using the denoising method of this invention, traditional radius filtering and statistical filtering had already been performed, demonstrating that the point cloud denoising method in this embodiment can effectively remove high-density point cloud noise that is difficult to filter out using traditional methods. Both measurements were set to perform 24 angle measurements, meaning the rotary table rotated 15° after each angle measurement. The rotation filtering parameters were set as follows: number of point clouds M = 4, threshold condition K = 3, and judgment distance D = 0.1 mm. Figure 10 and Figure 11 The different colors in the image represent point clouds measured from different angles.

Claims

1. A multi-angle three-dimensional measuring device, characterized in that, include: A projection device used to project coded structured light onto the object being measured; Image acquisition device, used to acquire structured light images modulated by the object under test; A rigid connector is used to fix the projection device and the image acquisition device together, and to make their optical centers at the same height. The connector allows the projection device and the image acquisition device to change their horizontal viewing angle and to adjust their pitch angle as a whole. The two-dimensional displacement platform assembly consists of two mutually perpendicular electric displacement platforms, used to jointly adjust the horizontal and vertical distances of the connecting member and its projection device and image acquisition device relative to the object being measured. An electric rotary platform is used to carry and drive the object being measured to rotate about a single axis. The controller is electrically connected to the projection device, image acquisition device, two-dimensional displacement platform group and electric rotary platform, and is used to control the coordinated work of each component to realize automatic calibration and automatic multi-angle measurement.

2. The multi-angle three-dimensional measuring device according to claim 1, characterized in that, The projection device and the image acquisition device are connected by a trigger line, so that the image acquisition device is triggered to acquire an image every time the projection device projects an image.

3. The multi-angle three-dimensional measuring device according to claim 1, characterized in that, The controller is configured to, upon receiving a signal from the image acquisition device that it has completed the acquisition of an image sequence, control the electric rotating platform to rotate by a predetermined angle, and then start the next measurement cycle.

4. The multi-angle three-dimensional measuring device according to claim 1, characterized in that, The rigid connector includes a cover plate, a base plate, and a projection device bracket and an image acquisition device bracket that are movably clamped between the two. By tightening the screws and nuts between the base plate and the cover plate, the friction force on the bracket can be adjusted to lock and release its position.

5. The multi-angle three-dimensional measuring device according to claim 1, characterized in that, The horizontal and vertical movement ranges of the two-dimensional displacement platform group are both no less than 20 cm, and the rotation angle range of the electric rotating platform is 0 to 360 degrees.

6. A point cloud denoising method, applicable to point cloud results obtained by the three-dimensional measuring device of claim 1, characterized in that the method... include: (1) Input the target point cloud C to be denoised and M control point clouds that are from the same object as the target point cloud, collected around the same rotation axis but at different angles and have overlapping parts; (2) Establish KD-tree data structures for the target point cloud and each of the reference point clouds respectively; (3) For the point cloud that needs to be filtered, traverse the points P in it; (4) Find the unique nearest neighbor among its several neighboring point clouds; (5) Calculate the Euclidean distance d between point P and each of the nearest neighbors found; (6) Count the number of Euclidean distances d less than the preset threshold D, and record it as the number of successful matches k; (7) If the number of successful matches k is greater than or equal to the preset effective number threshold K, then the judgment point P is a valid point and is retained; otherwise, the judgment point P is a noise point and is removed. (8) Output all valid points that are retained to form a denoised point cloud.

7. The point cloud denoising method according to claim 6, characterized in that, The preset threshold D is adaptively set according to the single measurement accuracy of the three-dimensional measuring device, and its value ranges from 1 to 5 times the average spacing of the point cloud.

8. The point cloud denoising method according to claim 6, characterized in that, The effective number threshold K has a range of 1 ≤ K ≤ M, and the value of K is positively correlated with the strictness of denoising.

9. The point cloud denoising method according to claim 6, characterized in that, The input target point cloud C and all the reference point clouds have been unified to the same coordinate system through coordinate transformation based on the rotation axis and angle information obtained from the calibration.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 6-9.

Images