A multi-camera laser point cloud splicing automation deduplication method

Abstract

The application discloses a multi-camera laser point cloud splicing automatic deduplication method, acquires laser source point cloud and target point cloud of multi-camera after splicing; carries out filtering processing on the acquired laser source point cloud and target point cloud; after filtering processing, saves the point cloud of the overlapping area of the laser source point cloud and the target point cloud; based on the saved point cloud of the overlapping area, preliminarily removes the point cloud of the overlapping area of the laser source point cloud, completes coarse deduplication area; carries out outlier deduplication on the preliminarily deduplicated laser source point cloud to complete fine deduplication, and adds the target point cloud to complete the deduplication operation of the multi-camera laser point cloud splicing. The application solves the error caused by calibration in the traditional splicing deduplication, thereby effectively improving the deduplication precision, guarantees the efficiency of the point cloud splicing deduplication, and reduces the processing complexity of the three-dimensional reconstruction and measurement related data of the point cloud.

Description

Technical Field

[0001] This invention relates to an automated deduplication method for multi-camera laser point cloud stitching. Background Technology

[0002] In highly automated industrial production, acquiring point cloud data of equipment workpieces and components is fundamental for measuring them and determining their quality. For large workpieces, a single camera cannot capture the data for the entire workpiece; therefore, point cloud data from multiple cameras must be combined to obtain the overall point cloud. In this process, deduplicating the stitched point cloud data is a crucial step.

[0003] Currently, multi-camera laser point cloud stitching deduplication methods involve manual data processing. This typically involves calibrating multiple cameras using calibration boards and blocks to calculate their positional relationships, stitching the point cloud data together, and then converting the acquired point cloud image into a 2D depth map to calculate the overlapping portions of the stitched point cloud. Based on the previous calibration, the coordinate relationships of the overlapping positions can be calculated, and deduplication is then performed on these overlapping areas. This entire process is time-consuming, labor-intensive, and lacks automation. Human fatigue and inherent calibration errors contribute to low efficiency and accuracy in point cloud deduplication. Ultimately, this results in unreliable reconstruction and measurement results from the stitched point cloud data. Summary of the Invention

[0004] This invention provides an automated deduplication method for multi-camera laser point cloud stitching to address the problems existing in the prior art.

[0005] The technical solutions adopted in this invention are as follows:

[0006] An automated deduplication method for multi-camera laser point cloud stitching, including

[0007] S10: Obtain the laser source point cloud and target point cloud from the stitched multi-camera setup;

[0008] S20: Filter the acquired laser source point cloud and target point cloud;

[0009] S30: After filtering, save the point cloud of the overlapping area between the laser source point cloud and the target point cloud;

[0010] S40: Based on the saved point cloud of the overlapping region, the point cloud that overlaps with the laser source point cloud is initially removed to complete the coarse deduplication.

[0011] S50: Performs outlier deduplication on the initially deduplicated laser source point cloud to complete fine deduplication, and adds it to the target point cloud to complete the deduplication operation of the multi-camera laser point cloud stitching.

[0012] Further, in step S20, the laser source point cloud and target point cloud acquired by the multiple cameras are filtered using a pass-through filtering method. The pass-through filtering method is as follows:

[0013] S201: For both the laser source point cloud and the target point cloud, set the filtering field to the X-axis direction, select an appropriate filtering range based on the focal length and resolution of the corresponding camera, and set the minimum filtering range to x_0 and the maximum range to x_1.

[0014] S202: Repeat step S201, and set the range for the filter field in the Y-axis direction and the filter field in the Z-axis direction, which are (y_0, y1) and (z_0, z_1).

[0015] S203: Traverse the entire laser source point cloud and target point cloud. If each point in the laser source point cloud and target point cloud is within the corresponding range of (x_0,x_1), (y_0,y1), (z_0,z_1), then save the point; otherwise, discard the point.

[0016] Further, in step S30, the distance between the midpoint of the laser source point cloud and the midpoint of the target point cloud is compared with a set radius threshold. If the distance between the two corresponding points is less than the set radius threshold, the two points are considered to be overlapping points, thereby finding the overlap range between the laser source point cloud and the target point cloud; more specifically:

[0017] S301: Establish an octree data structure for the laser source point cloud to store the filtered laser source point cloud data;

[0018] S302: Traverse all points in the target point cloud. Since it is necessary to determine whether the laser source point cloud and the target point cloud are overlapping points, a radius threshold is set. Then, a circle is drawn with each point in the laser source point cloud as the center and the threshold radius. If there is a point in the target point cloud that falls into the circle with the threshold radius, then that point is an overlapping point cloud with the laser source point cloud.

[0019] S303: Similarly, repeat steps S301 and S302 to obtain the point cloud that overlaps with the target point cloud.

[0020] Further, in step S40, the coarse deduplication step of the laser source point cloud and the overlapping region point cloud is as follows:

[0021] S401: Obtain the point cloud of the overlapping region;

[0022] S402: Classify the overlapping region point cloud, laser source point cloud and target point cloud by constructing a spatial binary tree, and then set the deduplication comparison value d1;

[0023] S403: For the laser source point cloud, draw a circle with each point in the overlapping region point cloud as the center and the comparison value d1 as the radius. If a point in the laser source point cloud falls within the circle with radius d1, then mark the index of the point in the laser source point cloud that falls within the circle with radius d1.

[0024] S405: Remove all points marked with indexes from step S403 to complete the coarse deduplication.

[0025] Furthermore, in step S50, the step of performing outlier removal on the initially deduplicated laser source point cloud to complete the fine deduplication is as follows:

[0026] S501: The deduplicated laser source point cloud is classified again by constructing a spatial binary tree, and the neighborhood radius r_0 is set according to the resolution of the camera corresponding to the laser source point cloud.

[0027] S502: Using each point in the deduplicated laser source point cloud as the center and r_0 as the radius, set the neighborhood of each point, set the distance threshold dmax between two points in each neighborhood, calculate the distance di between all points in each neighborhood, and then calculate the mean u and standard deviation Q of all distances.

[0028] S503: In each neighborhood, arbitrarily select a point P as a comparison point, and compare the di between all other points in the neighborhood and point P with the threshold dmax. If the di is less than the threshold dmax, select the next point, and so on until all points in the neighborhood are traversed. If there is a point greater than the threshold dmax, then the point is determined to be an outlier and removed.

[0029] S504: Repeat step S503, traverse all points in all domains, and complete fine deduplication;

[0030] S505: Obtain the deduplicated point cloud data and then add it to the target point cloud to complete the deduplication operation of multi-camera laser point cloud stitching.

[0031] The present invention has the following beneficial effects:

[0032] This invention solves the problem of calibration errors that are unavoidable in traditional point cloud deduplication, thereby effectively improving deduplication accuracy. While ensuring the efficiency of point cloud deduplication, it also reduces the processing complexity of subsequent 3D reconstruction and measurement data of point clouds. Attached Figure Description

[0033] Figure 1 This is a flowchart of the present invention.

[0034] Figure 2 This is a flowchart of the coarse deduplication process.

[0035] Figure 3 Flowchart for deduplication.

[0036] Figure 4 This is a schematic diagram of the coarse deduplication theory.

[0037] Figure 5a , Figure 5b and Figure 5c These are partial images of the tire taken by three separate cameras.

[0038] Figure 5d This is a picture of the assembled car tires. Detailed Implementation

[0039] The invention will now be further described with reference to the accompanying drawings.

[0040] like Figures 1 to 3 This invention provides an automated deduplication method for multi-camera laser point cloud stitching, comprising:

[0041] S10: Obtain the laser source point cloud and target point cloud from the stitched multi-camera setup;

[0042] S20: Filter the acquired laser source point cloud and target point cloud;

[0043] S30: After filtering, save the point cloud of the overlapping area between the laser source point cloud and the target point cloud;

[0044] S40: Based on the saved point cloud of the overlapping region, the point cloud that overlaps with the laser source point cloud is initially removed to complete the coarse deduplication.

[0045] S50: Performs outlier deduplication on the initially deduplicated laser source point cloud to complete fine deduplication, and adds it to the target point cloud to complete the deduplication operation of the multi-camera laser point cloud stitching.

[0046] In step S20, due to factors such as the industrial setting and lighting conditions, a pass-through filtering method is used to denoise the laser source point cloud and the target point cloud. Specifically:

[0047] S201: For both the laser source point cloud and the target point cloud, set the filtering field to the X-axis direction, select an appropriate filtering range based on the focal length and resolution of the corresponding camera, and set the minimum filtering range to x_0 and the maximum range to x_1.

[0048] S202: Repeat step S201, and set the range for the filter field in the Y-axis direction and the filter field in the Z-axis direction, which are (y_0, y1) and (z_0, z_1).

[0049] S203: Traverse the entire laser source point cloud and target point cloud. If each point is within (x_0, x_1) and (y_0, y1),

[0050] If the point is within the range of (z_0, z_1), then save the point; otherwise, discard the point.

[0051] In the denoising process of this example, the filtering ranges in the x, y, and z directions are set according to the resolution and focal length of the 3D camera used.

[0052] In this example, the deduplication range is set to (-50, 50) for the X-axis direction, (-120, 120) for the Y-axis direction, and (-50, 50) for the Z-axis direction. The pass-through filter range is then set to (-50, 50), (-120, 120), and (-50, 50) respectively.

[0053] In step S30: Given the large volume of point cloud data, it is necessary to classify the data structure. A point cloud octree data structure is used to classify the collected point clouds. By determining the number of points in the neighborhood, the point cloud data in overlapping areas can be searched, including:

[0054] S301: Establish an octree data structure for the laser source point cloud to store the filtered laser source point cloud data;

[0055] S302: Traverse all points in the target point cloud. Since it is necessary to determine whether a point is a duplicate, the neighborhood radius is set to be as small as possible. For example, based on the tire point cloud data, the neighborhood radius is set to 0.01mm. Query whether there is a point cloud in the corresponding laser source point cloud voxel (i.e., determine whether the laser source point cloud and the target point cloud exist simultaneously in the neighborhood radius of 0.01mm). If they exist, then the point is an overlapping point cloud of the target point cloud.

[0056] S303: Similarly, repeat step S301 to build an octree of the point cloud, store the target point cloud, and then repeat step S202 to obtain the overlapping point cloud of the laser source point cloud.

[0057] In step S40, based on the saved overlapping region point cloud, the point cloud coinciding with the laser source point cloud and the overlapping region is initially removed to complete the coarse deduplication, specifically:

[0058] S401: Obtain the point cloud of the overlapping region;

[0059] S402: Classify the overlapping region point cloud, laser source point cloud and target point cloud by constructing a spatial binary tree, and then set the deduplication comparison value d1;

[0060] S403: For the laser source point cloud, draw a circle with each point in the overlapping region point cloud as the center and the comparison value d1 as the radius. If a point in the laser source point cloud falls within the circle with radius d1, then mark the index of the point in the laser source point cloud that falls within the circle with radius d1.

[0061] like Figure 4Taking an ideal scenario as an example, the region formed by the laser source point cloud and the region formed by the target point cloud are two circles, and their intersection is the overlapping region. During coarse deduplication, each point within the overlapping region point cloud is taken as the center of the circle (e.g., ...). Figure 4 In the process, draw a circle with each point in the intersection as the center, and then find the points in the laser source point cloud that fall within the circle with radius d1.

[0062] S405: Remove all points marked with indexes from step S403 to complete the coarse deduplication.

[0063] The laser source point cloud initially deduplicated in step S50 will not have very high accuracy due to consideration of processing efficiency. Therefore, outlier deduplication is required to complete fine deduplication. Finally, it is added to the target point cloud to complete the deduplication operation for stitching large-field-of-view laser point clouds, including:

[0064] S501: Obtain the coarsely deduplicated origin point cloud. Due to its data structure, there are still some point cloud data that are not completely deduplicated. Then, the laser source point cloud after deduplication is classified again by constructing a spatial binary tree. The neighborhood radius size r_0 is set according to the resolution of the camera corresponding to the laser source point cloud.

[0065] S502: Using each point in the deduplicated laser source point cloud as the center and r_0 as the radius, set the neighborhood of each point, set the distance threshold dmax between two points in each neighborhood, calculate the distance di between all points in each neighborhood, and then calculate the mean u and standard deviation Q of all distances.

[0066] S503: In each neighborhood, arbitrarily select a point P as a comparison point, and compare the di between all other points in the neighborhood and point P with the threshold dmax. If the di is less than the threshold dmax, select the next point, and so on until all points in the neighborhood are traversed. If there is a point greater than the threshold dmax, then the point is determined to be an outlier and removed.

[0067] S504: Repeat step S503, traverse all points in all domains, and complete fine deduplication;

[0068] S505: Obtain the deduplicated point cloud data and then add it to the target point cloud to complete the deduplication operation of multi-camera laser point cloud stitching.

[0069] Three industrial 3D cameras are used to acquire point cloud data for deduplication. After acquisition, the point cloud data is converted into editable files in relevant point cloud data formats (txt, pcd, etc.). For example, commercial software such as Halcon (machine vision algorithm package), open-source libraries such as OpenCV (image processing algorithm library), and PCL (Point Cloud Library) can be used to process the point cloud data.

[0070] This invention can be designed using C++ programming. The graphical interface required by the user is designed in the QT application framework. In order to facilitate operation and observation, a human-computer interaction interface is set up. This interface outputs the deduplicated point cloud data and whether it is successful or not, and visualizes the point cloud data before and after deduplication.

[0071] Take car tires as an example ( Figure 5d (This is a stitched image of the car tires). Point cloud data of the car tires to be deduplicated was acquired using an industrial 3D camera. Figure 5a , Figure 5b and Figure 5c These are partial images of the tire taken by three cameras. The deduplicated point cloud data, processed using the method of this invention, is displayed on the human-computer interface as follows:

[0072] Camera point cloud count to be deduplicated: 3;

[0073] Number of point clouds to be deduplicated and stitched: Point cloud 1: 766183; Point cloud 2: 861752; Point cloud 3: 798702.

[0074] Coarse de-emphasis cloud count: Region 1: 412756, Region 2: 395742. Time required: 10.535s;

[0075] Number of key clouds removed: Region 1: 17041, Region 2: 19086; Time required: 5.354s;

[0076] Number of deduplicated and stitched point clouds: 1,575,636.

[0077] The above point cloud 1 is Figure 5a The point cloud corresponding to the camera; point cloud 2 is Figure 5b The point cloud corresponding to the camera; point cloud 3 is Figure 5c The point cloud corresponding to the camera.

[0078] Area 1 is Figure 5a and Figure 5b Overlapping region; Region 2 is Figure 5b and Figure 5c Overlapping areas.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. An automated deduplication method for multi-camera laser point cloud stitching, characterized in that: include S10: Obtain the laser source point cloud and target point cloud from the stitched multi-camera setup; S20: Filter the acquired laser source point cloud and target point cloud; S30: After filtering, save the point cloud of the overlapping area between the laser source point cloud and the target point cloud; S40: Based on the saved point cloud of the overlapping region, the point cloud that overlaps with the laser source point cloud is initially removed to complete the coarse deduplication. S50: Perform outlier deduplication on the initially deduplicated laser source point cloud to complete fine deduplication, and add it to the target point cloud to complete the deduplication operation of the multi-camera laser point cloud stitching. In step S40, the coarse deduplication step of the laser source point cloud and the overlapping region point cloud is as follows: S401: Obtain the point cloud of the overlapping region; S402: Classify the overlapping region point cloud, laser source point cloud and target point cloud by constructing a spatial binary tree, and then set the deduplication comparison value d1; S403: For the laser source point cloud, draw a circle with each point in the overlapping region point cloud as the center and the comparison value d1 as the radius. If a point in the laser source point cloud falls within the circle with radius d1, then mark the index of the point in the laser source point cloud that falls within the circle with radius d1. S405: Remove all points marked with indexes in step S403 to complete the coarse deduplication; In step S50, the steps for refining the laser source point cloud by removing outliers from the initially deduplicated point cloud are as follows: S501: The deduplicated laser source point cloud is classified again by constructing a spatial binary tree, and the neighborhood radius r_0 is set according to the resolution of the camera corresponding to the laser source point cloud. S502: Using each point in the deduplicated laser source point cloud as the center and r_0 as the radius, set the neighborhood of each point, set the distance threshold dmax between two points in each neighborhood, calculate the distance di between all points in each neighborhood, and then calculate the mean u and standard deviation Q of all distances. S503: In each neighborhood, arbitrarily select a point P as a comparison point, and compare the di between all other points in the neighborhood and point P with the threshold dmax. If the di is less than the threshold dmax, select the next point, and so on until all points in the neighborhood are traversed. If there is a point greater than the threshold dmax, then the point is determined to be an outlier and removed. S504: Repeat step S503, traverse all points in all domains, and complete fine deduplication; S505: Obtain the deduplicated point cloud data and then add it to the target point cloud to complete the deduplication operation of multi-camera laser point cloud stitching.

2. The automated deduplication method for multi-camera laser point cloud stitching as described in claim 1, characterized in that: In step S20, the laser source point cloud and target point cloud acquired by multiple cameras are filtered using a pass-through filtering method. The pass-through filtering method is as follows: S201: For both the laser source point cloud and the target point cloud, set the filtering field to the X-axis direction, select an appropriate filtering range based on the focal length and resolution of the corresponding camera, and set the minimum filtering range to x_0 and the maximum range to x_1. S202: Repeat step S201, and set the range for the filter field in the Y-axis direction and the filter field in the Z-axis direction, which are (y_0, y1) and (z_0, z_1). S203: Traverse the entire laser source point cloud and target point cloud. If each point in the laser source point cloud and target point cloud is within the corresponding range of (x_0,x_1), (y_0,y1), (z_0,z_1), then save the point; otherwise, discard the point.

3. The automated deduplication method for multi-camera laser point cloud stitching as described in claim 1, characterized in that: In step S30, the distance between the midpoint of the laser source point cloud and the midpoint of the target point cloud is compared with a set radius threshold. If the distance between the two points is less than the set radius threshold, the two points are considered to be overlapping points, thereby finding the overlapping range of the laser source point cloud and the target point cloud.

Images