Automatic plane quadrilateral contour extraction and measurement method based on color scattered point cloud

Abstract

The invention provides an automatic plane quadrilateral contour extraction and measurement method based on colored scattered point clouds, and belongs to the technical field of point cloud feature segmentation and point cloud measurement. The method comprises the steps that a point cloud is segmented into a plurality of three-dimensional planes, all points of the three-dimensional planes are projected to the planes to form a two-dimensional image, contour extraction is carried out on the two-dimensional image, and the contour of the point cloud is extracted; fitting the extracted two-dimensional image contour to obtain two-dimensional line segments; and finally, reversely solving the two-dimensional line segments to calculate corresponding three-dimensional point cloud line segments. All adjacent edge pairs are screened out through angles and distances of every two straight lines, and quadrangles on the same plane are determined by utilizing the number of overlapped edges of every four adjacent edge pairs. And measuring the length and the width of the extracted plane quadrangle through the opposite side distance. According to the method, the planar quadrilateral contour in the three-dimensional space can be automatically extracted and measured by using the colored disordered point cloud, and the method can be suitable for various environment scenes, has relatively good flexibility and portability, and can provide convenience for subsequent different types of detection tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud feature segmentation and point cloud measurement, and in particular relates to an automatic measurement algorithm for extracting contour lines in point clouds, extracting quadrilaterals, segmenting them, and measuring length, width and side lengths. Background Art

[0002] With the continuous advancement of 3D scanning technology, the application of point cloud data has gradually penetrated into many fields such as engineering measurement, architectural modeling, manufacturing, and shipbuilding. Point cloud data can describe the spatial geometric information of the surface of an object in detail, and is widely used in tasks such as precise measurement of ship cabins, geometric analysis of building doors and windows, and plane contour detection of building walls. Therefore, how to extract accurate geometric shapes from point cloud data, especially the quadrilateral contours of the same plane in three-dimensional space, has become the key to automated modeling and measurement.

[0003] At present, the main methods for extracting and measuring plane quadrilaterals in point cloud data can be divided into several categories: methods based on image stitching, methods based on edge detection and line fitting, methods based on plane segmentation and projection, and methods based on deep learning. However, these methods face some challenges in practical applications. When automatically extracting plane quadrilaterals from point clouds, there are the following deficiencies: strong noise sensitivity, insufficient line recognition accuracy, large quadrilateral fitting errors, and limited automation.

[0004] This method mainly solves the problem of limited automation of the current automatic extraction and measurement of quadrilaterals in space. Most methods still require manual intervention when dealing with complex scenes, such as manually selecting initial points or setting threshold parameters such as patents, which reduces the degree of automation and increases the complexity and time cost of operation. In the measurement of cabins, complex cabin structures, limited operating space, and the curvature of the surface of objects increase the difficulty of point cloud data processing. In the measurement of building exterior walls and interior walls, factors such as reflection noise and material differences may also cause high data noise, increasing the complexity of subsequent quadrilateral recognition. For example, the patent CN202210403924.0, a cabinet recognition method based on laser point clouds, identifies the contour of the cabinet according to the ratio of the convex hull area corresponding to the contour point set to the area of ​​the minimum circumscribed rectangle. This method is only suitable for input point clouds that only contain rectangular cabinets, and is sensitive to noise, which affects the accurate calculation of the convex hull. In actual applications, the cabinet may be partially blocked, resulting in incomplete point cloud data and the need for manual supplementation or correction of the blocked part. Or the patent is a method for positioning and segmenting train carriages based on point cloud voxel rectangular grids CN202211030151.2. This method uses the rectangular rasterization of the plane point cloud to limit the coordinate values ​​to determine the positioning. This method relies on regular shapes and coordinate limitations, and is also susceptible to noise. It also requires manual processing of occlusion and missing data, and the algorithm has limited scalability. It may perform well when processing specific types of train carriage structures, but it is difficult to generalize to general planar quadrilateral recognition and segmentation tasks.

[0005] In summary, although existing methods have made some progress in segmenting and measuring the contour length of planar quadrilaterals in space in point clouds, there are still significant deficiencies in noise processing, recognition accuracy, fitting accuracy, degree of automation, and algorithm scalability. In industrial application scenarios, such as geometric measurement of cabins, doors, windows, and walls of buildings, automated segmentation and measurement still rely on a lot of manual intervention. Therefore, there is an urgent need for a more effective and highly automated method to achieve efficient and accurate automatic extraction and measurement of planar quadrilaterals, reduce manual intervention, and improve the processing efficiency of complex scenes. Summary of the invention

[0006] The main purpose of the present invention is to disclose a method for automatic extraction and measurement of planar quadrilateral contours based on colored scattered point clouds. The method can automatically extract and measure the planar quadrilateral contours in three-dimensional space using colored disordered point clouds. The method can be applied to a variety of environmental scenarios, has good flexibility and portability, and can provide convenience for subsequent different types of detection tasks.

[0007] In order to achieve the above-mentioned purpose and overcome the shortcomings of the existing methods, the technical solution adopted by the present invention mainly includes the following steps:

[0008] Step 1: Preprocess the input complete color disordered point cloud data. Screen the point cloud to remove discrete points, noise points, and invalid points without valid information, and filter out isolated points or abnormal points that cannot form a valid geometric structure with the main point group to improve the accuracy and stability of subsequent processing.

[0009] Step 2: Extract the three-dimensional line segments of the processed colored disordered point cloud.

[0010] Step 2.1: Use the region growing algorithm to segment the point cloud according to proximity and surface normal vectors to divide each connected region. These regions represent different three-dimensional planes. After that, the region merging method can be applied to merge adjacent and parallel planes based on the similarity between the planes to obtain multiple segmented point cloud planes.

[0011] Step 2.2: Next, orthogonally project all points of each plane onto the self-plane and convert it into a binary image through a gridding process based on the plane adaptive parameters. c , which is the average value of all its points. Project the first point in Π (denoted as P0) onto the plane, and denote the projection point as P′0. Set to the x-axis, whose direction is from P c Points to P′0. The positive direction of the x-axis is denoted by v x , and the positive direction of the y-axis can be calculated by v y =v x ×n Π Get. Using the center point P c , the positive direction of the x-axis and y-axis, the two-dimensional plane coordinates P of each point i (x i ,y i ) can be calculated by the following formula:

[0012]

[0013] Step 2.3: Then use other common image processing methods such as FindContours, Canny edge detection, Sobel operator, etc. in Opencv to extract the contour of the obtained image, and perform least squares fitting on the extracted image contour to obtain a two-dimensional line segment.

[0014] Step 2.4: By reversely solving the formula in step 2.2, these two-dimensional line segments are projected back to the original three-dimensional plane to obtain the corresponding three-dimensional point cloud line segments, and the adjacent line segments are merged to obtain the three-dimensional line segments of the colored disordered point cloud.

[0015] Step 3: Get the set of possible adjacent edge pairs of all lines. Calculate the direction of the line segment point cloud obtained in step 2, and get the line vector of each line fitting. Among all the extracted 3D line segments, select two lines to form a line pair. Traverse all possible line pairs in a combined manner to ensure that each pair of lines participates in the angle calculation. Traverse the two lines to calculate their angles. For any two lines d k and d l (where k≠l), calculate their angle θ k :

[0016]

[0017] where · represents the dot product of the vectors, ∥d k ∥ represents the norm of the vector. Determine the angle θ kl Is it within the vertical range, i.e. θ kl satisfy: As an adjustable factor, ε can be increased when the point cloud data is incomplete or the straight line fitting is inaccurate.

[0018] Calculate the distance between two straight lines, use any straight line point cloud in a set of opposite edge point clouds as the starting point cloud to establish a kdtree index, use kdtree to find the distance between each point, record the minimum distance and maximum distance and calculate the average value, and if the distance is within the set range, it is determined to be a pair of adjacent edges.

[0019] Step 4: Get a set of three-dimensional quadrilaterals. When the number of times that two pairs of adjacent edges in the set of adjacent edge pairs obtained in step 3 have overlapping straight lines is greater than or equal to a set value, these four adjacent edge pairs are recorded as a quadrilateral. Let S be the set of all adjacent edge pairs. For any four adjacent edges (e1, e2, e3, e4) ∈ S, define a function f(e1, e2, e3, e4) to represent the number of adjacent edge pairs with overlapping straight lines in these four adjacent edge pairs. If f(e1, e2, e3, e4) ≥ α, then we regard (e1, e2, e3, e4) as a quadrilateral, where α is an adjustable factor, α = 4 in a complete point cloud, and α = 3 in an incomplete point cloud. By traversing all possible combinations of (e1, e2, e3, e4), we can get the set of all quadrilaterals in three-dimensional space.

[0020] Step 5: Automatically measure the length, width and side length of the quadrilateral. The four straight line point clouds obtained in step 4 that can form a quadrilateral, each two parallel straight line point clouds are regarded as a pair of opposite sides, and the length and width can be obtained by calculating the average distance between the point clouds of the opposite side groups. The side length can be obtained by measuring and calculating the length of each straight line point cloud.

[0021] Due to the adoption of the above technical solution, the technical effects achieved by the present invention are as follows:

[0022] The present invention automatically extracts and measures based on colored disordered point clouds, can adapt to a variety of environmental scenarios, has strong flexibility and portability, can process different types of point cloud data, and solves the limitations of the prior art that are restricted by specific environments or hardware.

[0023] The present invention simplifies the complex processing process of three-dimensional point cloud data in traditional methods by extracting the contour of a two-dimensional image and fitting line segments, thereby improving the computing efficiency and reducing the consumption of computing resources.

[0024] The present invention greatly reduces manual intervention and improves data processing efficiency by automatically dividing the colored scattered point cloud data into multiple three-dimensional planes and projecting them onto a two-dimensional image for contour extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart for extracting the outline of a planar quadrilateral in three-dimensional space;

[0026] Figure 2 It is a schematic diagram of the process of transforming the three-dimensional plane point cloud of a ship into the two-dimensional plane coordinates;

[0027] Figure 3 It is a schematic diagram of the straight line contour of the conversion process in the ship point cloud;

[0028] Figure 4 This is a schematic diagram of the three-dimensional straight line contour extracted from the ship point cloud;

[0029] Figure 5 It is a schematic diagram of adjacent edge pairs;

[0030] Figure 6 This is a schematic diagram for calculating the distance between two straight lines;

[0031] Figure 7 It is a schematic diagram of the number of overlaps of adjacent edge pairs;

[0032] Figure 8 This is a schematic diagram of the plane quadrilateral extraction results in the ship point cloud;

[0033] Fig. 9 This is a flow chart of the method for automatic extraction and measurement of planar quadrilateral contours based on colored scattered point clouds in this application. DETAILED DESCRIPTION

[0034] The specific implementation method of the present invention is described in detail below with reference to the accompanying drawings.

[0035] In this embodiment, the process of automatically extracting the cabin frame and measuring the frame length using the complete ship color unordered point cloud will be described in detail. Figure 1As shown, the present invention mainly obtains multiple three-dimensional planes by region expansion and region merging, projects the obtained planes in three directions, extracts the two-dimensional contour features of the projection surface, and then fits them into a two-dimensional straight line to restore to the three-dimensional point cloud to obtain the point cloud straight line contour. The number of coincidences of adjacent edge pairs in the point cloud contour straight line is used as the judgment basis to extract the plane quadrilateral contour in the disordered point cloud, and finally achieve automatic measurement of the three-dimensional quadrilateral in the point cloud.

[0036] Step 1: Input the complete color disordered point cloud of the ship and preprocess it, remove the discrete points by using radius filtering and discrete point removal algorithm, and remove invalid points by using depth filtering or voxel grid filtering;

[0037] Step 2: Extract the three-dimensional line segments of the processed colored disordered point cloud.

[0038] Step 2.1: Segment the point cloud into multiple 3D planes through region expansion and region merging.

[0039] Step 2.2: Next, all points belonging to each plane are orthogonally projected onto the self-plane and converted into a binary image through a gridding process based on the plane adaptive parameters. Figure 3 (a). The coordinate transformation is as follows Figure 2 As shown. First, calculate the center point P of plane Π c , which is the average value of all its points. Project the first point in Π (denoted as P0) onto the plane, and denote the projection point as P′0. Set to the x-axis, whose direction is from P c Points to P′0. The positive direction of the x-axis is denoted by v x , and the positive direction of the y-axis can be calculated by v y =v x ×n Π Get. Using the center point P c , the positive direction of the x-axis and y-axis, the 2D plane coordinates P of each point i (x i ,y i ) can be calculated by the following formula:

[0040]

[0041] Step 2.3: Then use the FindContours function in Opencv to extract the contour of the obtained image, and perform least square fitting on the extracted image contour to obtain a two-dimensional line segment such as Figure 3 (b).

[0042] Step 2.4: By reversely solving the formula in step 2.2, these two-dimensional line segments are projected back to the original three-dimensional plane to obtain the corresponding three-dimensional point cloud line segments, and the adjacent line segments are merged to finally obtain the contour line segment point cloud extracted from the color point cloud as shown in Figure 3 As shown in (c), the schematic diagram of the 3D straight line contour extracted from the final ship point cloud is as follows Figure 4 shown.

[0043] Step 3: Obtain the set of adjacent edge pairs that all straight lines may form. Figure 5 As shown in Figure 1, when the point cloud data is complete and the fitting is good, a pair of point clouds that can be used as adjacent edges can be fitted as shown in Figure 1. The entire point cloud can be expressed as P i =(x i ,y i ,z i ), i = 1, 2, ..., N, for each point cloud line segment, the direction vector d of the line is fitted by various methods such as least squares method, RANSAC, etc. k , where k = 1, 2,…,M.

[0044] For any two straight lines d k and d l (where k≠l), calculate their angle θ k :

[0045]

[0046] where · represents the dot product of the vectors, ∥d k ∥ represents the norm of the vector. Determine the angle θ kl Is it within the vertical range, i.e. θ kl satisfy: The two lines are considered perpendicular. If the point cloud data is incomplete or the line fitting is inaccurate, the ε value can be increased.

[0047] Calculate the distance between two vertical lines. If the distance is less than 0.1 meters, it is determined as a pair of adjacent edges. The average distance process of the adjacent edge group is as follows: any straight line point cloud in a set of adjacent edge point clouds is used as the starting point cloud to establish a kdtree index, and the kdtree is used to find the distance between each point, record the minimum distance and the maximum distance, and calculate the average value. Figure 6 shown.

[0048] Step 4: Get a set of three-dimensional quadrilaterals. Figure 7In the figure a, (e1, e2) and (e3, e4) have no overlapping edges, that is, the number of overlaps is 0 and does not meet the condition; in the figure b, (e1, e2) and (e3, e4) have no overlapping edges, that is, the number of overlaps is 0 and does not meet the condition; and when the number of times that the adjacent edges of each of the four adjacent edge pairs in the obtained set of adjacent edge pairs overlap is greater than or equal to 3 or 4, as Figure 7 As shown in Figure c, these four adjacent edge pairs will definitely form a quadrilateral on the same plane in three-dimensional space. By traversing in this way, we can obtain a set of plane quadrilaterals in three-dimensional space. Finally, the set of plane quadrilaterals obtained in the ship is as follows Figure 8 .

[0049] Step 5: Automatically measure the length, width and side length of the quadrilateral. The four straight line point clouds obtained in step 4 that can form a quadrilateral are used as a pair of opposite side groups. The average distance between the point clouds of the opposite side groups is calculated to obtain the length and width. The length of each straight line point cloud is measured to obtain its side length. The accurate extraction and measurement of the outline of the planar quadrilateral in the point cloud is achieved.

[0050] This specification provides detailed implementation methods and specific operation processes based on the control scheme of the present invention, but the present invention is not limited to the above implementation methods, so the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for automatically extracting and measuring the outline of a plane quadrilateral based on a color scattered point cloud, characterized in that The following steps are involved: Step 1: Preprocess the input complete color scattered point cloud and remove discrete points and invalid points according to distance and voxel volume; Step 2: extract the three-dimensional line segments of the processed colored disordered point cloud; Step 3: Obtain the set of adjacent edge pairs that may be formed by all straight lines, calculate the direction of the straight line segment point cloud obtained in step 2, and obtain the straight line vector of each straight line fitting; traverse the straight lines in pairs to calculate their angles; the angles are judged to be perpendicular to each other within a certain area, and the distance between the two perpendicular straight lines is calculated. If the distance is within a certain threshold, it is judged to be a pair of adjacent edge pairs; Step 4: Obtain a set of all point cloud line segments that can form a plane quadrilateral in three-dimensional space; when the number of times that two adjacent edges of each of the four adjacent edge pairs in the set of adjacent edge pairs obtained in step 3 have overlapping straight lines is greater than or equal to α, these four adjacent edge pairs are recorded as a quadrilateral, and the set of all point cloud line segments that can form a quadrilateral in three-dimensional space is obtained by traversing in this way; Step 5: Automatically measure the side length of the quadrilateral; take the four straight line point clouds obtained in step 4 that can form a quadrilateral, and take each two parallel straight line point clouds as a pair of opposite sides, and calculate the average distance between the point clouds of the opposite side groups, so as to obtain the length and width of the quadrilateral in the case of missing point clouds.

2. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a colored scattered point cloud according to claim 1, characterized in that: The specific method of step 2 is to divide the point cloud into multiple three-dimensional planes through area expansion and area merging, orthogonally project all the points belonging to each plane onto the self-plane, and convert it into a binary image through a gridding process based on plane adaptive parameters, and then use a two-dimensional contour extraction algorithm to extract the contour of the obtained image, perform least squares fitting on the extracted image contour to obtain two-dimensional line segments, project these two-dimensional line segments back to the original three-dimensional plane to obtain the corresponding three-dimensional point cloud line segments, and merge adjacent line segments to finally obtain the contour line segment point cloud extracted from the color point cloud.

3. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a colored scattered point cloud according to claim 2, characterized in that The step 2 specifically includes the following steps: Step 2.1: Segment the point cloud into multiple 3D planes by region expansion and region merging; Step 2.2: Orthogonally project all points of each plane onto the self-plane and convert it into a binary image through a gridding process based on the plane adaptive parameters; use the center point P of the plane ∏ c , which is the average value of all its points; project the first point in ∏, denoted as P0, onto the plane, and the projected point is denoted as P′0. Set to the x-axis, whose direction is from P c Points to P′0; the positive direction of the x-axis is denoted by v x , and the positive direction of the y-axis can be calculated by v y =v x ×n ∏ Get; using the center point P c , the positive direction of the x-axis and y-axis, the two-dimensional plane coordinates P of each point i (x i ,y i ) can be calculated by the following formula: Step 2.3: Use common image processing functions to extract the contour of the obtained image, and perform least square fitting on the extracted image contour to obtain a two-dimensional line segment; Step 2.4: By reversely solving the formula in step 2.2, these two-dimensional line segments are projected back to the original three-dimensional plane to obtain the corresponding three-dimensional point cloud line segments, and the adjacent line segments are merged to obtain the three-dimensional line segments of the colored disordered point cloud.

4. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a colored scattered point cloud according to claim 1, characterized in that The step 3 of obtaining the adjacent edge pairs specifically includes the following steps: The entire point cloud can be expressed as P i =(x i ,y i , z i ), i = 1, 2, ..., N, for each point cloud straight line segment, the direction vector d of the straight line is fitted or calculated by various methods k , where k = 1, 2, ..., M; calculate the angle between any two straight lines. k and d l , where k≠l, calculate their angle θ k : where · represents the dot product of the vectors, ||d k || represents the norm of the vector; determines the angle θ kl Is it within the vertical range, i.e. θ kl satisfy: As an adjustable factor, ε can be increased when the point cloud data is incomplete or the straight line fitting is inaccurate.

5. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a colored scattered point cloud according to claim 4, characterized in that The specific process of calculating the straight line point cloud distance in step 3 includes the following steps: Step 3.1: Take any point in the point cloud as P0 and its coordinates are P0 = (x0, y0, z0); Step 3.2: Find the point P1 that is farthest from P0 in Euclidean distance. For all points P in the point cloud i =(x i ,y i , z i ), calculate their Euclidean distance from P0: Step 3.3: Find the i0 The largest point P1, its coordinates are P1 = (x1, y1, z1); Step 3.4: Find the point P2 that is farthest from P1 in Euclidean distance; similarly calculate the Euclidean distance between all points in the point cloud and P0, and find the point P2 that has the largest distance, whose coordinates are P2 = (x2, y2, z2); the distance between point P1 and point P2 is the length of this straight line point cloud.

6. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a color scattered point cloud according to claim 1, characterized in that: The process of judging whether the four straight line point clouds form a quadrilateral in step 4 is as follows: Let S be the set of all adjacent edge pairs. For any four adjacent edge pairs (e1, e2, e3, e4)∈S, define a function f(e1, e2, e3, e4) to represent the number of adjacent edge pairs with coincident straight lines among these four adjacent edge pairs. If f(e1, e2, e3, e4)≥α, then (e1, e2, e3, e4) is regarded as a quadrilateral, where α is an adjustable factor. In a complete point cloud, α=4, and in an incomplete point cloud, α=3. By traversing all possible combinations of (e1, e2, e3, e4), the set of all quadrilaterals in three-dimensional space can be obtained.

7. The method for automatically extracting and measuring the outline of a plane quadrilateral based on a colored scattered point cloud according to claim 1, characterized in that: The process of calculating the average distance of the opposite edge group in step 5 is as follows: any straight line point cloud in a set of opposite edge point clouds is used as the starting point cloud to establish kdtre e Index, using kdtre e Find the distances between the respective points, record the minimum and maximum distances and calculate the average.

Images