Point cloud-based tunnel trend recognition method, computer equipment and storage media

Abstract

This invention discloses a tunnel trend recognition method, computer equipment, and storage medium based on point clouds. It processes tunnel point clouds acquired by lidar, first calculating the tunnel trend in both top and side views, and then filtering the point cloud in three-dimensional space based on the vehicle's travel clearance. The point cloud can effectively fit the tunnel trend, allowing real-time acquisition of the tunnel's direction at the current location. Based on this, the tunnel direction can assist trains in point cloud filtering, removing interference from non-obstacles such as tunnel walls and signal lights, providing better preprocessing for obstacle detection.

Description

Technical Field

[0001] This invention belongs to the field of tunnel point cloud and image processing technology, specifically involving a technology for calculating tunnel trends by combining point clouds and images. Background Technology

[0002] LiDAR can detect and perceive the 3D environment. Due to its advantages such as high ranging accuracy, strong directionality, fast response, and immunity to ground clutter, and its ability to effectively provide the information required by vehicle decision-making and control systems, it is the core guarantee for realizing autonomous driving and is now widely used in autonomous driving.

[0003] Accurate obstacle detection is a crucial step in safe driving. Directly detecting obstacles from the point cloud scanned by LiDAR may result in over- or under-detection. Therefore, it is necessary to filter the tunnel point cloud and extract the Regions of Interest (ROIs) that need to be detected. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a tunnel trend recognition method based on point cloud to achieve accurate recognition of tunnel trends.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] On the one hand, a tunnel trend recognition method based on point clouds is provided, which processes tunnel point clouds collected by lidar, including the following steps:

[0007] 1) Project the point cloud onto the yox and yoz planes respectively to obtain the top view and side view of the point cloud. The y direction is the direction of train travel in the tunnel, the x direction is the perpendicular direction of train travel and is parallel to the ground, and the z direction is the perpendicular direction of the ground.

[0008] 2) Using an edge extraction algorithm, extract the boundary points of the point cloud in the top view and side view, and denote them as point sets points1 and points2, respectively;

[0009] 3) Remove outliers from points1 and points2;

[0010] 4) For the contour formed by the point set points1, determine whether the current frame point cloud is a fork point cloud by using the width threshold and the branch number threshold. If it is a fork, proceed to step 5); otherwise, proceed to step 6.

[0011] 5) Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The left and right division is based on the central axis of the tunnel length direction. Perform curve fitting on the two sets of points1_left and points1_right to obtain the tunnel trend in the top view.

[0012] 6) Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The division between left and right is based on the central axis of the tunnel length direction. Calculate the difference in the number of points and the variance of the two point sets points1_left and points1_right. Based on the point count threshold and the variance threshold, determine whether to select points1_left or points1_right for curve fitting to obtain the tunnel trend in the top view direction.

[0013] 7) Divide the boundary points in points2 into points closer to the ground and points farther from the ground. Select the points closer to the ground for curve fitting to obtain the tunnel trend in the side view direction.

[0014] 8) Combine the two-dimensional curves from the top and side views to obtain the curve equation in three-dimensional space.

[0015] Preferably, the point cloud boundary point extraction algorithm in step 2) includes the following steps:

[0016] (21) Determine the radius r based on the angular resolution of the lidar;

[0017] (22) Select any point p in the point cloud and count the points in the point cloud that are within a distance of less than 2r from point p;

[0018] (23) Select any point q in the point set Points. Given p, q and r, calculate the center p_center. There are two centers that meet the conditions, denoted as p_center1(x1, y1) and p_center2(x2, y2). The formula for calculating the center is as follows:

[0019]

[0020]

[0021]

[0022]

[0023] in

[0024]

[0025] S 2 =(p x -q x ) 2 +(p y -q y ) 2 ;

[0026] (24) Calculate the distance from the remaining points in Points other than points p and q to the center p_center. If all distances are greater than r, then p is a boundary point. If none of the distances are greater than r, then replace point q in Points and continue to execute steps (23) and (24). If none of the points in Points meet the conditions of step (24), then point p is a non-boundary point.

[0027] Preferably, the point set of the point cloud boundary obtained in step 2) is converted into a depth map image, thereby obtaining ordered pixels. When converting into a depth map image, for the point cloud in the top view direction, the x direction of the image corresponds to the y direction of the point cloud, and the y direction of the image corresponds to the x direction of the point cloud; for the point cloud in the side view direction, the x direction of the image corresponds to the y direction of the point cloud, and the y direction of the image corresponds to the z direction of the point cloud. In the depth map, the pixel obtained through the point cloud is 255, otherwise it is 0.

[0028] The depth map is grown to obtain a continuous closed region of interest. Along the x-direction of the image, the derivative of each column of pixels is calculated and stored in the matrix Kij (i = 0, 1, 2... rows - 1, j = 0, 1, 2,... cols), where rows is the number of rows in the depth map and cols is the number of columns in the depth map.

[0029] Store the first non-zero derivative of each column of kij into array upArray, the last non-zero derivative of each column of kij into array downArray, and the column number of each column of kij containing exactly four non-zero derivatives into array colArray. Calculate the second derivatives of upArray and downArray respectively, save the points whose second derivatives are within the threshold range, and remove the points outside the threshold, thereby achieving filtering of abrupt change points.

[0030] If the depth map is a top-down depth map, count the length of continuous data in colArray. If the length is greater than the threshold Threshold2 and the difference between the row number of the last non-zero derivative and the row number of the first non-zero derivative in the same column of Kij is greater than the threshold Threshold3, then the point cloud of the current frame is considered to be the point cloud at the intersection.

[0031] The preferred mathematical model for two-dimensional polynomial fitting is as follows: Steps 5), 6), and 7) use the ransac algorithm for polynomial fitting, as follows:

[0032] (1) Select 4 points for polynomial fitting, and denote the parameter as a. i (i = 0, 1, 2, 3);

[0033] (2) Calculate the distance from the point to be fitted to the curve as the error d. If d is greater than the distance threshold T, then the point is an inlier; otherwise, it is an outlier.

[0034] (3) Repeat step (2) until all fitted points have been traversed;

[0035] (4) Select the parameter a with the largest number of interior points. i (i = 0, 1, 2, 3) are the parameters of the fitted cubic polynomial equation.

[0036] Preferably, in step 8), the two-dimensional curve fittings from the top and side views are combined to obtain the three-dimensional spatial curve equation expression.

[0037]

[0038] In another aspect, the present invention also provides a computer device, including at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to execute the aforementioned point cloud-based tunnel trend recognition method.

[0039] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the point cloud-based tunnel trend recognition method.

[0040] The technical solution adopted in this invention first calculates the tunnel trend in both top and side views, and then filters the point cloud in three-dimensional space according to the vehicle's driving clearance range. This has the following beneficial effects:

[0041] 1. Point clouds can be well fitted to tunnel trends and the tunnel direction at the current location can be obtained in real time. Based on this, the tunnel direction can be used to assist trains in point cloud filtering, removing interference from non-obstacles such as tunnel walls and signal lights, thus providing better preprocessing for obstacle detection.

[0042] 2. This invention removes outliers and determines whether a point cloud is a junction by converting it into an image. This transforms the disordered point cloud into ordered pixels, solving the problem of sorting the disordered point cloud when distinguishing upper and lower boundaries. This method avoids the sorting problem caused by the disorder of the point cloud, thus simplifying the calculation process.

[0043] The specific technical solutions adopted in this invention and their beneficial effects will be disclosed in detail in the following specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0044] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0045] Figure 1 This is a schematic diagram of the projection of point clouds into top and side views. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of the present invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of the present invention.

[0047] Example 1

[0048] The technical problem this invention aims to solve is tunnel trend identification. This embodiment provides a point cloud-based tunnel trend identification method. It processes tunnel point clouds collected by lidar, first calculating the tunnel trend in both top and side views, and then filtering the point cloud in three-dimensional space based on the vehicle's driving clearance. The method includes the following steps:

[0049] 1) such as Figure 1 As shown, the point cloud is projected onto the yox and yoz planes respectively, resulting in the top and side views of the point cloud.

[0050] 2) Using an edge extraction algorithm, extract the boundary points of the point cloud in the top view and side view, and denote them as point sets points1 and points2, respectively;

[0051] 3) Remove outliers from points1 and points2;

[0052] 4) For the contour formed by the point set points1, determine whether the current frame point cloud is a fork point cloud by using the width threshold and the branch number threshold. If it is a fork, proceed to step 5); otherwise, proceed to step 6.

[0053] 5) Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The division of left and right is based on the central axis of the tunnel length direction. The left and right boundary points at the junction are the left and right boundaries of the two tracks, respectively. Therefore, perform curve fitting on the two sets of point sets points1_left and points1_right to obtain two sets of curve equations. These two sets of curve equations are the tunnel trend in the top view direction of the two tracks at the junction.

[0054] 6) For non-intersection point clouds, the left and right boundaries are the boundaries of the same track. Therefore, it is necessary to select a boundary with a large number of points for curve fitting. Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The left and right division is based on the central axis of the tunnel length direction. Calculate the difference in the number of points and the variance of the two point sets points1_left and points1_right. Compare the number of points and the variance of the two point sets, and select the point set with the smaller variance and more points for curve fitting to obtain the tunnel trend in the top view.

[0055] 7) Based on the side-view projection direction, use the ground filtering algorithm to extract the set of points close to the ground in points2 and perform curve fitting to obtain the tunnel trend in the side-view direction;

[0056] 8) Combine the two-dimensional curves from the top and side views to obtain the curve equation in three-dimensional space.

[0057] The steps of point cloud projection, point cloud boundary extraction, outlier removal and junction detection, 2D curve fitting, and 3D curve fitting are as follows:

[0058] Point cloud projection

[0059] like Figure 1 As shown, the y-direction is the direction of train travel in the tunnel, the x-direction is perpendicular to the train's direction of travel and parallel to the ground, and the z-direction is perpendicular to the ground. Projecting the point cloud of the current frame into top and side views yields the point clouds in the yox and yoz planes, respectively.

[0060] Point cloud boundary extraction

[0061] Edge point cloud extraction is performed on the projected point cloud. Due to the curvature of the tunnel, a concave hull algorithm is used to extract boundary points. In this embodiment, the alphashape algorithm is used for concave hull extraction of the point cloud. The parameter α is set according to the sampling interval of the LiDAR, and α is continuously adjusted according to the actual situation to extract the boundaries of the point cloud. The specific algorithm flow is as follows:

[0062] 1. Determine the parameter r in the alphashape algorithm based on the angular resolution of the actual lidar being used;

[0063] 2. Select any point p in the point cloud, and count the points in the point cloud that are within a distance of less than 2r from point p;

[0064] 3. Select any point q in the point set Points. Given p, q, and r, calculate the center p_center. There are two centers that meet the conditions, denoted as p_center1(x1, y1) and p_center2(x2, y2). The formula for calculating the center is as follows:

[0065]

[0066]

[0067]

[0068]

[0069] in

[0070]

[0071] S 2 =(p x -q x ) 2 +(p y -q y ) 2 ;

[0072] 4. Calculate the distances from all points in Points other than p and q to the center p_center. If all distances are greater than r, then p is a boundary point. If not all distances are greater than r, then replace point q in Points and continue with steps 3 and 4. If none of the points in Points satisfy the condition in step 4, then point p is a non-boundary point.

[0073] 5. Anomaly removal and junction identification

[0074] Based on step 2), the point set of the point cloud boundary can be obtained. This boundary point set is then converted into a depth map, resulting in an ordered arrangement of pixels. When converting to an image, for a point cloud viewed from above, the x-direction of the image corresponds to the y-direction of the point cloud, and vice versa. Similarly, for a depth map obtained from a point cloud viewed from the side, the x-direction of the image corresponds to the y-direction of the point cloud, and vice versa. In the depth map, pixels obtained from the point cloud are represented as 255; otherwise, they are represented as 0.

[0075] Region growing is performed on the depth map to obtain continuous closed regions of interest. Along the x-axis of the image, the derivative of each column of pixels is calculated and stored in a matrix Kij (i = 0, 1, 2... rows - 1, j = 0, 1, 2,... cols), where rows is the number of rows in the depth map and cols is the number of columns. The first non-zero derivative of each column of Kij is stored in array upArray, the last non-zero derivative of each column of Kij is stored in array downArray, and the column number containing exactly four non-zero derivatives in each column of Kij is stored in colArray. The second derivatives of upArray and downArray are calculated separately. Points whose second derivatives fall within a threshold range are saved, while points outside the threshold are removed, thus achieving filtering of abrupt changes.

[0076] If the depth map is a top-down depth map, count the length of continuous data in colArray. If the length is greater than the threshold Threshold2 and the difference between the row number of the last non-zero derivative and the row number of the first non-zero derivative in the same column of Kij is greater than the threshold Threshold3, then the point cloud of the current frame is considered to be the point cloud at the intersection.

[0077] Two-dimensional curve fitting

[0078] The boundary point cloud obtained in step 3) is subjected to curve fitting. In this embodiment, the ransac algorithm is used to perform cubic curve fitting on the boundary point cloud.

[0079] The mathematical model for two-dimensional polynomial fitting is The process of performing polynomial fitting using the ransac algorithm is as follows:

[0080] (1) Select 4 points for polynomial fitting, and denote the parameter as a. i (i = 0, 1, 2, 3);

[0081] (2) Calculate the distance from the point to be fitted to the curve as the error d. If d is greater than the distance threshold T, then the point is an inlier; otherwise, it is an outlier.

[0082] (3) Repeat step (2) until all fitted points have been traversed;

[0083] (4) Select the parameter a with the largest number of interior points. i (i = 0, 1, 2, 3) are the parameters of the fitted cubic polynomial equation.

[0084] 3D curve fitting

[0085] By combining the two-dimensional curve fittings from the top and side views, the equation of the space curve is obtained as follows:

[0086]

[0087] In summary, this embodiment, in its accuracy test, uses the spatial curve equation as the axis of rotation and a certain length as the radius to rotate, forming a curved barrel shape. Ensuring no obstacles during train travel, the number of obstacles within the barrel-shaped space is counted. After repeated testing, this invention can effectively fit the tunnel trend. Furthermore, it can remove interference point clouds from non-obstacles such as tunnel walls and signal lights, providing a better preprocessing foundation for obstacle recognition.

[0088] This invention calculates the three-dimensional spatial trend of a tunnel using point cloud data and extracts boundary points from the projected point cloud using a concave hull algorithm. In the outlier removal process, a point cloud-to-depth map method is employed to convert disordered point clouds into ordered pixels, solving the problem of sorting disordered point clouds when distinguishing upper and lower boundaries.

[0089] Example 2

[0090] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a point cloud-based tunnel trend recognition method as described in Embodiment 1.

[0091] The computer devices in the embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptops, PDAs (personal digital assistants), and PADs (tablet computers), as well as fixed terminals such as desktop computers.

[0092] Computer devices may include processing units (such as a central processing unit), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or loaded from storage devices into random access memory (RAM). RAM also stores various programs and data required for the operation of the computer device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0093] A computer program carried on a computer-readable medium includes program code for performing an algorithm. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined in the methods of embodiments of this disclosure.

[0094] It should be noted that the computer-readable medium disclosed in this invention may be a computer-readable signal medium, a computer-readable medium, or any combination of the two.

[0095] The aforementioned computer-readable medium may be included in the aforementioned computer device; or it may exist independently and not assembled into the computer device.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the contents described in the accompanying drawings and the specific embodiments above. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.

Claims

1. A tunnel trend recognition method based on point clouds, which processes tunnel point clouds acquired by lidar, characterized in that, Includes the following steps: 1) Project the point cloud onto the yox and yoz planes respectively to obtain the top view and side view of the point cloud. The y direction is the direction of train travel in the tunnel, the x direction is the perpendicular direction of train travel and is parallel to the ground, and the z direction is the perpendicular direction of the ground. 2) Using an edge extraction algorithm, extract the boundary points of the point cloud in the top view and side view, and denote them as point sets points1 and points2, respectively; 3) Remove outliers from points1 and points2; 4) For the contour formed by the point set points1, determine whether the current frame point cloud is a fork point cloud by using the width threshold and the branch number threshold. If it is a fork, proceed to step 5); otherwise, proceed to step 6. 5) Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The left and right division is based on the central axis of the tunnel length direction. Perform curve fitting on the two sets of points1_left and points1_right to obtain the tunnel trend in the top view. 6) Divide the boundary points in points1 into the leftmost boundary point points1_left and the rightmost boundary point points1_right. The division between left and right is based on the central axis of the tunnel length direction. Calculate the difference in the number of points and the variance of the two point sets points1_left and points1_right. Based on the point count threshold and the variance threshold, determine whether to select points1_left or points1_right for curve fitting to obtain the tunnel trend in the top view direction. 7) Divide the boundary points in points2 into points closer to the ground and points farther from the ground. Select the points closer to the ground for curve fitting to obtain the tunnel trend in the side view direction. 8) Combine the two-dimensional curves from the top and side views to obtain the curve equation in three-dimensional space.

2. The tunnel trend recognition method based on point cloud according to claim 1, characterized in that, Step 2) The algorithm for extracting point cloud boundary points includes the following steps: (21) Determine the radius r based on the angular resolution of the lidar; (22) Select any point p in the point cloud and count the points in the point cloud that are within a distance of less than 2r from point p; (23) Select any point q in the point set Points. Given p, q and r, calculate the center p_center. There are two centers that meet the conditions, denoted as p_center1(x1, y1) and p_center2(x2, y2). The formula for calculating the center is as follows: in S 2 =(p x -q x ) 2 +(p y -q y ) 2 ; (24) Calculate the distance from the remaining points in Points other than points p and q to the center p_center. If all distances are greater than r, then p is a boundary point. If none of the distances are greater than r, then replace point q in Points and continue to execute steps (23) and (24). If none of the points in Points meet the conditions of step (24), then point p is a non-boundary point.

3. The tunnel trend recognition method based on point cloud according to claim 1, characterized in that, The point set of the point cloud boundary obtained in step 2) is converted into a depth map image, thus obtaining ordered pixels. When converting into a depth map image, for the point cloud in the top view direction, the x direction of the image corresponds to the y direction of the point cloud, and the y direction of the image corresponds to the x direction of the point cloud; for the point cloud in the side view direction, the x direction of the image corresponds to the y direction of the point cloud, and the y direction of the image corresponds to the z direction of the point cloud. In the depth map, the pixel obtained through the point cloud is 255, otherwise it is 0. The depth map is grown to obtain a continuous closed region of interest. Along the x-direction of the image, the derivative of each column of pixels is calculated and stored in the matrix Kij (i = 0, 1, 2... rows - 1, j = 0, 1, 2,... cols), where rows is the number of rows in the depth map and cols is the number of columns in the depth map. Store the first non-zero derivative of each column of kij into array upArray, the last non-zero derivative of each column of kij into array downArray, and the column number of each column of kij containing exactly four non-zero derivatives into array colArray. Calculate the second derivatives of upArray and downArray respectively, save the points whose second derivatives are within the threshold range, and remove the points outside the threshold, thereby achieving filtering of abrupt change points. If the depth map is a top-down depth map, count the length of continuous data in colArray. If the length is greater than the threshold Threshold2 and the difference between the row number of the last non-zero derivative and the row number of the first non-zero derivative in the same column of Kij is greater than the threshold Threshold3, then the point cloud of the current frame is considered to be the point cloud at the intersection.

4. The tunnel trend recognition method based on point cloud according to claim 1, characterized in that, The mathematical model for two-dimensional polynomial fitting is Steps 5), 6), and 7) use the ransac algorithm for polynomial fitting, as follows: (1) Select 4 points for polynomial fitting, and denote the parameter as a. i (i = 0, 1, 2, 3); (2) Calculate the distance from the point to be fitted to the curve as the error d. If d is greater than the distance threshold T, then the point is an inlier; otherwise, it is an outlier. (3) Repeat step (2) until all fitted points have been traversed; (4) Select the parameter a with the largest number of interior points. i (i = 0, 1, 2, 3) are the parameters of the fitted cubic polynomial equation.

5. A computer device, characterized in that, It includes at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform a point cloud-based tunnel trend recognition method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement a point cloud-based tunnel trend recognition method as described in any one of claims 1 to 4.

Images