A 3D reconstruction method for road environment integrating high-precision maps and laser sparse point clouds
By integrating high-precision maps with sparse point clouds from lasers, a 3D road environment model is generated, which solves the accuracy problem of vehicle-mounted LiDAR systems when reconstructing small objects and achieves efficient road environment reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DUOLUN TECH CO LTD
- Filing Date
- 2023-04-26
- Publication Date
- 2026-07-03
AI Technical Summary
In existing technologies, when collecting road environment data, vehicle-mounted LiDAR systems have difficulty effectively reconstructing 3D models of small objects such as road lights and traffic lights. Furthermore, there are few methods for integrating high-precision maps with vehicle-mounted LiDAR sparse point clouds, resulting in inaccurate road environment reconstruction.
By acquiring latitude and longitude coordinates and discrete points from high-precision maps, road vectorization curves are generated. Combined with point cloud data scanned by LiDAR, noise reduction is performed, and objects are identified using KD Tree and feature parameters to generate a 3D environment model.
It improves the robustness and efficiency of road environment reconstruction, and can accurately identify and reconstruct objects such as roads, vehicles, trees, streetlights and traffic lights, reducing human intervention.
Smart Images
Figure CN117078870B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of road environment reconstruction technology, and in particular to a method for three-dimensional reconstruction of road environment by integrating high-precision maps and laser sparse point clouds. Background Technology
[0002] High-precision maps are a new transportation technology developed to meet the needs of intelligent transportation and autonomous driving. Essentially still electronic maps, they offer significantly higher navigation accuracy than traditional navigation maps, with some even achieving a 10cm accuracy level. This makes them perfectly suitable for complex traffic scenarios such as automatic driving test scoring and automatic parking. Currently, high-precision maps are primarily used in advanced driver assistance systems (ADAS), playing a crucial role in geospatial fencing, perception supplementation, beyond-line-of-sight planning, assisted control, and sign recognition. The intelligent driving community generally believes that Level 3 and above autonomous driving technology devices must be equipped with high-precision maps; otherwise, there will be spatiotemporal blind spots in autonomous driving. Currently, some research has been conducted both domestically and internationally on the creation of high-precision road maps using vehicle-mounted radar data.
[0003] LiDAR (Light Detection and Ranging) is a hardware system developed based on the principle of laser ranging to acquire three-dimensional information of a target surface. Initially used primarily for indoor reverse engineering design, it is a successor to coordinate measuring machines (CMMs) for automatic ranging. Due to the significant interference from environmental reflection and diffraction of lasers, it was rarely used for reverse modeling of objects in outdoor environments in the early days. However, in recent years, with the continuous iteration of intelligent driving technology, LiDAR has become a standard physical ranging device in intelligent driving systems. Therefore, using laser scanning equipment to acquire three-dimensional data of road space has become a feasible solution. Based on the number of radar scan lines and the hardware differences of the acquisition platform, vehicle-mounted LiDAR can be divided into 8-line, 16-line, and 32-line acquisition systems. More lines mean more point cloud data acquired per unit time, but also a higher probability of acquiring noisy data and higher hardware costs. Therefore, 16-line acquisition radar is more suitable for the modeling needs of road environments in terms of cost and data processing difficulty.
[0004] Vehicle-mounted LiDAR systems primarily mount the LiDAR on a vehicle roof platform, acquiring target point cloud information through a rotating scanning device. Road information point clouds acquired by vehicle-mounted LiDAR systems have moderate density, and the spatial outlines of the ground, other vehicles, and roadside trees are visually clear. However, for road lights and traffic lights, due to the small volume tensor of the objects themselves, the outlines become blurred at higher vehicle speeds, resulting in low point cloud density, low resolution, and difficulty in modeling. While some research has been conducted in the fields of computer graphics and intelligent transportation engineering regarding the processing and 3D reconstruction of point clouds from vehicle-mounted LiDAR systems and related road environments, few researchers have addressed the issue of 3D road environment reconstruction methods that integrate high-precision maps and sparse point clouds from vehicle-mounted LiDAR systems. Therefore, this invention aims to propose a 3D road environment reconstruction method that integrates high-precision maps and sparse point clouds from vehicle-mounted LiDAR systems, thereby solving the technical problem of sparse point cloud environment reconstruction. Summary of the Invention
[0005] This invention provides a method for three-dimensional reconstruction of road environment by integrating high-precision maps and laser sparse point clouds, which can solve the problems pointed out in the background art.
[0006] A method for 3D reconstruction of road environments that integrates high-precision maps and laser sparse point clouds includes:
[0007] S1. Obtain latitude and longitude coordinates and high-precision map;
[0008] S2. Obtain the road vectorization curve based on discrete points in the high-precision map;
[0009] S3. Obtain road width information from high-precision maps, then perform a bilateral offset algorithm on the vectorized curves of the roadway lines to form the roadside boundaries, and then use the Delaunay triangulation algorithm to generate a spatial three-dimensional road surface mesh by using the roadside boundaries and the vectorized curves of the roadway lines as constraint edges.
[0010] S4. Scan the road environment with lidar to obtain point cloud data, and denoise the point cloud data through principal component analysis;
[0011] S5. Construct a K-DTree from the denoised point cloud data, map the local point cloud of the K-DTree using the principal plane subdivision projection method, extract the edge contour points of the projected point cloud model using the A* search algorithm, optimize the edge contour points by B-spline curve interpolation, and calculate several morphological feature parameters of the projected closed contour surface to achieve classification and recognition of different objects.
[0012] S6. The coordinates are transformed and fused into a three-dimensional road environment model.
[0013] The method for obtaining the road vectorized curve result in S2 is as follows:
[0014] Based on a database of high-precision maps, obtain several discrete points p on the map. i Connect two adjacent discrete points and construct a polygonal curve s by granular Laplacian smoothing to obtain the vectorized curve of the lane line;
[0015] The formula for generating the vectorized curve of the roadway is as follows:
[0016] p i =p i-1 *+ i+1 *(α-1))*(1-)+p i *
[0017] Where α is the internal smoothing coefficient between adjacent vertices, β is the external smoothing coefficient between current vertices, and p i-1 For; p i+1 They are respectively with discrete point p i Two adjacent points.
[0018] The formula for the two-sided bias algorithm in S3 is as follows:
[0019] s i+ = i +n i * i
[0020] s i- = i -n i * i
[0021] Where p i It is a point on the current curve, s i+ and s i- p i The point is located at one of the two boundaries of the current road, n i It is the direction of the offset vector corresponding to that point, calculated from the cross product of the vectors of the two adjacent points and the base plane. i This is the current road width information retrieved from a high-precision map for that point.
[0022] The specific method for denoising point cloud data using principal component analysis in S4 is as follows: After orthogonal linear transformation, the original point cloud data is transformed to obtain local strong component vectors, where the first component weight vector w (1) The following formula must be satisfied:
[0023]
[0024] The morphological characteristic parameters in S5 include, but are not limited to, one or more of the following: area, perimeter, aspect ratio, radius ratio, and Gaussian curvature.
[0025] The specific scheme for generating a 3D road environment model by coordinate fusion transformation in S6 is as follows: Based on the area and aspect ratio in the feature vector, the model tree is scaled in the X, Y and Z directions and positioned to the spatial road boundary, thereby realizing automatic modeling of the fused scene. Then, for all the classified feature vectors, a specific type of 3D mesh model is loaded and fused into the 3D scene.
[0026] Compared with existing technologies, the beneficial effects of this invention are: the road environment 3D reconstruction method that integrates high-precision maps and sparse point clouds obtained from vehicle-mounted laser scanning can ensure the robustness of road body recognition; after denoising the sparse point cloud through principal component algorithm, KD tree can be used again to accurately classify other vehicles, roadside trees, streetlights, traffic lights, and walls; the automatic reconstruction of the 3D road environment requires less manual intervention in the later stages, which can effectively improve the efficiency of road environment modeling based on 3D sparse point clouds. Attached Figure Description
[0027] Figure 1 This is a system structure diagram of the present invention.
[0028] Figure 2 This is the original sparse point cloud map collected by the vehicle-mounted radar of the present invention.
[0029] Figure 3 The wireframe diagram for reconstructing the scene model after fusing high-precision map and point cloud classification and recognition according to the present invention.
[0030] Figure 4 This is a rendering of the scene model reconstructed after the fusion of high-precision map and point cloud classification and recognition in this invention. Detailed Implementation
[0031] The following detailed description of a specific embodiment of the present invention is provided in conjunction with the accompanying drawings. However, it should be understood that the scope of protection of the present invention is not limited to the specific embodiment.
[0032] like Figures 1 to 4 As shown in the figure, an embodiment of the present invention provides a method for three-dimensional reconstruction of road environment by fusing high-precision maps and laser sparse point clouds, comprising:
[0033] S1. Obtain latitude and longitude coordinates and high-precision map;
[0034] S2. Obtain the vectorized curve of the road line based on discrete points in the high-precision map;
[0035] S3. Obtain road width information from high-precision maps, then perform a bilateral offset algorithm on the vectorized curves of the roadway lines to form the roadside boundaries, and then use the Delaunay triangulation algorithm to generate a spatial three-dimensional road surface mesh by using the roadside boundaries and the vectorized curves of the roadway lines as constraint edges.
[0036] S4. Scan the road environment with lidar to obtain point cloud data, and denoise the point cloud data through principal component analysis;
[0037] S5. Construct a K-DTree from the denoised point cloud data, map the local point cloud of the K-DTree using the principal plane subdivision projection method, extract the edge contour points of the projected point cloud model using the A* search algorithm, optimize the edge contour points by B-spline curve interpolation, and calculate several morphological feature parameters of the projected closed contour surface to achieve classification and recognition of different objects.
[0038] S6. The coordinates are transformed and fused into a three-dimensional road environment model.
[0039] like Figure 1 As shown, the present invention includes: a lidar vehicle, latitude and longitude coordinates, a high-precision map, road line vectorization, road triangulation, sparse point cloud, PCA calculation, denoising, KD Tree construction, feature classification and recognition, specific classification objects, and the finally constructed three-dimensional road environment model;
[0040] The lidar vehicle 1 supports 8-line, 16-line, and 32-line lidar acquisition devices. During the driving process, the lidar vehicle receives GPS / BeiDou navigation data to obtain latitude and longitude coordinates. It directly queries high-precision maps through the onboard computer to obtain a series of discrete points on the map. The discrete points are connected, and granular smoothing is performed to construct polygon curves, thereby obtaining the vectorized curve result of the road line. Then, using the road width information in the high-precision map, a bilateral offset algorithm is applied to the curve to form the two sides of the road boundary. The Delaunay triangulation algorithm is used to generate a spatial three-dimensional road surface mesh.
[0041] When a lidar vehicle scans a sparse point cloud, instead of immediately constructing a KD Tree, iterative principal component analysis (PCA) is performed on the point cloud to obtain directions and moduli with high point cloud density. Points far from the cylinder in that direction are removed for noise reduction, resulting in more stable and reliable feature-related point cloud data, which is then used to construct the KD Tree. Local K-DTree point clouds are obtained using the principal plane subdivision projection method. The A* search algorithm is applied to extract the edge contour points of the projected point cloud model. B-spline curve interpolation is used to optimize the edge contour points, and five morphological feature parameters—area, perimeter, aspect ratio, radius ratio, and Gaussian curvature—are calculated for the projected cover contour surface of the environment model. These parameters are used for feature classification and recognition, identifying other vehicles, roadside trees, streetlights, traffic lights, and walls. Finally, coordinate transformation is used to fuse the data into a 3D road environment model.
[0042] The specific content is explained as follows:
[0043] Figure 1The aforementioned lidar vehicle refers to a vehicle equipped with a 16-line lidar sensor mounted on its roof. After the vehicle travels on the target road, it activates its GPS / BeiDou navigation system. The system transmits the vehicle's longitude and latitude every second, and uses an onboard high-precision map database to obtain a series of discrete points p on the map. i Connecting discrete points, performing granular Laplacian smoothing, and constructing a polygonal curve yields the vectorized curve result of the roadway. Then, using road width information from the high-precision map, a bilateral offset algorithm is applied to the curve s to form the roadside boundaries s on both sides. i+ and s i- Using the Delaunay triangulation algorithm, s and s i- and s i- The constraint edges are used to generate a spatial three-dimensional road surface mesh;
[0044] The formula for generating the Laplace curve smoothing of the lane line vector is as follows:
[0045] p i =(p i-1 *a+p i+1 *(α-1))*(1-β)+p i *β
[0046] Where α is the internal smoothing coefficient between adjacent vertices, β is the external smoothing coefficient between current vertices, and p i-1 For; p i+1 They are respectively with discrete point p i Two adjacent points; generally, the smoothing sparsity α and β are both set to 0.5, indicating uniform smoothing; the curve can be smoothed multiple times to obtain a smooth road curve;
[0047] The formula for the two-sided bias algorithm for curve s is as follows:
[0048] s i+ =p i +n i *w i
[0049] s i- =p i -n i *W i
[0050] Where p i It is a point on the current curve, s i+ and s i- p i The point is located at one of the two boundaries of the current road, n i It is the direction of the offset vector corresponding to that point, calculated from the cross product of the vectors of the two adjacent points and the base plane. iThis is the current road width information retrieved from a high-precision map for that point;
[0051] The Delaunay triangulation algorithm is implemented using the Bowyer-Watson method, and the specific algorithm is as follows:
[0052] function DelaunayBowyerWatson(pointList)
[0053] triangulation: = Empty grid
[0054] Add a super-triangle to the mesh / / A triangle large enough to accommodate all the points is called a super-triangle.
[0055] Traverse the input point list one by one
[0056] Set badTriangles to empty
[0057] Iterate through all triangles in the current triangulation set.
[0058] If the current point is inside the circumcircle of the current triangle, then add the triangle to badTriangles.
[0059] Set the polygon to empty.
[0060] Iterate through all triangles in the badTriangles collection.
[0061] Traverse each edge of the triangle
[0062] If this edge is not shared by triangles in other badTriangles
[0063] Add the edge to the polygon set.
[0064] Iterate through each triangle in badTriangles
[0065] Remove the triangle from the triangulation set.
[0066] Traverse each edge of the polygon
[0067] Construct a new triangle using this edge and the currently traversed point, and add it to the triangulation set.
[0068] Traverse each triangle in the triangulation
[0069] If a triangle contains any vertex of the original super-triangle, then remove the triangle from the triangulation set.
[0070] Return the triangulation set
[0071] Figure 1 Principal Component Analysis (PCA) described above is used to obtain local statistical features of sparse point clouds. Essentially, it transforms the original data through an orthogonal linear transformation to obtain local strong component vectors. To maximize the variables, the first component weight vector w... (1) The following formula must be satisfied:
[0072]
[0073] Because of w (1) Since it is a unit vector, it is equivalent to the following formula:
[0074]
[0075] Figure 2 It is the original sparse point cloud of the actual road environment sampled. Figure 1 The denoising described herein involves performing principal component analysis on the point cloud iteratively to obtain the direction and modulus of the point cloud with high density, and then removing point clouds that are far from the cylinder in that direction to denoise the point cloud. During the removal process, the cylinder radius is adaptively assigned, with a smaller radius selected for high-noise point clouds and a larger radius selected for low-noise point clouds.
[0076] Figure 1 The KD Tree described above is a tree-like data structure for storing instance points in a K-dimensional space for fast retrieval. It is primarily used for searching key data in multidimensional space, such as range search and nearest neighbor search. A K-DTree is a special case of a binary space partitioning tree. In computer science, a KD Tree is a data structure that organizes points in a K-dimensional Euclidean space. KD Trees can be used in various applications, such as multidimensional key-value searches. The KD Tree is a special case of a binary space partitioning tree, and its implementation algorithm is as follows:
[0077] function kdtree(pointcloud,depth)
[0078] The depth-based axis selection allows the collar to pass through all values.
[0079] var int axis:=depth mod k;
[0080] Sort the point column and select a pivot as the median.
[0081] Create tree nodes to build subtrees
[0082] node.location:=median;
[0083] node.leftChild := kdtree(all nodes before the pivot, depth+1);
[0084] node.rightChild := kdtree(all nodes after the pivot, depth+1);
[0085] return node;
[0086] Figure 1 The feature classification and recognition described herein refers to obtaining a reliable KD Tree of point clouds, acquiring local K-DTree point clouds using the principal plane subdivision projection method, extracting edge contour points of the projected point cloud model using the A* search algorithm, optimizing the edge contour points through B-spline curve interpolation, and calculating five morphological feature parameters—area, perimeter, aspect ratio, radius ratio, and Gaussian curvature—of the projected closed contour surface of the environment model. These parameters are used to classify other vehicle models, roadside trees, streetlights, traffic lights, and wall models. The specific details are explained below:
[0087] The principal plane subdivision projection method for obtaining local point clouds of K-DTree involves traversing the nodes of the 3rd level of the KD Tree, calculating the least-squares plane of the sub-point cloud within each node, projecting the point cloud within that node onto this plane, searching for the boundary polygon of the projection model using the A* algorithm, optimizing the boundary line using a 3rd-order B-spline curve interpolation algorithm, and further calculating the area a, perimeter l, aspect ratio w, radius ratio r, and Gaussian curvature c of the contour surface. A normalized 5D vector is then constructed.<a,l,w,r,c> The 5-dimensional Euclidean distance minimum method is used to classify node information into 5 scene models. The maximum classification error is defined as ω. If the error of a node vector from each scene is greater than ω, the parent node of the node is searched, and the 5-dimensional vector of all point clouds in the parent node is recalculated. The classification error is then calculated. If it still exceeds the maximum error ω, the classification fails; otherwise, the classification succeeds.
[0088] After the scene model features are successfully classified and recognized, the corresponding pre-designed 3D mesh model, such as a tree model, is loaded from the existing background 3D model database. Based on the area and aspect ratio in the feature vector, the model tree is scaled in the X, Y, and Z directions and positioned to the spatial road boundary, thereby realizing automatic fusion scene modeling. According to the same principle, all classified feature vectors are loaded into a specific type of 3D mesh model and fused into the 3D scene. Figure 3This describes the wireframe display results after the road scene model is automatically loaded. The results include road lines, road surface mesh, road tree mesh, street light mesh, wall mesh, median strip mesh, etc.
[0089] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit and essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0090] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A method for 3D reconstruction of road environments by integrating high-precision maps and laser sparse point clouds, characterized in that, Includes the following steps: S1. Obtain latitude and longitude coordinates and high-precision map; S2. Obtain the vectorized curve of the road line based on discrete points in the high-precision map; S3. Obtain road width information from high-precision maps, then perform a bilateral offset algorithm on the vectorized curves of the roadway lines to form the roadside boundaries, and then use the Delaunay triangulation algorithm to generate a spatial three-dimensional road surface mesh by using the roadside boundaries and the vectorized curves of the roadway lines as constraint edges. S4. Scan the road environment with lidar to obtain point cloud data, and denoise the point cloud data through principal component analysis; S5. Construct a K-DTree from the denoised point cloud data, map the local point cloud of the K-DTree using the principal plane subdivision projection method, and apply A... The search algorithm extracts the edge contour points of the projected point cloud model, optimizes the edge contour points through B-spline curve interpolation, and calculates the area a, perimeter l, aspect ratio w, radius ratio r, and Gaussian curvature c of the projected closed contour surface; a normalized 5-dimensional vector is then constructed.<a,l,w,r,c> The method uses the minimum 5-dimensional Euclidean distance to classify node information into 5 scene models. The maximum classification error is defined as ω. If the error of a node vector from each scene is greater than ω, the parent node of the node is searched, and the 5-dimensional vector of all point clouds in the parent node is recalculated. The classification error is then calculated again. If it still exceeds the maximum error ω, the classification fails; otherwise, the classification succeeds. This method enables the classification and recognition of different objects. S6. Merge the coordinates into a 3D road environment model: Based on the area and aspect ratio in the feature vectors, the model tree is scaled in the X, Y, and Z directions and positioned to the spatial road boundary, thereby realizing automatic modeling of the fused scene. Then, for all the classified feature vectors, a specific type of 3D mesh model is loaded and fused into the 3D scene.
2. The method for three-dimensional reconstruction of road environment by fusing high-precision maps and laser sparse point clouds as described in claim 1, characterized in that, The method for obtaining the road vectorized curve result in S2 is as follows: Based on a database of high-precision maps, obtain several discrete points on the map. Connect two adjacent discrete points and construct a polygon curve using granular Laplacian smoothing. , to obtain the vectorized curve of the road line; The formula for generating the vectorized curve of the roadway is as follows: Where parameters It is the smoothness coefficient between adjacent vertices. ，β It is the external smoothness coefficient between the current vertices. for; They are respectively discrete points Two adjacent points.
3. The method for three-dimensional reconstruction of road environment by fusing high-precision maps and laser sparse point clouds as described in claim 1, characterized in that, The formula for the bilateral bias algorithm in S3 is as follows: in It is a point on the current curve. and They are respectively The point is located at the two boundaries of the current road. It is the direction of the offset vector corresponding to that point, calculated from the cross product of the vectors of the two adjacent points and the base plane. This is the current road width information retrieved from a high-precision map for that point.
4. The method for three-dimensional reconstruction of road environment by fusing high-precision maps and laser sparse point clouds as described in claim 1, characterized in that, The specific method for denoising point cloud data through principal component analysis in S4 is as follows: After orthogonal linear transformation, the original point cloud data is transformed to obtain local strong component vectors, where the first component weight vector is... The following formula must be satisfied: 。