A method for extracting characteristic lines of an urban road scene

Abstract

The present application belongs to the field of simultaneous localization and mapping of scene, and particularly relates to a method for extracting feature lines of urban road scene. The present application trains a classification model for acquiring feature lines of road scene by summarizing common and reliable line categories in urban road scene, constructing a simulation data set containing category labels corresponding to feature lines, and thereby classifying and extracting feature lines of urban road scene in a targeted manner, and the manual marking operation is saved, and the marking result is more accurate. Furthermore, by using the scale-invariant local neighborhood features corresponding to each point in the simulation point cloud, the scale factor is eliminated, the scale invariance is realized, and it is ensured that the classification model trained by using the simulation data set can accurately classify the real point cloud data. In addition, the classification deviation of each point is judged according to the classification results of different two-dimensional transformations of the real point cloud by the classification model, and the points with small classification deviation are selected as feature line candidate points with high classification accuracy, which are used to acquire feature lines.

Description

Technical Field

[0001] This invention belongs to the field of simultaneous localization and mapping of scenes, specifically relating to a method for extracting feature lines in urban road scenes. Background Technology

[0002] Traditional LiDAR odometry methods mainly include registration methods based on local feature descriptors and iterative nearest-point algorithms (Besl and McKay, 1992). However, due to the characteristics of urban road scenes with vehicular laser point clouds, traditional methods often perform poorly on such data. In recent years, deep learning-based odometry methods have shown encouraging results on publicly available datasets. These methods can be specifically divided into two main categories: pose regression networks based on 2D projection and point correspondence calculation networks. 2D projection-based methods inevitably lose original 3D information, resulting in lower odometry accuracy (Wang et al., 2021). The following mainly introduces networks based on point correspondence calculation.

[0003] To address the problem of finding exact matching point pairs, Lu et al. (2019) proposed the first end-to-end point cloud registration network, DeepVCP, based on virtual matching points. This network uses PointNet++ (Qi et al., 2017) to learn point-wise local features, selects keypoints through a point-weighted layer, and finally generates virtual matching points for the extracted keypoints through a corresponding point generation layer. Wei et al. (2020) proposed an attention-based point cloud registration network, LPDRegistration, based on LPD-Net (Liu et al., 2019). This method enhances the structural information of the point cloud and the correlation between two point clouds through self-attention and mutual attention mechanisms. Then, it generates K optimal matching points through a virtual corresponding point generation module, and finally solves for the transformation parameters using a differentiable SVD algorithm. This type of virtual point generation method directly estimates the virtual positions of keypoints in the source point cloud in the target point cloud without needing to remove incorrect matching point pairs, achieving registration accuracy and operational efficiency comparable to state-of-the-art geometry-based methods. Choy et al. (2020) proposed a robust and accurate registration network, DGR. This network first uses FCGF (Choy et al., 2019) to extract point-by-point features and employs a 6D convolutional neural network to evaluate the confidence of point matching relationships. Then, it uses a differentiable weighted Procrustes algorithm (Yuan et al., 2020) to estimate pose parameters. This network achieves high registration accuracy on both indoor and outdoor datasets, demonstrating strong adaptability. Huang et al. (2021) proposed a network, PREDATOR, that can simultaneously detect overlapping regions between two point clouds and solve for relative transformation parameters. This network encodes point clouds into feature-rich superpoints, then extracts common contextual information between the two point clouds through an overlap attention module and estimates the overlap score of each superpoint. Finally, a decoder decodes the common contextual information and overlap score to obtain the matching probability. This method achieves high registration accuracy even with low overlap. Wang et al. (2023) proposed RoReg, a point cloud registration framework based on group rotation equivariant features and local rotation estimation. RoReg incorporates rotation group features and local rotation into the entire point cloud registration process, including keypoint detection, feature description, corresponding point prediction, and transform estimation, significantly improving the reliability and efficiency of point cloud registration. In general, these methods are not very sensitive to initial location and can achieve high accuracy when the data quality is good. However, they mostly use points as primitives, resulting in large datasets, and they cannot directly find accurate point pairs for road scene point clouds with "near-dense, far-sparse" characteristics.

[0004] Feature line extraction of point clouds can be mainly divided into two categories: (1) converting point clouds into images and extracting point cloud feature lines based on image feature line methods; (2) extracting point cloud feature lines by analyzing point cloud geometric features (such as curvature and normal vectors).

[0005] Taking the second type of method as an example, in order to meet the requirement of quickly identifying building surfaces and edges in building modeling or vector information extraction, Xin Qunrong et al. (2018) used shared nearest neighbor clustering based on the normal vector and the distance from the point to the reference plane to quickly extract building surfaces, and extracted building edges through intersecting planes. This method can be applied to automatic modeling and mapping of 3D buildings, but it is only applicable to the edges of planes intersecting each other, and the type is limited. In order to address the problem that existing feature line extraction algorithms do not take into account time efficiency and the difference between point cloud boundaries and folded edges, Ren Qiancheng (2019) used the angle between adjacent vectors of nearby projection points to extract boundary lines, and used the k-means clustering of nearby point vectors to extract folded edges. A series of experiments on buildings, 3D models, and bridges proved that this method is applicable not only to planar structures, but also to the extraction of folded edges of non-planar structures. Chen Huwei et al. (2019), focusing on mechanical parts, proposed a feature line extraction scheme based on curvature abrupt change points, drawing on the ideas of region segmentation and edge detection in digital image processing. This scheme includes candidate point set separation through connected region clustering, point set refinement for local influence region erosion, and feature line branch truncation and sorting algorithms based on combined search criteria and bidirectional search using principal component analysis. Addressing the incomplete extraction of transition and detail feature lines in reverse engineering, Shi Hongxia and Wang Jianmin (2021) proposed a feature line extraction method based on adaptive neighborhood principal component analysis and normal vector region clustering. This method exhibits good adaptability and accuracy, effectively extracting sharp and detail features of point cloud models while preserving as many transition features as possible. To address feature line breakage and bifurcation caused by noise and non-uniform sampling, Yu Mengjuan and Nie Jianhui (2022) first calculated robust curvature values ​​and principal curvature directions to noise and sampling variations using PCPNET, then fitted the local curvature distribution with a weighted quadratic curve, and finally iteratively refined the feature lines using a bilateral filtering algorithm based on curvature and distance. This method overcomes the shortcomings of traditional methods that use simple threshold truncation, which prevents the proper extraction of feature lines in flat areas. It can extract feature lines completely even when dealing with complex features, noise, and non-uniform sampling. Wang Jin et al. (2023) proposed a method for extracting the centerline of highway overpasses. This method uses multiple filtering to filter non-road surface point clouds, Alpha shapes algorithm to identify road boundaries, and B-spline curve fitting to fit the road centerline. This method overcomes, to some extent, the problems of high noise in point cloud data and the significant impact of line shape changes on road centerline extraction.

[0006] Given the complex and ever-changing conditions of urban road scenarios, performing odometer readings directly on vehicle-mounted laser point clouds presents the following challenges:

[0007] (1) The original point cloud data is large in volume, and processing it directly with points as primitives is inefficient and cannot meet the requirements of real-time processing; (2) The point cloud density is "dense near and sparse far", and the local features of the points are inconsistent, making it difficult to find exact corresponding point pairs between two consecutive frames; (3) The motion of points of some dynamic objects in the scene (such as moving vehicles, pedestrians, and leaves) is uncertain and is not consistent with the motion of the LiDAR sensor. These problems make the existing point-based mileage calculation method unable to meet the requirements of "accurate and efficient" simultaneous localization and mapping of urban road scenes using vehicle-mounted laser point clouds.

[0008] In summary, lines offer significant advantages over points in odometry tasks within urban road scenes. Firstly, lines provide location, orientation, and size information, exhibiting higher descriptiveness, repeatability, and robustness. Secondly, feature line extraction effectively filters out points from dynamic objects such as vehicles, pedestrians, and leaves, preventing their involvement in relative pose calculations. Furthermore, using feature lines as odometry input effectively reduces the amount of input point cloud, thereby improving processing efficiency. However, most basic line extraction methods, while ensuring detection accuracy, either indiscriminately extract all feature lines in the scene or extract fixed types of feature lines from specific scenes (e.g., building outlines, edges, 3D model feature lines), failing to target specific objects and thus unable to selectively detect only reliable lines (feature lines) in urban road scenes. While existing deep learning-based feature line extraction methods can perform targeted detection, they rely on labeled datasets of reliable lines in urban road scenes for model training. Obtaining such datasets typically requires manually labeling a large number of feature lines, which is time-consuming, labor-intensive, and cannot guarantee accuracy. Summary of the Invention

[0009] The purpose of this invention is to provide a method for extracting feature lines in urban road scenes, which solves the problem that existing line detection methods have difficulty in achieving both specificity and accuracy in extracting feature lines in urban road scenes.

[0010] To achieve the above objectives, the present invention provides a method for extracting feature lines in urban road scenes, comprising the following steps:

[0011] 1) Simulate the real point cloud corresponding to the urban road scene using simulation primitives, and label the simulated points according to their categories to obtain the simulated point cloud corresponding to the urban road scene; the categories include the categories corresponding to feature lines;

[0012] 2) Using the simulated point cloud as the input point cloud data for the line extraction network, the line extraction network is trained to obtain a scale-invariant classification model; the line extraction network obtains the feature lines corresponding to the input point cloud data as output based on the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud.

[0013] 3) Classify the real point cloud corresponding to the urban road scene according to the scale-invariant classification model, verify the classification results, and obtain the feature lines of the urban road scene based on the points that pass the verification.

[0014] The beneficial effects of the above technical solution are as follows: Addressing the current lack of vehicle-mounted laser point cloud line labeling data and the time-consuming, labor-intensive, and inaccurate nature of manual labeling, this solution summarizes common and reliable line categories in vehicle-mounted laser point cloud road scenes and constructs a simulated dataset containing category labels corresponding to feature lines. By training a classification model for acquiring feature lines in urban road scenes using this labeled simulated dataset, targeted classification and extraction of feature lines in urban road scenes can be achieved, eliminating the need for manual labeling. Furthermore, since the category labels corresponding to feature lines are directly applied during the construction of the simulated dataset, the labeling results are more accurate. Regarding the issue of scale inconsistency between the simulated dataset and the real scene when using a classification model trained on the simulated dataset to classify real point cloud data, this solution eliminates the scaling factor by simulating scale-invariant local neighborhood features corresponding to each point in the point cloud, achieving scale invariance and ensuring that a classification model that accurately classifies real point cloud data can be trained using the simulated dataset.

[0015] Furthermore, the simulation primitives include simulated ground points, simulated rod-shaped objects, simulated building planes, and simulated intersecting planes of buildings; the categories also include the categories corresponding to the simulated ground points;

[0016] The categories corresponding to the feature lines include those corresponding to simulated rod-shaped objects, the intersection of simulated building planes and the ground, and the intersection between simulated intersecting planes of buildings;

[0017] The methods for simulating real point clouds corresponding to urban road scenes using simulation primitives include:

[0018] The simulation of ground points uses multiple rings to simulate ground points in the real point cloud corresponding to the urban road scene. The simulation of rod-shaped objects uses straight lines to simulate rod-shaped objects in the real point cloud corresponding to the urban road scene. The simulation of building planes uses planes to simulate the intersection lines between simulated building planes and the ground in the real point cloud corresponding to the urban road scene. The simulation of intersecting building planes uses intersecting planes to simulate the intersection lines between simulated building planes and planes in the real point cloud corresponding to the urban road scene. The points corresponding to the simulation primitives are obtained, thereby simulating the real point cloud corresponding to the urban road scene.

[0019] The beneficial effects of the above technical solution are as follows: based on the actual situation of urban road scenes, lines that can reflect the characteristics of urban road scenes are selected as the simulation primitives and categories corresponding to feature lines, so that the extracted feature lines of urban road scenes are more in line with the actual situation.

[0020] Furthermore, the method for simulating ground points in the real point cloud corresponding to the urban road scene is as follows:

[0021] Multiple concentric rings with the same center are set as simulated ground points. The diameter of each ring gradually increases and the distance between any two adjacent rings gradually increases from the inside to the outside. Each ring in the simulated ground points is sampled according to a fixed number of points to obtain the point cloud corresponding to the simulated ground points. The point cloud corresponding to the simulated ground points is cropped, and only points whose absolute y-coordinate value is less than a first set threshold are retained to form a simulated road.

[0022] The methods for simulating rod-shaped objects, building planes, and intersecting building planes in the real point cloud corresponding to urban road scenes are as follows:

[0023] A straight line is set as a simulated rod-shaped object, a plane is set as a simulated building plane, and intersecting planes are set as building intersection planes; the straight line, plane, and intersecting plane are sampled according to a set density to generate simulated point clouds corresponding to the simulated rod-shaped object, simulated building plane, and simulated building intersection plane, respectively; a set number or a set proportion of points are cropped from each point cloud in the generated simulated point cloud, and the cropped point clouds are placed at a set position on the simulated road.

[0024] The beneficial effects of the above technical solution are: it can make the generated simulated point cloud closer to the "dense near and sparse far" characteristics of the real point cloud, and it is more in line with the actual road scene; in addition, the generated simulated point cloud corresponding to the simulated rod-shaped object, the simulated building plane and the simulated building intersection plane can simulate the situation where the object is incomplete due to occlusion in the real scene.

[0025] Furthermore, methods for simulating real point clouds corresponding to urban road scenes using simulation primitives also include:

[0026] Add a set amount of Gaussian noise with zero mean and standard deviation to the points corresponding to the simulation primitives in each coordinate axis direction, and randomly crop a set number of points corresponding to vegetation and vehicles from the real point cloud data as random noise points. Combine the random noise points with the points corresponding to the simulation primitives after adding zero mean and Gaussian noise as the simulated points.

[0027] The beneficial effects of the above technical solution are: it can simulate the noise of real point clouds, making the simulated point clouds closer to the real point clouds of urban road scenes.

[0028] Furthermore, the method for classifying the simulated points is as follows:

[0029] Points corresponding to simulated rod-shaped objects are marked as Category 1, points corresponding to the intersection of the plane and the ground in the plane of the simulated building are marked as Category 2, points corresponding to the intersection of the planes between the planes of the simulated building are also marked as Category 2, and other points are marked as Category 3.

[0030] Furthermore, the method for obtaining the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud is as follows:

[0031]

[0032] In the formula, f represents a scale-invariant local feature, p′ represents a point in the simulated point cloud, and p′ i Let p′ be the i-th closest point to p.

[0033] Search for the k nearest points {p1, p2, ..., pk} of point p. k} Calculate the scale-invariant local features between point p and the k nearest points, and use them as the scale-invariant local neighborhood features corresponding to point p.

[0034] Furthermore, the line extraction network includes a feature encoder and a decoder;

[0035] The feature encoder is used to extract the feature vector of scale-invariant local neighborhood features corresponding to each point in the input point cloud data;

[0036] The decoder is used to convert the feature vector of the scale-invariant local neighborhood features corresponding to each point into a tensor of size N×M, where N represents the number of input points and M represents the number of categories for class labeling the simulated points; based on the tensor, the probability of each point belonging to each category and the category with the highest probability are obtained, and the point is labeled as the category with the highest probability.

[0037] Furthermore, the serial number of the first category is set to 1, the serial number of the second category is set to 2, and the serial number of the third category is set to 0;

[0038] The method for classifying the real point cloud corresponding to the urban road scene according to the scale-invariant classification model is as follows:

[0039] The real point cloud corresponding to the urban road scene is subjected to random perturbation in the 2D XOY plane. One perturbation data is generated each time. After k perturbations, a perturbation data set PC = {pc1, pc2, ..., pc3} is generated. i ,…,pc k}; where the disturbance data pc i The perturbation result obtained by perturbing the real point cloud for the i-th time;

[0040] Each perturbation data point in the perturbation data set is used as input point cloud data. A scale-invariant classification model is used to predict the category of each point in each perturbation data point. If the category number of a point in a perturbation data point is greater than 0, the label value of that point in the perturbation data point is set to 1. Finally, the label values ​​of that point in all perturbation data points are averaged. If the average label value of that point is greater than a set label value threshold, then that point is used as a candidate point for feature lines.

[0041] The beneficial effects of the above technical solution are as follows: In response to the problem that scale-invariant classification models trained on simulated data perform poorly when directly applied to real data, the geometric adaptation method is used to judge the classification deviation of each point in the real point cloud data based on the classification results of different two-dimensional transformations of the real point cloud corresponding to the urban road scene by the scale-invariant classification model, and to select points with smaller classification deviations as candidate feature lines. Therefore, candidate feature lines with higher classification accuracy can be obtained.

[0042] Furthermore, the classification results are validated, and the feature lines of the urban road scene are obtained based on the points that pass the validation as follows:

[0043] All candidate feature line points are clustered using a region growing algorithm, and clusters with fewer than a set threshold number of points are filtered out. Then, a straight line is fitted to the remaining clusters to obtain a fitted line, and outliers are removed. Next, the distance from each of the remaining points after removing outliers to the fitted line is calculated, and points with a distance less than a set threshold are retained. Finally, the retained points with a distance less than the set threshold are clustered again, and after filtering out discrete points, points that pass the verification are obtained. The line formed by the points that pass the verification is used as the feature line of the urban road scene.

[0044] The beneficial effects of the above technical solution are as follows: by clustering, fitting, filtering points near the fitted line, and clustering again, the classification results are filtered and verified. Based on the points that pass the verification, the feature lines of the urban road scene are obtained, which can remove discrete and non-real feature line points and improve the accuracy of urban road scene feature line extraction.

[0045] Furthermore, the feature encoder employs a graph attention network as its backbone network;

[0046] The graph attention network is used to extract the local neighborhood features of each point in the input point cloud data and assign different attention scores to each neighboring node of that point; where the neighboring nodes of a point refer to the k nearest points of that point;

[0047] After extracting scale-invariant local neighborhood features for each point using a graph attention network, the attention scores of all neighboring nodes of each point are normalized using the softmax function. The linear combination of the normalized attention score and the corresponding scale-invariant local neighborhood features for each point is calculated, and the linear combination is used as the feature vector corresponding to the local neighborhood features of each point in the input point cloud data. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating the principle of the method for extracting feature lines in urban road scenes according to an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of the simulation primitives in an embodiment of the method for extracting feature lines in urban road scenes according to the present invention;

[0050] Figure 3 This is a schematic diagram of a simulated point cloud corresponding to an urban road scene in an embodiment of the method for extracting feature lines from an urban road scene according to the present invention;

[0051] Figure 4 This is a schematic diagram of the line extraction network structure in an embodiment of the method for extracting feature lines in urban road scenes according to the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0053] Example of a method for extracting feature lines in urban road scenes

[0054] This embodiment presents a technical solution for feature line extraction in urban road scenes, referring to... Figure 1 It includes the following steps:

[0055] 1) Simulate the real point cloud corresponding to the urban road scene using simulation primitives, and label the simulated points according to their categories to obtain the simulated point cloud corresponding to the urban road scene; the simulated point cloud corresponding to the urban road scene is... Figure 1 A synthetic dataset used to train scale-invariant classification models.

[0056] Generally, lines in vehicle-mounted laser point cloud urban road scenes are mainly divided into four categories: (1) intersection lines between buildings and the ground; (2) intersection lines between different facades of buildings; (3) independent stationary pole-shaped objects, such as streetlights and road signs; (4) virtual lines, lines formed by edge points on the scanning ring due to occlusion or equipment limitations. Among them, the first three types of lines are relatively reliable and can be used as feature lines reflecting urban road scenes. Therefore, in this embodiment, according to the characteristics of vehicle-mounted laser point clouds, the simulation primitives include simulated ground points, simulated pole-shaped objects, simulated building planes, and simulated intersecting planes of buildings; the categories when classifying include the categories corresponding to feature lines and the categories corresponding to simulated ground points; among them, the categories corresponding to feature lines include the categories corresponding to simulated pole-shaped objects, the intersection lines between simulated building planes and the ground, and the intersection lines between simulated intersecting planes of buildings;

[0057] Reference Figure 2 The methods for simulating real point clouds corresponding to urban road scenes using simulation primitives include:

[0058] ① The simulated ground points are represented by multiple circular rings, simulating ground points in the real point cloud corresponding to the urban road scene:

[0059] like Figure 2 As shown, multiple concentric rings with the same center are set as simulated ground points. To make the simulated point cloud closer to the "dense near and sparse far" characteristic of the real point cloud, the diameter of each ring gradually increases and the distance between any two adjacent rings gradually increases from the inside to the outside. Each ring in the simulated ground points is sampled according to a fixed number of points to obtain the point cloud corresponding to the simulated ground points. The point cloud corresponding to the simulated ground points is then cropped, retaining only points whose absolute y-coordinate value is less than a first set threshold (in this embodiment, the first set threshold is 3, that is, only points whose absolute y-coordinate value is less than 3 meters) to form a simulated road, making the formed simulated road more consistent with the actual road scene.

[0060] ② Simulate rod-shaped objects, building planes, and intersecting planes of buildings to simulate rod-shaped objects, the intersection lines of building planes with the ground, and the intersection lines of building planes with each other in the real point cloud corresponding to the urban road scene:

[0061] like Figure 2As shown, a straight line is set as a simulated rod-shaped object, a plane is set as a simulated building plane (the intersection of the plane and the ground is marked with gray lines), and intersecting planes are set as building intersection planes (the intersections of the planes and the ground are marked with gray lines). The set straight line, plane, and intersecting plane are sampled according to a set density to generate simulated point clouds corresponding to the simulated rod-shaped object, simulated building plane, and simulated building intersection plane, respectively. In order to simulate the situation where objects are incomplete due to occlusion in a real scene, a set number or a set proportion of points are cropped from each point cloud in the generated simulated point cloud (in this embodiment, 10% of the points are cropped from each point cloud). The cropped point clouds are placed at a set position on the simulated road. According to the layout of the actual urban road scene, the set position in this embodiment is on both sides of the simulated road.

[0062] In this embodiment, the sampling of the rings in the simulated ground points, as well as the sampling of straight lines, planes, and intersecting planes, is uniform sampling. In other embodiments, the corresponding point cloud data can also be obtained through other sampling methods.

[0063] Following the above simulation method, the points corresponding to the simulated primitives are obtained, thereby simulating the real point cloud corresponding to the urban road scene. The method for classifying the simulated points is as follows:

[0064] Points corresponding to simulated rod-shaped objects are designated as Category 1; points corresponding to the intersection lines of the planes of simulated buildings and the ground are designated as Category 2; points corresponding to the intersection lines between planes of simulated buildings are also designated as Category 2; and other points are designated as Category 3. In this embodiment, simulated independent rod-shaped objects are designated as Category 1, the intersection lines of simulated buildings and the ground, and the intersection lines between simulated building planes are designated as Category 2, and the rest are designated as Category 0. The categories can be distinguished by color, for example, points in Category 1 are marked as green, points in Category 2 as red, and points in Category 3 as black, or by grayscale values ​​(i.e., color depth), which will not be described in detail here.

[0065] In this embodiment, to simulate the noise of a real point cloud, point clouds with varying degrees of noise need to be generated. Therefore, Gaussian noise with zero mean and standard deviation of 0.01mr (mesh resolution) is added to the points corresponding to all simulation primitives in each coordinate axis direction. Furthermore, this embodiment also randomly cropped 2000 points (including vegetation and vehicles) from the real scan data corresponding to the real point cloud as random noise points. These random noise points, along with the points corresponding to the simulation primitives after adding zero mean and Gaussian noise, are used together to obtain the simulated points. Figure 3As shown, the simulation dataset generated in this step (i.e., the simulated points) contains 2000 simulated point clouds, each containing approximately 12000 points. The lightest colored points correspond to the simulated rod-shaped objects, the second lightest colored points correspond to the intersection lines between the plane and the ground in the simulated building plane, and the points correspond to the intersection lines between the planes of the simulated building. The black (darkest) points are the other points.

[0066] This step primarily addresses the current lack of labeled line data for vehicle-mounted laser point clouds and the time-consuming, labor-intensive, and inaccurate nature of manual labeling. By summarizing common and reliable line categories in vehicle-mounted laser point cloud road scenes, a simulated dataset containing category labels for feature lines is constructed. This labeled simulated dataset is then used to train a classification model for acquiring feature lines in urban road scenes, eliminating the need for manual labeling. Furthermore, since category labeling is performed directly during the construction of the simulated dataset, the labeling results are more accurate. The simulated point cloud is also made as close as possible to real point clouds in urban road scenes, providing a training set that more closely resembles real-world point cloud data for the line extraction network.

[0067] 2) Using simulated point cloud as input point cloud data for line extraction network, train the line extraction network to obtain a scale-invariant classification model; the line extraction network obtains the feature lines corresponding to the input point cloud data as output based on the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud.

[0068] Line detection (line extraction) can be viewed as a point cloud classification problem. The main challenge in training a classification model that can accurately classify real point cloud data using a simulated dataset is the scaling problem, specifically the inconsistency between the scale of the simulated dataset and the real-world scene.

[0069] (1) In real vehicle-mounted laser point clouds, the density of the point cloud decreases with the scanning distance. When the target is far away from the sensor, the voxel downsampling cannot completely normalize the density.

[0070] (2) The simulation data generation did not take into account the proportion of line segments.

[0071] If the above issues are not addressed, the network model will be unable to produce reasonable predictions when the training and testing data are at different scales. Therefore, this embodiment eliminates the scaling factor and achieves scale invariance by simulating the scale-invariant local neighborhood features corresponding to each point in the point cloud, ensuring that a classification model that accurately classifies real point cloud data can be trained using a simulated dataset. The specific calculation method for the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud is as follows:

[0072]

[0073] In the formula, f represents the scale-invariant local feature, and p′ To simulate points in a point cloud, p i ′ For p ′ Find the i-th closest point to the center point; search p ′ The k nearest points to a point {p ′ 1, p ′ 2, ..., p ′ k}, calculate p respectively ′ The scale-invariant local features between a point and its k nearest neighbors are used as p. ′ The scale-invariant local neighborhood features corresponding to a point. That is, the method for obtaining the scale-invariant local features f is as follows: for each point in the point cloud, using it as the center point, calculate its neighborhood points through Euclidean distance and set a threshold, where p' is the center point. i Let i be a neighboring point of the corresponding center point, and i be the index of the neighboring point.

[0074] like Figure 4 As shown, the line extraction network includes a feature encoder and a decoder. The feature encoder extracts the feature vector corresponding to the local neighborhood features (i.e., the scale-invariant local neighborhood features corresponding to each point) of the input point cloud data. The decoder converts the feature vector corresponding to the local neighborhood features into a tensor of size N×M, where N represents the number of input points and M represents the number of categories for classifying the simulated points. For example, in this embodiment, the number of categories is 3, so M=3. Then, the tensor is processed by the Softmax function to obtain the probability of each point belonging to each category. Finally, the argmax function is used to obtain the category with the highest probability for each point, and the point is marked as the category with the highest probability. For example, simulated independent rods are marked as category 1, simulated intersections of buildings and the ground and simulated intersections of building planes are marked as category 2, and the rest are marked as category 0. If a point has the highest probability of belonging to category 0, it is marked as category 0; if it has the highest probability of belonging to category 1, it is marked as category 1; if it has the highest probability of belonging to category 2, it is marked as category 2.

[0075] The feature encoder uses a graph attention network (GAT) as the backbone network. GAT is a novel neural network architecture that runs on graph-structured data. It uses a masked self-attention layer to address the drawback of graph-based convolution relying on prior access to the global graph structure or all its nodes, and assigns different weights to different nodes in the neighborhood.

[0076] In this embodiment, a graph attention network is used to extract local neighborhood features of each point in the input point cloud data and assign different attention scores to each neighboring node of that point. The neighboring nodes of a point refer to the k nearest points of that point. During feature line extraction, the line extraction network first uses KNNGraph in DGL to convert the input point cloud data into a graph structure, calculates the distance between each point and its corresponding neighboring points, selects the k nearest neighbors (i.e., the closest points), and constructs the edges between each point and its k nearest neighbors. Then, the graph attention network extracts the scale-invariant local neighborhood features of each point, and uses the softmax function to normalize the attention scores of all neighboring nodes of each point. The normalized attention score of a point corresponds to all its neighboring nodes, and the scale-invariant local features of that point also correspond to all its neighboring nodes. Therefore, a linear combination of the normalized attention score and the corresponding scale-invariant local neighborhood features of each point is calculated, and this linear combination is used as the feature vector corresponding to the local neighborhood features of each point in the input point cloud data.

[0077] Specifically, the local feature encoding function of GAT is as follows:

[0078] h(x j , S x i )=h(x j , S x i -x j )

[0079] Where, x j It is the j-th feature point in the point cloud data. S x i x in the feature space S j The corresponding neighborhood point, h, is the learnable model. In the original GAT layer, x represents the coordinates of a point in Euclidean space. In this embodiment, the scale-invariant local features f of the 20 nearest neighbors of each point are calculated;

[0080] Therefore, in this embodiment, the first EdgeConv layer is input with scale-invariant local features f, replacing the coordinates x in Euclidean space, but E f i It is still the midpoint p in Euclidean space j The features of the i-th neighboring point, rather than f in the feature space. j The neighborhood points; since the original Euclidean space is lost when generating scale-invariant local features, preserving the adjacency relationship in the original Euclidean space can reduce further information loss. The corresponding local feature encoding function of GAT is as follows:

[0081] h(f j , E f i )=h(f j , E f i -f j )

[0082] Among them, f j It is a scale-invariant local feature of the j-th point in point cloud data. E f i f in the feature space E j The scale-invariant local features of the corresponding neighborhood points, h is a learnable model.

[0083] In this embodiment, the loss function Lseg used for training the line extraction network is a standard cross-entropy loss, calculated as follows:

[0084]

[0085] Where L is the cross-entropy loss, and p = [p0, p1, ..., p c-1 ] is a probability distribution, where each element p i Let y represent the probability that a sample belongs to the i-th class; y = [y0, y1, ..., y2]. c-1 ] is the one-hot representation of the sample label. When the sample belongs to the i-th class, y i =1; otherwise y i =0; c is the sample label. When the value of L meets the set threshold range, the training is considered complete, and the trained line extraction network is used as a scale-invariant classification model.

[0086] 3) Classify the real point cloud corresponding to the urban road scene according to the scale-invariant classification model, and verify the classification results. Obtain the feature line of the urban road scene based on the points that pass the verification.

[0087] Since scale-invariant classification models trained on simulated data still produce biased classification results when directly applied to real data, a geometric adaptation scheme is adopted to classify the real point clouds corresponding to urban road scenes based on the scale-invariant classification model. Specifically:

[0088] The real point cloud corresponding to the urban road scene is subjected to random perturbation in the 2D XOY plane. One perturbation data is generated each time. After k perturbations, a perturbation data set PC = {pc1, pc2, ..., pc3} is generated. i ,…,pc k}; where the disturbance data pc iThis refers to the perturbation result obtained by perturbing the real point cloud for the i-th time. In this embodiment, the method of performing two-dimensional random perturbation on the point cloud is to randomly rotate each point in the point cloud. Since the deviation of point cloud data in urban road scenes usually occurs on the ground, i.e., in the x-axis and y-axis directions, the random rotation transformation of each point in the point cloud mainly rotates around the z-axis. That is, the XOY plane is selected when performing two-dimensional planar random perturbation. Performing two-dimensional planar (XOY plane) random perturbation on the point cloud is to randomly generate several transformation matrices that can rotate the point cloud around the Z-axis for data augmentation. Random perturbation of the point cloud only changes the relative coordinate values ​​of the points; the point cloud as a whole remains unchanged. At the same time, because the classification model has instability and error when predicting the category of points, using the classification model to classify these several generated random perturbation point clouds is equivalent to judging the category of each point in a frame of point cloud several times. Then, based on the results of multiple judgments, the most likely category of the point can be determined.

[0089] Each perturbation data point in the perturbation data set is used as input point cloud data. A scale-invariant classification model is used to predict the category of each point in each perturbation data point. Following the classification method described above, simulated independent rods are labeled as category 1, simulated building-to-ground intersections and simulated building-to-plane intersections are labeled as category 2, and the rest are labeled as category 0. If the category number of a point in a given perturbation data point is greater than 0, it means that the point is classified as a simulated independent rod, a simulated building-to-ground intersection, or a simulated building-to-plane intersection, and the label value of that point in the perturbation data point is set to 1. Finally, the label values ​​of that point in all perturbation data points are averaged. If the averaged label value of that point is greater than a set label value threshold, then that point is used as a candidate feature line point. The resulting candidate feature lines are the classification results of the scale-invariant classification model for classifying the real point cloud corresponding to the urban road scene. In this embodiment, the label value threshold is set to 0.8.

[0090] The above classification method is equivalent to judging the classification deviation of each point in the real point cloud data based on the classification results of different two-dimensional transformations of the real point cloud (the larger the average label value, the more times the point is classified as a simulated independent rod, a simulated intersection of a building and the ground, or a simulated intersection of two planes of a building, and the smaller the classification deviation). Points with smaller classification deviations are selected as candidate feature lines, thus obtaining candidate feature lines with higher classification accuracy.

[0091] Since the candidate feature lines corresponding to the above classification results still contain some discrete, non-real feature line points, this embodiment also verifies the classification results and obtains the urban road scene feature lines based on the points that pass the verification, specifically:

[0092] First, the candidate feature line points are clustered using a region growing algorithm. Clusters with fewer than a set threshold number of points are filtered out; in this embodiment, clusters with fewer than 5 points are filtered out. Then, RANSAC is used to fit the remaining clusters to obtain a fitted line, and outliers are removed. Next, the distance from each of the remaining points after outlier removal to the fitted line is calculated, and points with a distance less than a set threshold are retained. Finally, the retained points with a distance less than the set threshold are clustered again, and discrete points are filtered out. The final remaining points are the points that pass the verification. The line formed by the points that pass the verification is used as the feature line of the urban road scene. The above verification can remove discrete and non-realistic feature line points, improving the accuracy of feature line extraction in urban road scenes.

[0093] This invention has the following characteristics:

[0094] 1) To address the current lack of vehicle-mounted laser point cloud line labeling data and the time-consuming, labor-intensive, and inaccurate nature of manual labeling, this paper summarizes common and reliable line categories in vehicle-mounted laser point cloud road scenes and constructs a simulated dataset containing category labels corresponding to feature lines. By training a classification model for obtaining feature lines in urban road scenes using this labeled simulated dataset, feature lines in urban road scenes can be selectively classified and extracted, eliminating the need for manual labeling. Furthermore, since the category labels corresponding to feature lines are directly applied during the construction of the simulated dataset, the labeling results are more accurate.

[0095] 2) To address the issue of inconsistent scale between the simulated dataset and the real scene when a classification model trained on a simulated dataset is used to classify real point cloud data, the scaling factor is eliminated by using scale-invariant local neighborhood features corresponding to each point in the simulated point cloud to achieve scale invariance, thus ensuring that a classification model that can accurately classify real point cloud data can be trained using a simulated dataset.

[0096] 3) To address the issue that scale-invariant classification models trained on simulated data perform poorly when directly applied to real data, a geometric adaptation method is used. Based on the classification results of different two-dimensional transformations of the real point cloud corresponding to the urban road scene by the scale-invariant classification model, the classification deviation of each point in the real point cloud data is judged, and points with smaller classification deviations are selected as candidate feature lines. Therefore, candidate feature lines with higher classification accuracy can be obtained.

[0097] 4) By clustering, fitting, filtering points near the fitted line, and clustering again, the classification results are filtered and verified. Based on the points that pass the verification, the feature lines of the urban road scene are obtained, which can remove discrete and non-real feature line points and improve the accuracy of urban road scene feature line extraction.

[0098] It should be understood that the above-described specific embodiments of the present invention are merely illustrative or explanatory of the principles of the present invention, and do not constitute a limitation thereof.

Claims

1. A method for extracting feature lines in an urban road scene, characterized in that, Includes the following steps: 1) Simulate the real point cloud corresponding to the urban road scene using simulation primitives, and label the simulated points according to their categories to obtain the simulated point cloud corresponding to the urban road scene, so as to construct a simulated dataset containing the category labels corresponding to the feature lines; the categories include the categories corresponding to the feature lines. 2) Using the simulated point cloud as the input point cloud data for the line extraction network, the line extraction network is trained, and the trained line extraction network is used as a scale-invariant classification model; the line extraction network obtains the category of each point in the input point cloud data as the output based on the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud. 3) Classify the real point cloud corresponding to the urban road scene according to the scale-invariant classification model, verify the classification results, and obtain the feature lines of the urban road scene based on the points that pass the verification.

2. The method for extracting feature lines from urban road scenes according to claim 1, characterized in that, The simulation primitives include simulated ground points, simulated rod-shaped objects, simulated building planes, and simulated intersecting planes of buildings; the categories also include the categories corresponding to the simulated ground points. The categories corresponding to the feature lines include those corresponding to simulated rod-shaped objects, the intersection of simulated building planes and the ground, and the intersection between simulated intersecting planes of buildings; The methods for simulating real point clouds corresponding to urban road scenes using simulation primitives include: The simulation of ground points uses multiple rings to simulate ground points in the real point cloud corresponding to the urban road scene. The simulation of rod-shaped objects uses straight lines to simulate rod-shaped objects in the real point cloud corresponding to the urban road scene. The simulation of building planes uses planes to simulate the intersection lines between simulated building planes and the ground in the real point cloud corresponding to the urban road scene. The simulation of intersecting building planes uses intersecting planes to simulate the intersection lines between simulated building planes and planes in the real point cloud corresponding to the urban road scene. The points corresponding to the simulation primitives are obtained, thereby simulating the real point cloud corresponding to the urban road scene.

3. The method for extracting feature lines in urban road scenes according to claim 2, characterized in that, The method for simulating ground points in the real point cloud corresponding to urban road scenes is as follows: Multiple concentric rings with the same center are set as simulated ground points. The diameter of each ring gradually increases and the distance between any two adjacent rings gradually increases from the inside to the outside. Each ring in the simulated ground points is sampled according to a fixed number of points to obtain the point cloud corresponding to the simulated ground points. The point cloud corresponding to the simulated ground points is cropped, and only points whose absolute y-coordinate value is less than a first set threshold are retained to form a simulated road. The methods for simulating rod-shaped objects, building planes, and intersecting building planes in the real point cloud corresponding to urban road scenes are as follows: A straight line is set as a simulated rod-shaped object, a plane is set as a simulated building plane, and intersecting planes are set as building intersection planes; the straight line, plane, and intersecting plane are sampled according to a set density to generate simulated point clouds corresponding to the simulated rod-shaped object, simulated building plane, and simulated building intersection plane, respectively; a set number or a set proportion of points are cropped from each point cloud in the generated simulated point cloud, and the cropped point clouds are placed at a set position on the simulated road.

4. The method for extracting feature lines in urban road scenes according to claim 2, characterized in that, The method of simulating real point clouds corresponding to urban road scenes using simulation primitives also includes: Add a set amount of Gaussian noise with zero mean and standard deviation to the points corresponding to the simulation primitives in each coordinate axis direction, and randomly crop a set number of points corresponding to vegetation and vehicles from the real point cloud data as random noise points. Combine the random noise points with the points corresponding to the simulation primitives after adding zero mean and Gaussian noise as the simulated points.

5. The method for extracting feature lines from urban road scenes according to any one of claims 1-4, characterized in that, The method for classifying the simulated points is as follows: Points corresponding to simulated rod-shaped objects are marked as Category 1, points corresponding to the intersection of the plane and the ground in the plane of the simulated building are marked as Category 2, points corresponding to the intersection of the planes between the planes of the simulated building are also marked as Category 2, and other points are marked as Category 3.

6. The method for extracting feature lines from urban road scenes according to any one of claims 1-4, characterized in that, The method for obtaining the scale-invariant local neighborhood features corresponding to each point in the simulated point cloud is as follows: In the formula, f For scale-invariant local features, Simulate points in a point cloud. For The first point centered on i The closest point; search Pointed k The closest point { , , ..., }, calculate separately Point and k Local features that are scale-invariant between the nearest points, as The scale-invariant local neighborhood features corresponding to a point.

7. The method for extracting feature lines from urban road scenes according to any one of claims 1-4, characterized in that, The line extraction network includes a feature encoder and a decoder; The feature encoder is used to extract the feature vector of scale-invariant local neighborhood features corresponding to each point in the input point cloud data; The decoder is used to convert the feature vector of the scale-invariant local neighborhood features corresponding to each point into a value of size . N × M The tensor, in which N Indicates the number of input points. M This represents the number of categories for classifying the simulated points; based on the tensor, the probability of each point belonging to each category and the category with the highest probability are obtained, and the point is marked as the category with the highest probability.

8. The method for extracting feature lines from urban road scenes according to claim 5, characterized in that, Set the index of the first category to 1, the index of the second category to 2, and the index of the third category to 0. The method for classifying the real point cloud corresponding to the urban road scene according to the scale-invariant classification model is as follows: The real point cloud corresponding to the urban road scene is subjected to random perturbation in the 2D XOY plane. One perturbation data is generated each time, and a perturbation data set is generated after k perturbations. PC ={ PC 1 , pc 2 , …,pc i ,…,pc k }; among which, perturbation data PC i The perturbation result obtained by perturbing the real point cloud for the i-th time; Each perturbation data point in the perturbation data set is used as input point cloud data. A scale-invariant classification model is used to predict the category of each point in each perturbation data point. If the category number of a point in a perturbation data point is greater than 0, the label value of that point in the perturbation data point is set to 1. Finally, the label values ​​of that point in all perturbation data points are averaged. If the average label value of that point is greater than a set label value threshold, then that point is used as a candidate point for feature lines.

9. The method for extracting feature lines from urban road scenes according to any one of claims 1-4, characterized in that, The classification results are validated, and the feature lines of the urban road scene are obtained from the points that pass the validation as follows: All candidate feature line points are clustered using a region growing algorithm, and clusters with fewer than a set threshold number of points are filtered out. Then, a straight line is fitted to the remaining clusters to obtain a fitted line, and outliers are removed. Next, the distance from each of the remaining points after removing outliers to the fitted line is calculated, and points with a distance less than a set threshold are retained. Finally, the retained points with a distance less than the set threshold are clustered again, and after filtering out discrete points, points that pass the verification are obtained. The line formed by the points that pass the verification is used as the feature line of the urban road scene.

10. The method for extracting feature lines in an urban road scene according to claim 7, characterized in that, The feature encoder uses a graph attention network as its backbone network. The graph attention network is used to extract local neighborhood features of each point in the input point cloud data and assign different attention scores to each neighboring node of that point; where a neighboring node of a point refers to that point's... k The closest point; After extracting scale-invariant local neighborhood features for each point using a graph attention network, the attention scores of all neighboring nodes of each point are normalized using the softmax function. The linear combination of the normalized attention score and the corresponding scale-invariant local neighborhood features for each point is calculated, and the linear combination is used as the feature vector corresponding to the local neighborhood features of each point in the input point cloud data.

Images