Laser radar point cloud feature extraction and matching method and device, vehicle and product
By calculating the smoothness and segmentation strategy of lidar point cloud data, feature points are extracted and descriptors are generated, the error and complexity problems of lidar point cloud feature extraction and matching are solved, and the positioning and mapping performance of the autonomous driving system is improved.
Patent Information
- Application Number
- CN202411146823.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2025-01-03
AI Technical Summary
In the prior art, there are large errors and high complexity in the extraction and matching of point cloud features of lidar, and there is a lack of computing cost control and real-time balance, which affects the high-precision positioning and mapping application of autonomous driving systems.
By acquiring the point cloud dataset, the edge point cloud dataset is determined by calculating the smoothness, feature points are extracted based on the preset segmentation strategy and clustering algorithm, descriptors are generated, and feature points similarity is calculated to achieve matching.
It improves the accuracy of feature point extraction and matching accuracy, reduces the computational complexity, and achieves more effective performance improvement, which is suitable for real-time applications of autonomous driving systems.
Smart Images
Figure CN119251514A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of digital image technology, and in particular to a laser radar point cloud feature extraction and matching method, device, vehicle and product. Background Art
[0002] LiDAR sensor is one of the important sensors in the autonomous driving system. It can directly provide the measurement information of the three-dimensional position of the target and plays a very important role in the high-precision positioning module and the map construction module. Among them, the feature extraction and matching technology of the point cloud data collected by LiDAR is the key technology of the sensor in the application of high-precision positioning and mapping.
[0003] In the related technologies, according to different point cloud data processing strategies, the above technologies are divided into two categories: feature extraction through manually designed rules and feature extraction based on deep learning. Among them, the manually designed features are mostly implemented based on histograms, and the representation of feature points is achieved through local or global statistical information. However, due to the similarity of local information in the lidar point cloud data, it may cause mismatching during the matching process and affect subsequent performance; feature points based on global information have high complexity in the calculation process and lack real-time performance. At the same time, the accuracy of their feature results in point-to-point matching is insufficient. The lidar point cloud feature extraction method based on deep learning has made rapid progress in recent years and can achieve accurate feature extraction and matching. However, this solution requires a lot of computing resources. In the autonomous driving system based on embedded devices, there is a lack of balance between computing cost control and computing real-time performance. Summary of the invention
[0004] The present application provides a laser radar point cloud feature extraction and matching method, device, vehicle and product to solve the problems in related technologies, such as large errors and high complexity in laser radar point cloud feature extraction and matching, and lack of balance between computing cost control and computing real-time performance. The method improves the accuracy of feature point extraction and the precision of feature point matching, facilitates the application of high-precision positioning and mapping of autonomous driving, and achieves more effective performance improvement.
[0005] The first aspect of the present application provides a method for extracting and matching laser radar point cloud features, comprising the following steps:
[0006] Acquire a point cloud data set, calculate the smoothness of each point cloud data in the point cloud data set, and determine an edge point cloud data set according to the smoothness;
[0007] Segmenting the edge point cloud data set based on a preset segmentation strategy to obtain a plurality of sub-regions of the edge point cloud data set, processing each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, obtaining a point cloud feature point set based on the feature points of each sub-region, and determining a descriptor of each feature point in the point cloud feature point set;
[0008] The descriptor distance between two adjacent feature points in the point cloud feature point set is calculated, the similarity between the two adjacent feature points is obtained according to the descriptor distance between the two adjacent feature points, and the matching result between the two adjacent feature points is obtained according to the similarity.
[0009] Optionally, in some embodiments, calculating the smoothness of each point cloud data in the point cloud data set and determining the edge point cloud data set according to the smoothness includes:
[0010] Determine the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and calculate the smoothness of each point cloud data according to the line bundle information and the adjacent point cloud data;
[0011] The edge point cloud data set is determined according to the smoothness of each point cloud data and a first preset threshold.
[0012] Optionally, in some embodiments, segmenting the edge point cloud dataset based on a preset segmentation strategy to obtain a plurality of sub-regions of the edge point cloud dataset includes:
[0013] Divide the edge point cloud data set into a plurality of first regions, and count the number of elements in each first region;
[0014] The range of each first region is adjusted according to the number of elements in each first region and the expected number of segmentations to obtain a plurality of sub-regions of the edge point cloud data set.
[0015] Optionally, in some embodiments, processing each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, and obtaining a point cloud feature point set according to the feature points of each sub-region includes:
[0016] Processing each sub-region based on a preset clustering algorithm to obtain a clustering result for each sub-region;
[0017] Determine the number of elements in each clustering result and the number of line bundles of the elements in each clustering result, and judge the validity of the corresponding sub-region according to the number of elements in each clustering result and the number of line bundles of the elements in the clustering result of each sub-region;
[0018] The center point of the effective sub-region is used as the feature point of the corresponding effective sub-region, and the point cloud feature point set is obtained according to the feature point of each effective sub-region.
[0019] Optionally, in some embodiments, determining a descriptor of each feature point in the point cloud feature point set includes:
[0020] Based on a preset direction, the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, and a plurality of data points adjacent to the current feature point in the edge point cloud data set are determined in each second area;
[0021] The Euclidean distance between the current feature point and each data point is calculated respectively, and a descriptor of the current feature point is obtained according to the Euclidean distances of all second regions of the current feature point.
[0022] Optionally, in some embodiments, after equally dividing the acquisition area into a plurality of second areas with the current feature point as the center, the method further includes:
[0023] The number of elements in each second region is determined, and the main direction of the descriptor of the current feature point is determined according to the number of elements in each second region.
[0024] Optionally, in some embodiments, before acquiring the point cloud dataset, the method further includes:
[0025] An original point cloud data set is collected, abnormal point cloud data in the original point cloud data set is removed, and the point cloud data set is generated according to the remaining point cloud data.
[0026] The second aspect of the present application provides a laser radar point cloud feature extraction and matching device, including:
[0027] A first extraction module is used to obtain a point cloud data set, calculate the smoothness of each point cloud data in the point cloud data set, and determine an edge point cloud data set according to the smoothness;
[0028] A second extraction module is used to segment the edge point cloud data set based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, process each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, obtain a point cloud feature point set based on the feature points of each sub-region, and determine a descriptor for each feature point in the point cloud feature point set;
[0029] The matching module is used to calculate the descriptor distance between two adjacent feature points in the point cloud feature point set, obtain the similarity of the two adjacent feature points according to the descriptor distance between the two adjacent feature points, and obtain the matching result of the two adjacent feature points according to the similarity.
[0030] Optionally, in some embodiments, the first extraction module is specifically used to:
[0031] Determine the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and calculate the smoothness of each point cloud data according to the line bundle information and the adjacent point cloud data;
[0032] The edge point cloud data set is determined according to the smoothness of each point cloud data and a first preset threshold.
[0033] Optionally, in some embodiments, the second extraction module is specifically used to:
[0034] Divide the edge point cloud data set into a plurality of first regions, and count the number of elements in each first region;
[0035] The range of each first region is adjusted according to the number of elements in each first region and the expected number of segmentations to obtain a plurality of sub-regions of the edge point cloud data set.
[0036] Optionally, in some embodiments, the second extraction module is specifically used to:
[0037] Processing each sub-region based on a preset clustering algorithm to obtain a clustering result for each sub-region;
[0038] Determine the number of elements in each clustering result and the number of line bundles of the elements in each clustering result, and judge the validity of the corresponding sub-region according to the number of elements in each clustering result and the number of line bundles of the elements in the clustering result of each sub-region;
[0039] The center point of the effective sub-region is used as the feature point of the corresponding effective sub-region, and the point cloud feature point set is obtained according to the feature point of each effective sub-region.
[0040] Optionally, in some embodiments, the second extraction module is specifically used to:
[0041] Based on a preset direction, the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, and a plurality of data points adjacent to the current feature point in the edge point cloud data set are determined in each second area;
[0042] The Euclidean distance between the current feature point and each data point is calculated respectively, and a descriptor of the current feature point is obtained according to the Euclidean distances of all second regions of the current feature point.
[0043] Optionally, in some embodiments, after the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, the second extraction module is further used to:
[0044] The number of elements in each second region is determined, and the main direction of the descriptor of the current feature point is determined according to the number of elements in each second region.
[0045] Optionally, in some embodiments, before acquiring the point cloud data set, the first extraction module is further used to:
[0046] An original point cloud data set is collected, abnormal point cloud data in the original point cloud data set is removed, and the point cloud data set is generated according to the remaining point cloud data.
[0047] The third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the laser radar point cloud feature extraction and matching method as described in the above embodiment.
[0048] The fourth aspect of the present application provides a computer program product on which a computer program is stored, and the program is executed to implement the laser radar point cloud feature extraction and matching method as described in the above embodiment.
[0049] Thus, by obtaining a point cloud data set, calculating the smoothness of each point cloud data in the point cloud data set, and determining the edge point cloud data set based on the smoothness, the edge point cloud data set is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, and each sub-region is processed based on a preset clustering algorithm to obtain the feature points of each sub-region, and a point cloud feature point set is obtained based on the feature points of each sub-region, and the descriptor of each feature point in the point cloud feature point set is determined, and the distance between the descriptors of two adjacent feature points in the point cloud feature point set is calculated, and the similarity of the two adjacent feature points is obtained based on the distance between the descriptors of the two adjacent feature points, and the matching result of the two adjacent feature points is obtained based on the similarity. Thus, the problems in the related technology that the laser radar point cloud feature extraction and matching have large errors and high complexity, and lack the balance between the computational cost control and the computational real-time performance are solved, the accuracy of feature point extraction and the accuracy of feature point matching are improved, and the application of high-precision positioning and mapping of autonomous driving is facilitated, and more effective performance improvement is achieved.
[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0052] Figure 1A flowchart of a laser radar point cloud feature extraction and matching method provided according to an embodiment of the present application;
[0053] Figure 2 is a schematic diagram of edge feature segmentation according to a specific embodiment of the present application;
[0054] Figure 3 A flowchart of feature point extraction according to a specific embodiment of the present application;
[0055] Figure 4 A schematic block diagram of a laser radar point cloud feature extraction and matching device provided according to an embodiment of the present application;
[0056] Figure 5 It is a block diagram of a vehicle provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0057] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0058] The following describes the laser radar point cloud feature extraction and matching method, device, vehicle and product of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the laser radar point cloud feature extraction and matching mentioned in the above background technology has large errors and high complexity, and lacks the balance between computational cost control and computational real-time performance, the present application provides a laser radar point cloud feature extraction and matching method, in which a point cloud data set is obtained, the smoothness of each point cloud data in the point cloud data set is calculated, and the edge point cloud data set is determined according to the smoothness; the edge point cloud data set is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, each sub-region is processed based on a preset clustering algorithm to obtain feature points of each sub-region, a point cloud feature point set is obtained according to the feature points of each sub-region, and a descriptor of each feature point in the point cloud feature point set is determined; the distance between the descriptors of two adjacent feature points in the point cloud feature point set is calculated, the similarity of the two adjacent feature points is obtained according to the distance between the descriptors of the two adjacent feature points, and the matching result of the two adjacent feature points is obtained according to the similarity. As a result, the problems of large errors and high complexity in lidar point cloud feature extraction and matching, and the lack of balance between computing cost control and computing real-time performance are solved. The accuracy of feature point extraction and the precision of feature point matching are improved, which will help the application of high-precision positioning and mapping of autonomous driving and achieve more effective performance improvement.
[0059] Specifically, Figure 1A flowchart of the laser radar point cloud feature extraction and matching method provided in an embodiment of the present application.
[0060] like Figure 1 As shown, the laser radar point cloud feature extraction and matching method includes the following steps:
[0061] In step S101, a point cloud data set is acquired, the smoothness of each point cloud data in the point cloud data set is calculated, and an edge point cloud data set is determined according to the smoothness.
[0062] In the embodiment of the present application, a point cloud dataset is collected in real time by a laser radar. The point cloud dataset represents the laser radar point cloud collected at a given time k. Each point cloud data sampled by the laser radar includes the three-dimensional position information of the point in the laser radar coordinate system and the laser radar beam information. For example, the point cloud dataset P k ={p 1 , p 2 , ..., p N} represents the point cloud collected by the laser radar at the kth moment, which is the sum of several points. N represents the number of points contained in the point cloud. Each point cloud data p i ∈R 4 ={x, y, z, r} represents a point sampled by the laser radar, the subscript i represents the serial number, x, y and z represent the three-dimensional position information of the point in the laser radar coordinate system, and r represents the beam information of the laser radar.
[0063] It is understandable that in the field of two-dimensional visual image processing, edge extraction strategies based on pixel grayscale gradient changes can achieve effective edge feature detection and have been widely used. The present application applies this idea to feature extraction of three-dimensional lidar point clouds. Specifically, the embodiments of the present application use the smoothness of local measurements in point cloud data to extract edge feature points in the point cloud. In fact, lidar sensors can collect information about targets, where the edge of the target is an effective means of describing it, satisfying the characteristics of downsampling and description effectiveness in feature point extraction strategies.
[0064] Therefore, the embodiment of the present application calculates the smoothness of each point cloud data in the point cloud data set and determines the edge point cloud data set according to the smoothness. In other words, the embodiment of the present application realizes edge extraction based on the local smoothness of the point cloud data.
[0065] Optionally, in some embodiments, the smoothness of each point cloud data in the point cloud data set is calculated, and the edge point cloud data set is determined based on the smoothness, including: determining the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and calculating the smoothness of each point cloud data based on the line bundle information and the adjacent point cloud data; determining the edge point cloud data set based on the smoothness of each point cloud data and a first preset threshold.
[0066] It is understandable that the calculation of the smoothness of point cloud data usually involves the distance or angle difference between adjacent points in the point cloud. The smaller the distance, the closer the point is to its adjacent points, and the higher the smoothness. Therefore, the embodiment of the present application needs to determine the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and determine the window range size for calculating the smoothness, so as to calculate the smoothness of each point cloud data in the point cloud data set.
[0067] Furthermore, the embodiment of the present application sets a threshold for screening edge points, that is, the first preset threshold of the embodiment of the present application, and the edge points of the point cloud dataset are screened by the first preset threshold to generate the edge point cloud dataset of the embodiment of the present application.
[0068] Specifically, the calculation method of the smoothness of a certain point cloud data in the point cloud data set can be mathematically described as follows: Given a point cloud set P collected at time k k , the point at which smoothness needs to be calculated is Then the smoothness can be calculated by formula (1):
[0069]
[0070] in, is the smoothness of a point in the point cloud dataset, Is the point cloud data Adjacent points with the same line information, W represents the window size required for calculating smoothness.
[0071] It should be noted that the above-mentioned window range size W can be determined by statistical analysis or experimental testing, and is not specifically limited here.
[0072] Furthermore, the original point cloud dataset P of the embodiment of the present application k The edge features of the edge point cloud data set of the embodiment of the present application can be expressed by the following formula (2):
[0073]
[0074] in, is the edge point cloud dataset, P k is the original point cloud dataset, is the smoothness of a point in the point cloud dataset, and eThres is the threshold information used for edge point screening.
[0075] Therefore, the embodiments of the present application can more accurately estimate the smoothness of each point by reasonably selecting adjacent point cloud data and considering the line bundle information, thereby using smoothness as an important indicator for point cloud feature extraction, improving the accuracy of edge detection, reducing computational costs while ensuring accuracy, and by calculating the smoothness first and then screening the edge points, the amount of data for subsequent processing can be significantly reduced.
[0076] Optionally, in some embodiments, before acquiring the point cloud data set, the method further includes: collecting an original point cloud data set, removing abnormal point cloud data in the original point cloud data set, and generating a point cloud data set based on the remaining point cloud data.
[0077] It can be understood that given the laser radar point cloud collected at time k, feature point extraction is to sample a subset of the point cloud The elements in the subset can achieve downsampling of the original data and can achieve effective description of the original data. Before extracting feature points, the embodiment of the present application needs to preprocess the original point cloud data set, that is, remove abnormal points, so as to retain the valid point cloud data in the detection area and obtain the point cloud data set of the embodiment of the present application.
[0078] Among them, the abnormal point cloud data of the embodiment of the present application includes: detection data with a distance that is too close or too far, and unconventional values.
[0079] Optionally, the embodiments of the present application can identify and eliminate these abnormal points through statistical analysis, distance threshold judgment, cluster analysis and other methods.
[0080] Therefore, the impact of noise on subsequent processing can be effectively reduced, the overall quality of point cloud data can be improved, and the smoothness of each point cloud in the point cloud data set can be calculated, and the edge point cloud data set can be determined based on the smoothness. This greatly improves the processing efficiency and accuracy of point cloud data, which is the basis for subsequent analysis and application.
[0081] In step S102, the edge point cloud data set is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, each sub-region is processed based on a preset clustering algorithm to obtain feature points of each sub-region, a point cloud feature point set is obtained based on the feature points of each sub-region, and a descriptor of each feature point in the point cloud feature point set is determined.
[0082] It is understandable that the feature point extraction of point cloud data is performed on the basis of edge features. In theory, edge features can effectively describe the original point cloud data and have the characteristics of feature points. However, in actual application, due to the high complexity of the environment, there will be a certain degree of interference, and the existence of external points in the edge feature data will affect subsequent use. Therefore, the present application applies a clustering method, that is, a preset clustering algorithm in an embodiment of the present application, to further extract edge feature data and ultimately achieve more stable feature points.
[0083] In addition, given the edge features As well as the desired number of cluster centers, the above functions can be achieved using a clustering algorithm. In the actual application of autonomous driving scenarios, edge features are mostly distributed vertically. Clustering based on violence will cause a certain edge feature to be vertically segmented, and its computational cost is also high. In order to solve the above problems, this application first segments the point cloud data. Due to the uneven distribution of data in space, the segmentation process also follows the principle of uneven segmentation, which is based on histogram statistics.
[0084] Optionally, in some embodiments, the edge point cloud dataset is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud dataset, including: dividing the edge point cloud dataset into multiple first regions, and counting the number of elements in each first region; adjusting the range of each first region according to the number of elements in each first region and the expected number of segmentations to obtain multiple sub-regions of the edge point cloud dataset.
[0085] Specifically, Figure 2 This is a schematic diagram of edge feature segmentation of a specific embodiment of the present application, such as Figure 2 As shown in (a), the embodiment of the present application can divide the edge point cloud data set into 8 equal parts to obtain 8 first regions, which are respectively recorded as N1-N8. Then, the embodiment of the present application counts the number of elements in each equally divided region, and combines the number of regions N finally divided into s , that is, the expected number of divisions in the embodiment of the present application, each equally divided area is further divided to obtain Figure 2 The segmentation effect shown in (b) in FIG. 1 is as follows. Then, the number of sub-regions that are further divided in each equally divided region can be calculated by formula (3).
[0086]
[0087] in, is the number of equally divided regions, N e is the total number of elements, N s is the number of regions that are finally segmented.
[0088] Therefore, the embodiment of the present application can divide the edge feature plane into N s The subsequent feature point extraction is performed in each area separately.
[0089] Optionally, in some embodiments, each sub-region is processed based on a preset clustering algorithm to obtain feature points of each sub-region, and a point cloud feature point set is obtained based on the feature points of each sub-region, including: processing each sub-region based on a preset clustering algorithm to obtain a clustering result for each sub-region; determining the number of elements in each clustering result and the number of line bundles of the elements in each clustering result, and judging the validity of the corresponding sub-region based on the number of elements in each clustering result and the number of line bundles of the elements in the clustering result of each sub-region; taking the center point of the valid sub-region as the feature point of the corresponding valid sub-region, and obtaining a point cloud feature point set based on the feature points of each valid sub-region.
[0090] Among them, the preset clustering algorithm in the embodiment of the present application is the Kmeans++ algorithm.
[0091] Specifically, the number of point clouds contained in the i-th region after segmentation is defined as The present application embodiment uses the Kmeans++ algorithm to The data is processed to obtain clustering results Among them, C is the number of clustering results.
[0092] Furthermore, in order to ensure the validity of the clustering results, the embodiment of the present application performs a threshold screening process on each clustering result. Specifically, for the jth clustering result in the i-th region The result is considered valid only when the number of elements in the result exceeds the set threshold sThres and the number of wire bundles contained in the elements exceeds the threshold rThres. is set as a feature point, and the set of feature points obtained by performing the above operation on the segmentation results in each region is the point cloud feature point finally extracted, that is, the point cloud feature point set of the embodiment of the present application, which can be defined as
[0093] For example, Figure 3 This is a flowchart of feature point extraction in a specific embodiment of the present application, such as Figure 3As shown, the embodiment of the present application obtains original point cloud data through a laser radar, and in the abnormal point removal stage, retains the valid point cloud data in the detection area, and removes the detection data that is too close or too far and has unconventional values; then, edge features are extracted from the valid point cloud data, that is, edge extraction is achieved based on the local smoothness of the point cloud data; finally, in the feature point extraction stage, the embodiment of the present application further processes the edge points, eliminates the outliers, and achieves robust feature extraction.
[0094] Therefore, the embodiment of the present application performs more refined segmentation in high-density areas through segmentation processing, which helps to more accurately identify edge features and feature points and significantly reduce the computational burden of the clustering algorithm on global data; combined with the clustering method, point cloud areas with similar characteristics are effectively identified and extracted, thereby obtaining more stable feature points; in addition, the embodiment of the present application can also meet the real-time requirements in autonomous driving scenarios by optimizing the computational efficiency of clustering algorithms and segmentation methods.
[0095] Furthermore, how to match the elements in the two sets of feature data, that is, how to calculate the similarity of feature points, is a problem to be discussed in the subsequent content of this application. In the related technology, the nearest neighbor-based matching strategy is one of the effective means to solve the above problems. However, the nearest neighbor matching process calculates the similarity based on the Euclidean distance of the feature points, such as the iterative closest point and ICP (Iterative Closest Point) algorithm, which may cause mismatches and cause the algorithm to fall into a local optimum.
[0096] In addition, based on the above embodiments, it can be understood that each element in the point cloud feature point is obtained by extracting the original data. In essence, the representative points in the original data only contain three-dimensional position information and wiring harness information, and cannot provide rich similarity information in the matching process. Therefore, the embodiment of the present application generates an additional set of corresponding descriptors for each feature point, represented by a vector with a length of 256 dimensions. The descriptors of the embodiment of the present application should have the following characteristics: when two feature points have a high degree of similarity, the distance between their descriptors is small; when the similarity between two feature points is low, the distance between their descriptors is large.
[0097] Optionally, in some embodiments, determining a descriptor for each feature point in a point cloud feature point set includes: based on a preset direction, dividing the acquisition area into a plurality of second areas with the current feature point as the center, and determining a plurality of data points adjacent to the current feature point in an edge point cloud data set in each second area; respectively calculating the Euclidean distance between the current feature point and each data point, and obtaining a descriptor for the current feature point based on the Euclidean distances of all second areas of the current feature point.
[0098] Among them, the preset direction of the embodiment of the present application is the horizontal direction.
[0099] Specifically, the descriptor proposed in the embodiment of the present application is implemented based on the local attributes in the point cloud data. For a given point cloud data set P k , extract features and get edge features With the feature point set and for A feature in First, p i The space is divided into 64 equal parts horizontally with the position as the center (with Figure 2 The segmentation method is similar to that shown in (a) in FIG. 5 ), and 64 second regions are obtained.
[0100] Then, find the edge features in each second region In and p i The four nearest points, that is, multiple data points in the embodiment of the present application, have their Euclidean distances calculated separately (when the number of nearest points is less than 4, 0 is used to fill in the gaps), and the data in the entire area are concatenated to realize the extraction of the descriptor, which can effectively characterize the local characteristics of the feature points.
[0101] Optionally, in some embodiments, after the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, the method further includes: determining the number of elements in each second area, and determining the main direction of the descriptor of the current feature point according to the number of elements in each second area.
[0102] It is understandable that the above generation method cannot overcome the problem of rotational invariance, that is, when the lidar sensor rotates, the order of the vector elements in its descriptor will change. In order to solve the problem of rotational invariance, this application designs the main direction of the descriptor and determines the order of the descriptor vector elements through the main direction.
[0103] Specifically, the core idea is that the rotation process of the laser radar does not cause a change in the layout relationship of the point cloud data. Therefore, after the detection area is divided into 64 equal parts, the embodiment of the present application first calculates the number of elements in each segmented area, takes the area with the largest number of elements as the starting position, and traverses all areas in a counterclockwise direction to finally generate the descriptor.
[0104] Therefore, the embodiment of the present application can provide richer similarity information and enhance the recognition and matching capabilities in the point cloud data matching process by generating a set of corresponding descriptors for each feature point. Moreover, the design of the descriptor makes feature points with high similarity have a smaller descriptor distance, while feature points with low similarity have a larger descriptor distance, thereby greatly improving the matching accuracy. In addition, the embodiment of the present application solves the rotational invariance problem by designing the main direction of the descriptor, that is, when the lidar sensor rotates, the order of the vector elements of the descriptor will not change, maintaining the consistency of the descriptor, thereby improving the accuracy of feature point matching.
[0105] It should be noted that the generation of descriptors in the embodiments of the present application is automated and standardized. The process is relatively simple and easy to implement and deploy in practical applications, which improves the quality and efficiency of point cloud data processing and provides a more powerful and flexible tool for the analysis and application of point cloud data.
[0106] In step S103, the descriptor distance between two adjacent feature points in the point cloud feature point set is calculated, the similarity between the two adjacent feature points is obtained according to the descriptor distance between the two adjacent feature points, and the matching result between the two adjacent feature points is obtained according to the similarity.
[0107] Among them, the two adjacent feature points in the embodiment of the present application can be two adjacent frames of point cloud data, can be two frames of point cloud data at different positioning points, etc.
[0108] Based on the above embodiment, it can be known that given two frames of laser radar point cloud data set P k and P m , the feature data can be obtained by the feature extraction method of the embodiment of the present application and Furthermore, the embodiment of the present application calculates the descriptor of each feature point in the feature point set. Thus, the embodiment of the present application can determine the similarity between two adjacent feature point descriptors in the point cloud feature point set based on the calculated distance between the descriptors to obtain the final matching result.
[0109] The present application embodiment provides a method for calculating the distance between descriptors. Specifically, given two descriptors d i ∈R 256 , d j ∈R 256 The distance calculation includes three steps: offset ratio calculation, binarization processing and Hamming distance calculation. The output result of the offset ratio calculation process is d os ∈R 256 , the calculation result is shown in formula (4):
[0110]
[0111] Among them, di and d j is the descriptor, and r is the position of the element in the descriptor vector.
[0112] The output of the binarization process is d bin ∈R 256 , the calculation result is shown in formula (5):
[0113]
[0114] Wherein, bThres is the threshold of binarization processing.
[0115] The calculation of the final descriptor distance is based on the idea of Hamming distance, that is, statistical The number of non-zero elements in .
[0116] Therefore, the present application can more accurately judge the similarity between feature points by calculating the distance between descriptors, and improve the robustness of feature matching through a combination of offset ratio calculation, binarization processing and Hamming distance calculation, wherein the calculation of the Hamming distance is based on the statistical number of non-zero elements, and has lower computational complexity than other distance metrics, such as Euclidean distance or cosine similarity, and the binarization processing and Hamming distance calculation help to reduce the impact of noise and other interference factors, thereby improving the reliability of matching.
[0117] According to the laser radar point cloud feature extraction and matching method proposed in the embodiment of the present application, by acquiring a point cloud data set, calculating the smoothness of each point cloud data in the point cloud data set, and determining the edge point cloud data set based on the smoothness, the edge point cloud data set is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, and each sub-region is processed based on a preset clustering algorithm to obtain the feature points of each sub-region, and a point cloud feature point set is obtained based on the feature points of each sub-region, and the descriptor of each feature point in the point cloud feature point set is determined, and the distance between the descriptors of two adjacent feature points in the point cloud feature point set is calculated, and the similarity of the two adjacent feature points is obtained based on the distance between the descriptors of the two adjacent feature points, and the matching result of the two adjacent feature points is obtained based on the similarity. Thus, the problems of large errors and high complexity in laser radar point cloud feature extraction and matching in the related technology, and the lack of balance between computational cost control and computational real-time performance are solved, and the accuracy of feature point extraction and the accuracy of feature point matching are improved, which is helpful for the application of high-precision positioning and mapping of autonomous driving, and achieves more effective performance improvement.
[0118] Next, the laser radar point cloud feature extraction and matching device proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings.
[0119] Figure 4It is a block diagram of a laser radar point cloud feature extraction and matching device according to an embodiment of the present application.
[0120] like Figure 4 As shown, the laser radar point cloud feature extraction and matching device 10 includes: a first extraction module 100, a second extraction module 200 and a matching module 300.
[0121] Specifically, the first extraction module 100 is used to obtain a point cloud data set, calculate the smoothness of each point cloud data in the point cloud data set, and determine the edge point cloud data set based on the smoothness; the second extraction module 200 is used to segment the edge point cloud data set based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, process each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, obtain a point cloud feature point set based on the feature points of each sub-region, and determine the descriptor of each feature point in the point cloud feature point set; the matching module 300 is used to calculate the distance between the descriptors of two adjacent feature points in the point cloud feature point set, obtain the similarity of the two adjacent feature points based on the distance between the descriptors of the two adjacent feature points, and obtain the matching results of the two adjacent feature points based on the similarity.
[0122] Optionally, in some embodiments, the first extraction module 100 is specifically used to: determine the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and calculate the smoothness of each point cloud data based on the line bundle information and the adjacent point cloud data; determine the edge point cloud data set based on the smoothness of each point cloud data and a first preset threshold.
[0123] Optionally, in some embodiments, the second extraction module 200 is specifically used to: divide the edge point cloud dataset into multiple first regions, and count the number of elements in each first region; adjust the range of each first region according to the number of elements in each first region and the expected number of segmentations, to obtain multiple sub-regions of the edge point cloud dataset.
[0124] Optionally, in some embodiments, the second extraction module 200 is specifically used to: process each sub-region based on a preset clustering algorithm to obtain a clustering result for each sub-region; determine the number of elements in each clustering result and the number of line bundles of the elements in each clustering result, and judge the validity of the corresponding sub-region based on the number of elements in each clustering result and the number of line bundles of the elements in the clustering result of each sub-region; use the center point of the valid sub-region as the feature point of the corresponding valid sub-region, and obtain a point cloud feature point set based on the feature points of each valid sub-region.
[0125] Optionally, in some embodiments, the second extraction module 200 is specifically used to: divide the acquisition area into multiple second areas based on a preset direction and with the current feature point as the center, and determine multiple data points adjacent to the current feature point in the edge point cloud data set in each second area; calculate the Euclidean distance between the current feature point and each data point respectively, and obtain the descriptor of the current feature point based on the Euclidean distance of all the second areas of the current feature point.
[0126] Optionally, in some embodiments, after the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, the second extraction module 200 is further used to: determine the number of elements in each second area, and determine the main direction of the descriptor of the current feature point according to the number of elements in each second area.
[0127] Optionally, in some embodiments, before acquiring the point cloud dataset, the first extraction module 100 is further used to: collect the original point cloud dataset, remove abnormal point cloud data in the original point cloud dataset, and generate a point cloud dataset based on the remaining point cloud data.
[0128] It should be noted that the aforementioned explanation of the embodiment of the laser radar point cloud feature extraction and matching method is also applicable to the laser radar point cloud feature extraction and matching device of this embodiment, and will not be repeated here.
[0129] According to the laser radar point cloud feature extraction and matching device proposed in the embodiment of the present application, by acquiring a point cloud data set, calculating the smoothness of each point cloud data in the point cloud data set, and determining the edge point cloud data set based on the smoothness, the edge point cloud data set is segmented based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, and each sub-region is processed based on a preset clustering algorithm to obtain the feature points of each sub-region, and a point cloud feature point set is obtained based on the feature points of each sub-region, and the descriptor of each feature point in the point cloud feature point set is determined, and the distance between the descriptors of two adjacent feature points in the point cloud feature point set is calculated, and the similarity of the two adjacent feature points is obtained based on the distance between the descriptors of the two adjacent feature points, and the matching result of the two adjacent feature points is obtained based on the similarity. Thus, the problems of large errors and high complexity in laser radar point cloud feature extraction and matching in the related technology, and the lack of balance between computational cost control and computational real-time performance are solved, the accuracy of feature point extraction and the accuracy of feature point matching are improved, and the application of high-precision positioning and mapping of autonomous driving is facilitated, and more effective performance improvement is achieved.
[0130] Figure 5 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:
[0131] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .
[0132] When the processor 502 executes the program, the laser radar point cloud feature extraction and matching method provided in the above embodiment is implemented.
[0133] Furthermore, the vehicle also includes:
[0134] The communication interface 503 is used for communication between the memory 501 and the processor 502 .
[0135] The memory 501 is used to store computer programs that can be executed on the processor 502 .
[0136] The memory 501 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.
[0137] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0138] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.
[0139] The processor 502 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.
[0140] An embodiment of the present application also provides a computer program product, on which a computer program is stored, and when the program is executed, the above-mentioned laser radar point cloud feature extraction and matching method is implemented.
[0141] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0142] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0143] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0144] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.
[0145] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0146] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A laser radar point cloud feature extraction and matching method, characterized in that: The following steps are involved: Acquire a point cloud data set, calculate the smoothness of each point cloud data in the point cloud data set, and determine an edge point cloud data set according to the smoothness; Segmenting the edge point cloud data set based on a preset segmentation strategy to obtain a plurality of sub-regions of the edge point cloud data set, processing each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, obtaining a point cloud feature point set based on the feature points of each sub-region, and determining a descriptor of each feature point in the point cloud feature point set; The descriptor distance between two adjacent feature points in the point cloud feature point set is calculated, the similarity between the two adjacent feature points is obtained according to the descriptor distance between the two adjacent feature points, and the matching result between the two adjacent feature points is obtained according to the similarity.
2. The method according to claim 1, characterized in that The step of calculating the smoothness of each point cloud data in the point cloud data set and determining the edge point cloud data set according to the smoothness includes: Determine the line bundle information of each point cloud data and the adjacent point cloud data of each point cloud data, and calculate the smoothness of each point cloud data according to the line bundle information and the adjacent point cloud data; The edge point cloud data set is determined according to the smoothness of each point cloud data and a first preset threshold.
3. The method according to claim 1, characterized in that The edge point cloud data set is segmented based on a preset segmentation strategy to obtain a plurality of sub-regions of the edge point cloud data set, including: Divide the edge point cloud data set into a plurality of first regions, and count the number of elements in each first region; The range of each first region is adjusted according to the number of elements in each first region and the expected number of segmentations to obtain a plurality of sub-regions of the edge point cloud data set.
4. The method according to claim 3, characterized in that The method of processing each sub-region based on a preset clustering algorithm to obtain a feature point of each sub-region and obtaining a point cloud feature point set according to the feature point of each sub-region includes: Processing each sub-region based on a preset clustering algorithm to obtain a clustering result for each sub-region; Determine the number of elements in each clustering result and the number of line bundles of the elements in each clustering result, and judge the validity of the corresponding sub-region according to the number of elements in each clustering result and the number of line bundles of the elements in the clustering result of each sub-region; The center point of the effective sub-region is used as the feature point of the corresponding effective sub-region, and the point cloud feature point set is obtained according to the feature point of each effective sub-region.
5. The method according to claim 4, characterized in that The determining of the descriptor of each feature point in the point cloud feature point set includes: Based on a preset direction, the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, and a plurality of data points adjacent to the current feature point in the edge point cloud data set are determined in each second area; The Euclidean distance between the current feature point and each data point is calculated respectively, and a descriptor of the current feature point is obtained according to the Euclidean distances of all second regions of the current feature point.
6. The method according to claim 5, characterized in that After the acquisition area is equally divided into a plurality of second areas with the current feature point as the center, the method further includes: The number of elements in each second region is determined, and the main direction of the descriptor of the current feature point is determined according to the number of elements in each second region.
7. The method according to claim 1, characterized in that Before obtaining the point cloud dataset, it also includes: An original point cloud data set is collected, abnormal point cloud data in the original point cloud data set is removed, and the point cloud data set is generated according to the remaining point cloud data.
8. A laser radar point cloud feature extraction and matching device, characterized in that: include: A first extraction module is used to obtain a point cloud data set, calculate the smoothness of each point cloud data in the point cloud data set, and determine an edge point cloud data set according to the smoothness; A second extraction module is used to segment the edge point cloud data set based on a preset segmentation strategy to obtain multiple sub-regions of the edge point cloud data set, process each sub-region based on a preset clustering algorithm to obtain feature points of each sub-region, obtain a point cloud feature point set based on the feature points of each sub-region, and determine a descriptor for each feature point in the point cloud feature point set; The matching module is used to calculate the descriptor distance between two adjacent feature points in the point cloud feature point set, obtain the similarity of the two adjacent feature points according to the descriptor distance between the two adjacent feature points, and obtain the matching result of the two adjacent feature points according to the similarity.
9. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the laser radar point cloud feature extraction and matching method as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the laser radar point cloud feature extraction and matching method as described in any one of claims 1 to 7.