Laser point cloud parameterized semantic feature matching method, device, equipment and storage medium

Abstract

The embodiments of the present invention provide a method, apparatus, device and storage medium for parameterized semantic feature matching of laser point clouds. The parameterized semantic feature matching method of laser point clouds includes: obtaining a semantic point cloud for parameterized processing from a laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; performing parameterized processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combining the parameterized semantic features of corresponding types into a current parameterized semantic feature frame; combining multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph; performing line-to-line and surface-to-surface feature matching based on at least the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar. The embodiments of the present invention improve the accuracy of the laser radar pose obtained based on matching the geometric features of the laser point cloud.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a laser point cloud parameterized semantic feature matching method, apparatus, device and storage medium. Background Art

[0002] Simultaneous localization and mapping (SLAM) provides the data foundation for control decisions in intelligent driving and intelligent robotics, and is crucial for their safe operation. LiDAR (LiDAR) offers high measurement accuracy and is unaffected by ambient light. Therefore, LiDAR-based positioning and mapping is the mainstream solution for this purpose. Laser odometry is a fundamental technology in LiDAR-based positioning and mapping solutions. The higher the accuracy of laser odometry, the more beneficial it is for positioning and mapping.

[0003] Existing laser odometry calculations typically use the geometric features of a laser point cloud to match a laser point cloud frame (referred to as a point cloud frame) to a laser point cloud submap (referred to as a submap) to determine the LiDAR's pose. However, excessive noise on the submap's surface can affect the accuracy of the point cloud frame-submap match, and thus the accuracy of the determined LiDAR pose. Therefore, improving the accuracy of LiDAR poses derived from matching the geometric features of laser point clouds has become a challenge for those skilled in the art. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a laser point cloud parameterized semantic feature matching method, apparatus, device, and storage medium to solve or alleviate the above-mentioned problems.

[0005] According to a first aspect of an embodiment of the present invention, a laser point cloud parametric semantic feature matching method is provided, comprising: obtaining a semantic point cloud for parametric processing from a laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; performing parametric processing on the semantic point cloud segments to obtain parametric semantic features of corresponding types; combining the parametric semantic features of corresponding types into a current parametric semantic feature frame; combining multiple historical parametric semantic feature frames into a parametric semantic feature subgraph; and performing line-to-line and surface-to-surface feature matching based at least on the current parametric semantic feature frame and the parametric semantic feature subgraph to obtain the posture of the laser radar that collected the laser point cloud frame.

[0006] According to a second aspect of an embodiment of the present invention, a laser point cloud parametric semantic feature matching device is provided, comprising: an acquisition module, which acquires a semantic point cloud for parametric processing from a laser point cloud frame; a segmentation module, which segments the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; a parametric processing module, which performs parametric processing on the semantic point cloud segments to obtain parametric semantic features of corresponding types; a first combination module, which combines the parametric semantic features of corresponding types into a current parametric semantic feature frame; a second combination module, which combines multiple historical parametric semantic feature frames into a parametric semantic feature subgraph; and a matching module, which performs line-to-line and surface-to-surface feature matching based on at least the current parametric semantic feature frame and the parametric semantic feature subgraph to obtain the posture of the laser radar that collected the laser point cloud frame.

[0007] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect.

[0008] According to a fourth aspect of an embodiment of the present invention, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect is implemented.

[0009] In the solution of the embodiment of the present invention, since the corresponding type of semantic point cloud segmentation includes reliable local information, the semantic point cloud segmentation is parameterized to effectively extract these local information. Therefore, line-to-line and surface-to-surface feature matching is performed based on the parameterized semantic feature frame obtained through the above information, and higher-precision feature frame matching can be performed (for example, on the basis of traditional geometric feature matching), thereby obtaining a higher-precision lidar pose. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0011] Figure 1 is a schematic flow chart of a feature matching method according to an embodiment of the present invention;

[0012] Figure 2 is a schematic flow chart of a feature matching method according to another embodiment of the present invention;

[0013] Figure 3 is a schematic diagram of a gridding process according to another embodiment of the present invention;

[0014] Figure 4A is a schematic diagram of a parameterized semantic feature frame according to another embodiment of the present invention;

[0015] Figure 4B A schematic diagram of parameterized semantic feature matching of a rod-shaped object according to another embodiment of the present invention;

[0016] Figure 5 is a schematic block diagram of a feature matching device according to another embodiment of the present invention;

[0017] Figure 6 This is a hardware structure of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0019] The specific implementation of the embodiment of the present invention is further described below with reference to the accompanying drawings of the embodiment of the present invention. Figure 1 This is a schematic flow chart of a feature matching method according to an embodiment of the present invention. Positioning and mapping are crucial for autonomous driving or other robotic systems. Figure 1 The laser point cloud parameterized semantic feature matching method performs positioning and mapping based on laser point cloud frames collected by a laser radar (LiDAR) and can be executed by any appropriate electronic device with data processing capabilities, including but not limited to servers, mobile terminals (such as mobile phones, PADs, etc.), and PCs. LiDAR (LiDAR) has the characteristics of high measurement accuracy and is not affected by ambient light. LiDAR-based positioning and mapping algorithms are the mainstream solution in the field of autonomous driving. Laser odometry (LO) is a fundamental problem in positioning and mapping. The higher the accuracy of LO, the more beneficial it is for positioning and mapping. The embodiments of the present invention can be based on LO scenarios. Laser SLAM is a SLAM algorithm that utilizes laser point clouds. Lidar-only Odometry (LO) addresses the front-end issues of laser SLAM. That is, the higher the accuracy of LO, the smaller the trajectory drift problem, the more beneficial it is for back-end loop optimization, and improves the overall accuracy and global consistency of SLAM. Figure 1 Feature matching methods include:

[0020] 110: Obtain semantic point cloud for parameterization processing from the laser point cloud frame.

[0021] It should be understood that a motion-compensated laser point cloud can be input. For example, a semantic point cloud with semantic information can be generated by running a point cloud semantic segmentation algorithm (e.g., RangeNet++, PointNet++, KPConv, etc.) to serve as the semantic point cloud for parameterization. Point cloud semantics in this context refers to the semantic labeling of each point cloud using a semantic segmentation algorithm, such as "ground," "building," "pole," or "traffic sign."

[0022] 120: Segment the semantic point cloud according to the type of the semantic point cloud to obtain the corresponding type of semantic point cloud segmentation.

[0023] It should be understood that semantic point cloud segments contain more local information. Segmentation can be performed through clustering, plane segmentation, gridding, fitting, and other methods to obtain semantic point cloud segments. Fitting in this article means expressing the point cloud of a specified area as a mathematical equation that can describe the shape of the point cloud in that area. In addition, the types of semantic point clouds include but are not limited to ground labels, pole labels, traffic sign labels, building labels, other road signs or ground markers, etc. Preferably, the types of semantic point clouds include ground labels, pole labels, traffic sign labels, and building labels.

[0024] 130: Segment the semantic point cloud and perform parameterization processing to obtain the corresponding type of parameterized semantic features.

[0025] It should be understood that the object of parameterized processing may be a parameter indicating local information of semantic point cloud segmentation. The parameters in the text may include at least one of vectorized parameters, confidence, bounding box center point and semantic label. Preferably, the parameters in the parameterized processing include vectorized parameters, confidence, bounding box center point and semantic label. The vectorized semantic feature Vectorized Semantic Feature (vectorized semantic feature, an example of a parameterized semantic feature) as a parameterized semantic feature is a lightweight point cloud feature of an embodiment of the present invention, which extracts vectorized features from the semantic point cloud and constructs at least one type of parameterized semantic feature. For example, ground vectorized semantic features, building vectorized semantic features, pole-shaped object vectorized semantic features, and traffic sign vectorized semantic features are constructed.

[0026] 140: Combining corresponding types of parameterized semantic features into a current parameterized semantic feature frame.

[0027] It should be understood that corresponding types of parameterized semantic features can be combined based on the laser radar coordinate system. It should also be understood that at least two types of parameterized semantic features can be combined, and corresponding types of parameterized semantic features can include coordinate information based on the laser radar coordinate system.

[0028] 150: Combine multiple historical parameterized semantic feature frames into parameterized semantic feature subgraphs.

[0029] It should be understood that after extracting four parameterized semantic features (e.g., vectorized semantic features) from the current frame point cloud, a current parameterized semantic feature frame can be formed. Multiple historical parameterized semantic feature frames are converted to a global coordinate system (world coordinate system) through the calculated positions to form a parameterized semantic feature subgraph. Parameterization such as vectorization in this article refers to fitting a mathematical equation to a specified point cloud and recording certain parameters (confidence, semantic labels, bounding box center points, etc.) to represent some characteristics of the point cloud.

[0030] In one example, the number of historical parameterized semantic feature frames can be fixed, i.e., the fixed number of historical parameterized semantic feature frames used when performing feature matching at different times. In another example, a parameterized semantic feature subgraph can also be generated based on all historical parameterized semantic feature frames before a specific time point.

[0031] 160: Perform line-to-line and surface-to-surface feature matching based on at least the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the position and pose of the laser radar that collects the laser point cloud frame.

[0032] It should be understood that the pose in this article refers to the position (x, y, z) and attitude angle of the lidar or the vehicle equipped with the lidar, for example, it can include roll (roll angle), pitch (pitch angle) and yaw (yaw angle). The trajectory in this article refers to a set of vehicle poses. The real-time positioning and mapping algorithm in this article calculates the trajectory of the vehicle given an image or laser point cloud. It generally includes a front-end mileage calculation method and a back-end loop detection algorithm. The front-end is responsible for calculating the initial trajectory, and there is a drift problem caused by cumulative errors. The back-end is responsible for eliminating the cumulative error problem of the front-end.

[0033] It should also be understood that point cloud registration / matching in this context involves calculating the transformation matrix of two point clouds with partially or fully overlapping areas, but in different poses, under the same coordinate system. This matrix can be used to accurately align the two point clouds. For example, point cloud registration can be performed using iterative closest point (IP) registration. ICP registration refers to a point cloud registration algorithm that establishes relationships between points using a k-nearest neighbor search (KNN).

[0034] It should also be understood that line-to-line matching can be performed for rod-shaped objects, and face-to-face matching can be performed for at least one of the ground, traffic signs, and buildings. For example, line-to-line matching is performed on the parametric semantic features of rod-shaped objects, and the matching process for the parametric semantic features of the ground, traffic signs, or buildings is similar to that for the parametric semantic features described above, except that face-to-face matching constraints are constructed. Then, the four types of parametric semantic feature matching constraints are tightly coupled with the frame-to-subgraph constraints used in geometric feature matching to jointly estimate the graph matching pose.

[0035] It should also be understood that in addition to line-to-line and surface-to-surface feature matching based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph, geometric feature matching can also be performed based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph.

[0036] It should also be understood that a first initial pose can be estimated based on parametric feature matching and geometric feature matching between the current parameterized semantic feature frame and the parameterized semantic feature subgraph. A second initial pose can be determined based on inter-frame matching between the current parameterized semantic feature frame and the previous parameterized semantic feature frame. The pose of the laser radar that captured the laser point cloud frame can be determined based on the first initial pose and the second initial pose.

[0037] In the solution of the embodiment of the present invention, since the corresponding type of semantic point cloud segmentation includes reliable local information, the semantic point cloud segmentation is parameterized to effectively extract these local information. Therefore, line-to-line and surface-to-surface feature matching is performed based on the parameterized semantic feature frame obtained through the above information, and higher-precision feature frame matching can be performed (for example, on the basis of traditional geometric feature matching), thereby obtaining a higher-precision lidar pose.

[0038] SuMa++ also uses a laser point cloud to calculate vehicle pose and combines the semantic information of the point cloud to improve positioning accuracy. However, SuMa++ only has limited constraints for point-to-surface matching based on surfels. The solution in this embodiment not only incorporates traditional point-to-surface and point-to-line constraints, but also line-to-line and surface-to-surface constraints based on parameterized semantic features, thereby improving the accuracy of the LiDAR pose obtained by matching the geometric features of the laser point cloud.

[0039] Furthermore, when performing point cloud registration from a frame to a subimage, LOAM can encounter "ghosting" of the subimage due to previous positioning errors. KNN nearest neighbor search-based registration can cause the subimage to match "towards the point cloud surface," affecting registration accuracy and increasing LO drift. However, the algorithm in this embodiment of the present invention incorporates parameterized semantic feature matching, which can alleviate or even eliminate this problem.

[0040] In addition, IMLS-SLAM can solve the problem of "point cloud surface matching" by constructing an implicit moving least squares surface. However, this method requires calculating the normal vector of the point cloud and maintaining a large subgraph, resulting in low data processing efficiency and the inability to achieve real-time performance. The solution of the embodiment of the present invention can construct lightweight vectorized semantic features and parameterized semantic features based on point cloud semantic information, and solve the above-mentioned "surface matching" problem by matching parameterized semantic features. Compared with the above solution, it improves data processing efficiency and, because it uses the dual-threaded structure of LOAM, it can achieve real-time performance while ensuring good LO accuracy.

[0041] In another implementation of the present invention, obtaining a semantic point cloud for parameterized processing from a laser point cloud frame includes: removing a semantic point cloud with dynamic semantic information from the laser point cloud frame to obtain a semantic point cloud for parameterized processing.

[0042] Since the semantic point cloud with dynamic semantic information is removed from the semantic point cloud used for parameterized processing, it is more conducive to improving the robustness of matching, thereby improving the effect of parameterized feature matching.

[0043] As an example, the so-called semantic point cloud with dynamic semantic information includes but is not limited to obstacle classes, such as vehicles, pedestrians, bicycles, motorcycles, etc. The semantic point cloud with dynamic semantic information is removed from the semantic point cloud used for parameterization processing, that is, corner points, surface feature points, etc. are not extracted from it.

[0044] As an example, if the semantic point cloud used for parameterization processing is the ground, lawn, etc., no corner feature points are extracted.

[0045] In another implementation of the present invention, the semantic point cloud is segmented according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types, including: network segmentation of the semantic point cloud according to ground semantic labels to obtain multiple groups of point clouds corresponding to multiple grids respectively, wherein the distance between each group of point clouds and the center of the semantic point cloud has a positive correlation with the resolution of the grid corresponding to the group of point clouds, wherein the semantic point cloud segments are parameterized to obtain parameterized semantic features of corresponding types, including: determining parameterized semantic features based on at least one of the plane fitting parameters, confidence, bounding box center point position and ground semantic labels of each of the multiple groups of point clouds.

[0046] Because the distance between each point cloud group and the center of the semantic point cloud is positively correlated with the resolution of the grid corresponding to that point cloud group, the number of point clouds corresponding to each grid can be effectively uniform. Therefore, using multiple grids to perform network segmentation of the semantic point cloud can improve the robustness of the ground parameterization processing and thus improve the accuracy of parametric feature matching. In addition, each of the plane fitting parameters, confidence level, bounding box center point location, and ground semantic label of each of the multiple point clouds reliably reflects the local information of each of the multiple point clouds using a relatively small amount of data. Therefore, determining the parametric semantic feature based on at least one of these factors improves the efficiency of the parametric processing.

[0047] As an example, the positions of multiple grids in a semantic point cloud can be determined based on ground semantic labels, where the multiple grids include a first grid and a second grid. The distance between each point cloud group and the center of the semantic point cloud is positively correlated with the resolution of the grid corresponding to the point cloud group. For example, the distance indicated by the position of the first grid can be greater than the distance indicated by the position of the second grid. The sizes of the multiple grids can be determined based on their respective positions in the semantic point cloud, such that the size of the first grid is greater than the size of the second grid. The semantic point cloud is then network-segmented based on the positions and sizes of the multiple grids.

[0048] Specifically, Figure 3 This is a schematic diagram of the gridding processing of another embodiment of the present invention. As shown in the figure, in a single-frame laser point cloud, it is dense near and sparse far away. The distance between the center of each group of point clouds and the semantic point cloud has a positive correlation with the resolution of the grid corresponding to the group of point clouds. For example, in the laser radar coordinate system, the ground point cloud is divided into a series of grids from near to far according to the resolution from small to large based on the semantic label. The plane fitting parameters, confidence, bounding box center point position and ground semantic label of each group of point clouds are determined as parameterized semantic features. For example, assuming that the ground point cloud in each grid is distributed in a plane, the ground point cloud in each grid is plane fitted to obtain a vectorized equation, that is, the vectorized parameter of the ground semantic feature. The confidence of the ground semantic feature and the bounding box center point can be calculated based on the plane fitting parameters.

[0049] In another implementation of the present invention, the semantic point cloud is segmented according to the type of the semantic point cloud to obtain corresponding types of semantic point cloud segments, including: clustering the semantic point cloud according to the semantic labels of the rod-shaped objects to obtain multiple clusters, wherein the semantic point cloud segments are parameterized to obtain corresponding types of parameterized semantic features, including: determining the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and at least one of the semantic labels of the rod-shaped objects of each of the multiple clusters.

[0050] Because multiple clusters can effectively reflect the point clouds of different rod-shaped objects, the robustness of the rod-shaped object parameterization process is improved, thereby enhancing the accuracy of parametric feature matching. Furthermore, each of the plane fitting parameters, confidence level, bounding box center point location, and rod-shaped object semantic label for each cluster reliably reflects the local information of each cluster using a relatively small amount of data. Therefore, using at least one of these parameters to determine the parametric semantic feature improves the efficiency of the parametric processing.

[0051] Specifically, the semantic point cloud is clustered based on the rod semantic labels to obtain multiple clusters. For example, rod-shaped point clouds can be filtered out based on the semantic labels and then clustered. A linear fit can be performed on each cluster to obtain linear fit parameters. The bounding box center can be calculated based on the linear fit parameters.

[0052] In another implementation of the present invention, the semantic point cloud is segmented according to the type of the semantic point cloud to obtain corresponding types of semantic point cloud segments, including: clustering the semantic point cloud according to the semantic labels of the traffic signs to obtain multiple clusters; based on the multiple clusters, multiple groups of main plane points are determined respectively as corresponding types of semantic point cloud segments, wherein the semantic point cloud segments are parameterized to obtain corresponding types of parameterized semantic features, including: determining the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and traffic sign semantic labels of the multiple groups of main plane points.

[0053] Because multiple clusters can effectively reflect the point clouds of different traffic signs, multiple sets of principal plane points are determined based on these clusters. This improves the point cloud of each traffic sign's principal plane, thereby enhancing the robustness of the parameterization process and, in turn, the accuracy of the parameterized feature matching. Furthermore, each of the plane fitting parameters, confidence levels, bounding box center point locations, and traffic sign semantic labels for each of the multiple sets of principal plane points reliably reflects their local information using a relatively small amount of data. Therefore, using at least one of these factors to determine the parameterized semantic features improves the efficiency of the parameterization process.

[0054] In another implementation of the present invention, multiple groups of principal plane points are determined based on multiple clusters, including: performing multiple iterative processing based on each cluster to determine a group of principal plane points of the cluster to obtain multiple groups of principal plane points, wherein, in each iterative processing, plane fitting is performed based on a current group of points greater than a target point number threshold, and built-in points are removed from the current group of points until a group of principal plane points less than the target point number threshold is obtained after multiple iterative processing.

[0055] Since built-in points are removed by plane fitting in each iterative process, the set of principal plane points obtained after the iteration can better reflect the characteristics of the principal plane, thereby improving the parsing efficiency of the principal plane data.

[0056] Specifically, the semantic point cloud is clustered based on the semantic labels of traffic signs to obtain multiple clusters. For example, traffic sign point clouds can be filtered and clustered based on the semantic labels. A multi-plane extraction algorithm can be used to determine multiple groups of principal plane points based on the multiple clusters. For example, the plane fitting parameters for each of the multiple groups of principal plane points can be obtained using a fitted vectorized equation.

[0057] In another implementation of the present invention, the semantic point cloud is segmented according to the type of the semantic point cloud to obtain corresponding types of semantic point cloud segments, including: performing plane segmentation processing on the semantic point cloud according to the semantic labels of the building to obtain multiple groups of point clouds; based on the multiple groups of point clouds, multiple groups of main plane points are respectively determined as corresponding types of semantic point cloud segments, wherein the semantic point cloud segments are parameterized to obtain corresponding types of parameterized semantic features, including: determining the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and traffic sign semantic labels of each of the multiple groups of main plane points.

[0058] Because multiple point clouds can effectively reflect the point clouds of different buildings, multiple sets of principal plane points are determined based on these points. This improves the point cloud for each building's principal plane, thereby enhancing the robustness of the parameterization process and, in turn, the accuracy of the parameterized feature matching. Furthermore, each of the plane fitting parameters, confidence levels, bounding box center point locations, and building semantic labels for each of the multiple sets of principal plane points reliably reflects their local information using a relatively small amount of data. Therefore, using at least one of these factors to determine the parameterized semantic features improves the efficiency of the parameterization process.

[0059] In another implementation of the present invention, multiple groups of principal plane points are determined based on multiple groups of point clouds, including: performing multiple iterative processing based on each group of point cloud segments to determine a group of principal plane points of the group of point clouds to obtain multiple groups of principal plane points, wherein in each iterative processing, plane fitting is performed based on a current group of points greater than a target point number threshold, and built-in points are removed from the current group of points until a group of principal plane points less than the target point number threshold is obtained after multiple iterative processing.

[0060] Since built-in points are removed by plane fitting in each iterative process, the set of principal plane points obtained after the iteration can better reflect the characteristics of the principal plane, thereby improving the parsing efficiency of the principal plane data.

[0061] Specifically, based on the semantic labels of buildings, the semantic point cloud is plane-segmented to generate multiple point cloud groups. Since building point clouds are often contiguous and large in area, plane segmentation can be used to divide them into several point cloud segments. A multi-plane extraction algorithm can be employed to obtain parameterized semantic features of the building. For example, a multi-plane extraction algorithm can be employed to determine multiple groups of principal plane points based on multiple clusters. For example, the plane fitting parameters for each of the multiple groups of principal plane points can be obtained by fitting vectorized equations.

[0062] Figure 4A FIG. 4 shows a parameterized semantic feature frame according to another embodiment of the present invention. Figure 4A As shown in the figure, multiple historical parametric semantic feature frames are combined into a parametric semantic feature subgraph, resulting in four types of parametric semantic feature rendering effects. The numbers in the figure represent the confidence level of each parametric semantic feature. Specifically, after extracting four parametric semantic features from the current frame point cloud, the current parametric semantic feature frame is formed. Multiple historical parametric semantic feature frames can be transformed into the entire coordinate system using the calculated pose to form a historical parametric semantic feature subgraph.

[0063] In addition, line-to-line and surface-to-surface feature matching is performed at least based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the lidar pose. For example, line-to-line matching (e.g., for rod-shaped object parameterized semantic features) and surface-to-surface matching (for ground parameterized semantic features, traffic sign parameterized semantic features, or building parameterized semantic features) are performed between the parameterized semantic feature frame and the parameterized semantic feature subgraph based on at least one (e.g., the four) parameterized semantic features described above.

[0064] Specifically, Figure 4BAnother embodiment of the present invention illustrates rod-shaped parametric semantic feature matching. As shown in the figure, when matching based on rod-shaped parametric semantic features, each rod-shaped parametric semantic feature in the current parametric semantic feature frame can be labeled as f0. A parametric semantic feature subgraph is searched for rod-shaped parametric semantic features whose center is less than a predetermined threshold from f0, and these features are labeled F1, F2, F3, ..., FN, respectively.

[0065]

[0066] The weighted average value F_avg of F1 to FN can be calculated using the following formula.

[0067] In another implementation of the present invention, the semantic point cloud segmentation is parameterized to obtain parameterized semantic features of corresponding types, including: determining the parameterized semantic features based on the linear fitting parameters, confidence, bounding box center point position and at least one of the type of the semantic point cloud segmentation, wherein the linear fitting parameters of the semantic point cloud segmentation are obtained by performing linear fitting on the semantic point cloud segmentation, the confidence of the semantic point cloud segmentation is determined based on the ratio between the number of linear fitting built-in points of the semantic point cloud segmentation and the total number of points of the semantic point cloud, and the bounding box center point position of the semantic point cloud segmentation is determined based on the coordinates of the bounding box center point of the semantic point cloud segmentation in the laser sensor coordinate system.

[0068] In another implementation of the present invention, line-to-line and surface-to-surface feature matching is performed based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar that collects the laser point cloud frame, including: searching the position of the center point of the bounding box through a proximity search algorithm to determine the line-to-line feature matching points and surface-to-surface feature matching points between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein, when searching based on the proximity search algorithm, the weight of the residual function is determined based on the number of times the point-to-point feature matching points and surface-to-surface feature matching points are searched, so that the weight of the residual function is positively correlated with the number of times; transformation processing is performed based on the point-to-point feature matching points and surface-to-surface feature matching points to determine the pose of the laser radar that collects the laser point cloud frame.

[0069] Since the number of point-to-point feature matching points and face-to-face feature matching points searched reflects the accuracy of parameterized feature matching, the weight of the residual function is set to be positively correlated with the number, thereby improving the accuracy of parameterized feature matching.

[0070] Specifically, the center point position of the bounding box is searched through a neighboring search algorithm to determine the line-to-line feature matching points and face-to-face feature matching points between the current parameterized semantic feature frame and the parameterized semantic feature subgraph. By designing weights for the matching residual function, the accuracy of establishing associations in the matching is improved. The weight of the residual function is positively correlated with the number of times. For example, during a KNN search, the more times the semantic label is consistent same_label_cnt (for example, point-to-point feature matching points or face-to-face feature matching points), the greater the weight W_residual of the residual function. In a specific example, W_residual = 1.0 / (1.0+exp(-2*same_label_cnt+2)).

[0071] In another implementation of the present invention, line-to-line and surface-to-surface feature matching is performed at least based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar that collects the laser point cloud frame, including: determining the parameterized semantic feature matching constraints and geometric feature matching constraints between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraints indicate the matching relationship of the line-to-line and surface-to-surface feature matching; based on the parameterized semantic feature matching constraints and the geometric feature matching constraints, determining the pose of the laser radar that collects the laser point cloud frame.

[0072] Alternatively, a dual-threaded design can be employed, leveraging high-frequency frame-to-frame matching to estimate the coarser inter-frame matching pose of a LiDAR or LiDAR-equipped vehicle. Low-frequency frame-to-subgraph matching can be used to calculate the more precise graph matching pose of the LiDAR or LiDAR-equipped vehicle. Frame-to-frame matching provides initial matching values ​​for frame-to-subgraph matching, which can be determined by the vehicle's uniform velocity model. Inter-frame matching poses and graph matching poses can be integrated to achieve real-time laser odometry.

[0073] In addition, both frame-to-frame matching and frame-to-subimage matching can utilize previously extracted corner points and surface geometric features to construct point-to-line constraints and point-to-surface constraints.

[0074] like Figure 2 , which shows a schematic flow chart of a feature matching method according to another embodiment of the present invention. Specifically, line-to-line matching can be performed between f0 and F_avg, and tightly coupled with the frame-to-subimage matching in geometric feature matching, thereby resolving the "surface-toward" matching problem in the latter when ghosting occurs in the subimage.

[0075] In another implementation of the present invention, the method further includes: based on the posture of the lidar, adding the current parameterized semantic feature frame to the historical parameterized semantic feature subgraph through coordinate transformation, and removing the first parameterized semantic feature frame in the historical parameterized semantic feature subgraph to update the historical parameterized semantic feature subgraph.

[0076] Since the current parameterized semantic feature frame is added to the historical parameterized semantic feature subgraph through coordinate transformation, and the first parameterized semantic feature frame in the historical parameterized semantic feature subgraph is removed, the dynamic update of the historical parameterized semantic feature subgraph is achieved while ensuring the number of feature frames in the historical parameterized semantic feature subgraph, thereby improving the accuracy of parameterized feature matching.

[0077] In addition, graph matching poses and inter-frame matching poses can be integrated to implement real-time laser odometry (parameterized semantic features LO). Based on the LiDAR pose, the current parameterized semantic feature frame is added to the historical parameterized semantic feature subgraph through coordinate transformation. For example, the parameterized semantic feature subgraph is updated using the integrated pose, that is, the parameterized semantic feature frame of the current frame is transformed to the global coordinate system according to the integrated pose and then added to the parameterized semantic feature subgraph.

[0078] In addition, the first parameterized semantic feature frame in the historical parameterized semantic feature subgraph is removed to update the historical parameterized semantic feature subgraph. For example, the earliest parameterized semantic feature frame is correspondingly removed from the parameterized semantic feature subgraph. Figure 2 shown.

[0079] Figure 5 It is a schematic block diagram of a feature matching device according to another embodiment of the present invention. Figure 5 The laser point cloud parameterized semantic feature matching device can be applied to any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. The device includes:

[0080] An acquisition module 510 acquires a semantic point cloud for parameterization processing from the laser point cloud frame;

[0081] A segmentation module 520 segments the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types;

[0082] The parameterization processing module 530 performs parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types.

[0083] The first combining module 540 combines corresponding types of parameterized semantic features into a current parameterized semantic feature frame.

[0084] The second combining module 550 combines multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph.

[0085] The matching module 560 performs line-to-line and surface-to-surface feature matching based on at least the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the position and posture of the laser radar that collects the laser point cloud frame.

[0086] It should be understood that the pose in this article refers to the position (x, y, z) and attitude angles (roll, pitch, yaw) of the lidar or the vehicle equipped with it. The trajectory in this article refers to a set of vehicle poses. The real-time positioning and mapping algorithms in this article calculate the vehicle trajectory given an image or laser point cloud. They generally include a front-end mileage calculation method and a back-end loop detection algorithm. The front-end is responsible for calculating the initial trajectory, which may cause drift due to accumulated errors, while the back-end is responsible for eliminating the accumulated errors of the front-end.

[0087] It should also be understood that the term "laser SLAM" in this article specifically refers to SLAM algorithms that utilize only laser point clouds. Furthermore, the term "Lidar-only Odometry" (LO) in this article refers to the aforementioned SLAM front-end problem. The higher the LO accuracy, the less likely it is to cause trajectory drift, which in turn facilitates back-end loop closure optimization, improving overall SLAM accuracy and global consistency. The embodiments of this invention are primarily based on LO scenarios.

[0088] It should also be understood that the point cloud registration / matching in this article is to use the transformation matrix of two point clouds with partially or completely overlapping areas under the same coordinate system to achieve accurate registration of the two point clouds using this matrix, which serves as the basis of the LO algorithm.

[0089] It should also be understood that ICP registration in this article refers to a point cloud registration algorithm that establishes relationships between points using a k-nearest neighbor search (KNN). Point cloud semantics in this article refers to assigning semantic labels to each point cloud using a semantic segmentation algorithm, such as "ground," "building," "pole," or "traffic sign." As shown in the rendering below, blue represents the ground, magenta represents vehicles, gray represents buildings, green represents vegetation, and light green represents lawn.

[0090] It should also be understood that the fitting in this article refers to representing the point cloud of a specified area as a mathematical equation, which can describe the shape of the point cloud in this area. The vectorization or parameterization in this article refers to representing the mathematical equation by fitting the specified point cloud, and recording certain parameters (confidence, semantic labels, center points of bounding boxes, etc.) to represent some characteristics of the point cloud. The vectorized semantic feature (parameterized semantic feature) in this article is a lightweight point cloud feature of an embodiment of the present invention. The semantic point cloud is vectorized and feature extracted to construct four types of parameterized semantic features, namely, ground parameterized semantic features, building parameterized semantic features, rod-shaped parameterized semantic features, and traffic sign parameterized semantic features.

[0091] The submap ghosting problem in this paper is the appearance of "ghosting" in the accumulated submaps due to previous positioning errors.

[0092] In the solution of the embodiment of the present invention, since the corresponding type of semantic point cloud segmentation includes reliable local information, the semantic point cloud segmentation is parameterized to effectively extract these local information. Therefore, line-to-line and surface-to-surface feature matching is performed based on the parameterized semantic feature frame obtained through the above information, and higher-precision feature frame matching can be performed (for example, on the basis of traditional geometric feature matching), thereby obtaining a higher-precision lidar pose.

[0093] In another implementation of the present invention, the acquisition module is specifically used to remove the semantic point cloud with dynamic semantic information from the laser point cloud frame to obtain the semantic point cloud for parameterized processing.

[0094] In another implementation of the present invention, the segmentation module is specifically used to: perform network segmentation on the semantic point cloud according to the ground semantic label to obtain multiple groups of point clouds corresponding to multiple grids respectively, wherein the distance between each group of point clouds and the center of the semantic point cloud has a positive correlation with the resolution of the grid corresponding to the group of point clouds, wherein the parameterization processing module is specifically used to: determine the parameterized semantic features based on the plane fitting parameters, confidence, bounding box center point position and at least one of the ground semantic labels of each of the multiple groups of point clouds.

[0095] As an example, the segmentation module is specifically used to: determine the positions of multiple grids in the semantic point cloud based on ground semantic labels, the multiple grids include a first grid and a second grid, and the distance indicated by the position of the first grid is greater than the distance indicated by the position of the second grid; determine the sizes of the multiple grids based on the positions of the multiple grids in the semantic point cloud, so that the size of the first grid is greater than the size of the second grid; and perform network segmentation on the semantic point cloud based on the positions and sizes of the multiple grids.

[0096] In another implementation of the present invention, the segmentation module is specifically used to: cluster the semantic point cloud according to the semantic label of the rod-shaped object to obtain multiple clusters, and the parameterization processing module is specifically used to: determine the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and at least one of the semantic labels of the rod-shaped object of each of the multiple clusters.

[0097] In another implementation of the present invention, the segmentation module is specifically used to: cluster the semantic point cloud according to the semantic label of the traffic sign to obtain multiple clusters; based on the multiple clusters, determine multiple groups of main plane points as corresponding types of semantic point cloud segments, wherein the parameterization processing module is specifically used to: determine the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and the semantic labels of the traffic signs of the multiple groups of main plane points.

[0098] In another implementation of the present invention, multiple iterative processing is performed based on each cluster to determine a set of main plane points of the cluster to obtain multiple sets of main plane points, wherein, in each iterative processing, plane fitting is performed based on a current set of points greater than a target point number threshold, and built-in points are removed from the current set of points until the set of main plane points less than the target point number threshold is obtained after the multiple iterative processing.

[0099] In another implementation of the present invention, the segmentation module is specifically used to: perform plane segmentation processing on the semantic point cloud according to the building semantic label to obtain multiple groups of point clouds; based on the multiple groups of point clouds, determine multiple groups of main plane points respectively as corresponding types of semantic point cloud segments, wherein the parameterization processing module is specifically used to: determine the parameterized semantic features based on the linear fitting parameters, confidence levels, bounding box center point positions and the semantic labels of the traffic signs of the multiple groups of main plane points.

[0100] In another implementation of the present invention, the parametric processing module is specifically used to: perform multiple iterative processing based on each group of point cloud segments to determine a group of main plane points of the group of point clouds to obtain multiple groups of main plane points, wherein in each iterative processing, plane fitting is performed based on a current group of points greater than a target point number threshold, and built-in points are removed from the current group of points until the group of main plane points less than the target point number threshold is obtained after the multiple iterative processing.

[0101] In another implementation of the present invention, the parameterized processing module is specifically used to determine the parameterized semantic features based on at least one of the linear fitting parameters, confidence, bounding box center point position and type of the semantic point cloud segmentation, wherein the linear fitting parameters of the semantic point cloud segmentation are obtained by performing linear fitting on the semantic point cloud segmentation, the confidence of the semantic point cloud segmentation is determined based on the ratio between the number of linear fitting built-in points of the semantic point cloud segmentation and the total number of points of the semantic point cloud, and the bounding box center point position of the semantic point cloud segment is determined based on the coordinates of the bounding box center point of the semantic point cloud segmentation in the laser sensor coordinate system.

[0102] In another implementation of the present invention, the matching module is specifically used to: search the position of the center point of the bounding box through a proximity search algorithm to determine the line-to-line feature matching points and the face-to-face feature matching points between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein, when searching based on the proximity search algorithm, the weight of the residual function is determined based on the number of times the point-to-point feature matching points and the face-to-face feature matching points are searched, so that the weight of the residual function is positively correlated with the number; based on the transformation processing of the point-to-point feature matching points and the face-to-face feature matching points, the posture of the laser radar that collects the laser point cloud frame is determined.

[0103] In another implementation of the present invention, the matching module is specifically used to: determine the parameterized semantic feature matching constraints and the geometric feature matching constraints between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraints indicate the matching relationship of the line-to-line and surface-to-surface feature matching; based on the parameterized semantic feature matching constraints and the geometric feature matching constraints, determine the posture of the laser radar that collects the laser point cloud frame.

[0104] In another implementation of the present invention, the device also includes: an updating module, which adds the current parameterized semantic feature frame to the historical parameterized semantic feature subgraph through coordinate transformation based on the posture of the lidar, and removes the first parameterized semantic feature frame in the historical parameterized semantic feature subgraph to update the historical parameterized semantic feature subgraph.

[0105] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0106] Figure 6 The hardware structure of the electronic device of another embodiment of the present invention is as follows; Figure 6As shown, the hardware structure of the electronic device may include: a processor 601, a communication interface 602, a storage medium 603 and a communication bus 604;

[0107] The processor 601, the communication interface 602, and the storage medium 603 communicate with each other via the communication bus 604;

[0108] Optionally, the communication interface 602 may be an interface of a communication module;

[0109] Among them, the processor 601 can be specifically configured to: obtain a semantic point cloud for parameterized processing from a laser point cloud frame; segment the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; perform parameterized processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combine the parameterized semantic features of corresponding types into a current parameterized semantic feature frame; combine multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph; perform line-to-line and surface-to-surface feature matching based on at least the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the posture of the laser radar that collected the laser point cloud frame.

[0110] The aforementioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor.

[0111] The above-mentioned storage medium can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc.

[0112] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the storage medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program configured for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0113] Computer program code configured to perform the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or portion of code, which contains one or more executable instructions configured to implement the specified logical function. The above-mentioned specific embodiments have specific sequential relationships, but these sequential relationships are merely exemplary. During the specific implementation, these steps may be fewer, more, or the execution order may be adjusted. In other words, in some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as combinations of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0115] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and the names of these modules do not necessarily limit the modules themselves.

[0116] As another aspect, the present invention further provides a storage medium storing a computer program, which implements the method described in the above embodiment when executed by a processor.

[0117] As another aspect, the present invention further provides a storage medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above storage medium carries one or more programs, and when the above one or more programs are executed by the device, the device: obtains a semantic point cloud for parameterized processing from a laser point cloud frame; segments the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; performs parameterized processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combines the parameterized semantic features of corresponding types into a current parameterized semantic feature frame; combines multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph; performs line-to-line and surface-to-surface feature matching on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the position and posture of the laser radar that collected the laser point cloud frame.

[0118] The terms "first," "second," "the first," or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these terms do not limit the corresponding components. The above terms are configured solely for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, even though both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0119] When one element (for example, a first element) is referred to as being “(operably or communicably) coupled” or “(operably or communicably) coupled to” or “connected to” another element (for example, a second element), it should be understood that the one element is directly connected to the other element or that the one element is indirectly connected to the other element via yet another element (for example, a third element). Conversely, it should be understood that when an element (for example, a first element) is referred to as being “directly connected” or “directly coupled” to another element (the second element), there is no element (for example, a third element) interposed therebetween.

[0120] The above description is merely an illustration of the preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A method for matching parameterized semantic features of laser point clouds, comprising: From the laser point cloud frame, obtain the semantic point cloud for parameterization processing; Segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; The semantic point cloud is segmented and parameterized to obtain corresponding types of parameterized semantic features; Combining the corresponding types of parameterized semantic features into a current parameterized semantic feature frame; Combining multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph; Line-to-line and surface-to-surface feature matching is performed at least based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the position and pose of the laser radar that collects the laser point cloud frame.

2. The method according to claim 1, wherein The step of obtaining a semantic point cloud for parameterized processing from the laser point cloud frame includes: The semantic point cloud having dynamic semantic information is removed from the laser point cloud frame to obtain the semantic point cloud for parameterized processing.

3. The method according to claim 1, wherein The segmenting of the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types includes: According to the ground semantic labels, the semantic point cloud is network-segmented to obtain multiple groups of point clouds corresponding to multiple grids, wherein the distance between each group of point clouds and the center of the semantic point cloud has a positive correlation with the resolution of the grid corresponding to the group of point clouds. The segmenting of the semantic point cloud and parameterizing it to obtain parameterized semantic features of corresponding types include: The parameterized semantic feature is determined based on at least one of the plane fitting parameters, confidence levels, bounding box center points, and the ground semantic labels of each of the multiple groups of point clouds.

4. The method according to claim 1, wherein The segmenting of the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types includes: According to the semantic labels of the rod-shaped objects, the semantic point cloud is clustered to obtain multiple clusters. The segmenting of the semantic point cloud and parameterizing it to obtain parameterized semantic features of corresponding types include: The parameterized semantic feature is determined based on at least one of the linear fitting parameters, confidences, center points of bounding boxes of the respective clusters, and the semantic labels of the rods.

5. The method according to claim 1, wherein The segmenting of the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types includes: Clustering the semantic point cloud according to the semantic labels of the traffic signs to obtain multiple clusters; Based on the multiple clusters, multiple groups of main plane points are determined as corresponding types of semantic point cloud segments, The segmenting of the semantic point cloud and parameterizing it to obtain parameterized semantic features of corresponding types include: The parameterized semantic features are determined based on the linear fitting parameters, confidence levels, bounding box center points of the plurality of groups of principal plane points and the semantic labels of the traffic signs.

6. The method according to claim 5, wherein: The determining of a plurality of groups of principal plane points based on the plurality of clusters comprises: Multiple iterative processes are performed based on each cluster to determine a set of principal plane points of the cluster to obtain multiple sets of principal plane points, wherein, in each iterative process, plane fitting is performed based on a current set of points that is greater than a target point number threshold, and built-in points are removed from the current set of points until the set of principal plane points that is less than the target point number threshold is obtained after the multiple iterative processes.

7. The method according to claim 1, wherein The segmenting of the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types includes: Performing plane segmentation processing on the semantic point cloud according to the building semantic labels to obtain multiple groups of point clouds; Based on the multiple groups of point clouds, multiple groups of main plane points are determined as corresponding types of semantic point cloud segments, The segmenting of the semantic point cloud and parameterizing it to obtain parameterized semantic features of corresponding types include: The parameterized semantic features are determined based on the linear fitting parameters, confidence levels, bounding box center point positions, and traffic sign semantic labels of the plurality of groups of principal plane points.

8. The method according to claim 7, wherein: Determining a plurality of groups of principal plane points based on the plurality of groups of point clouds includes: Multiple iterative processes are performed based on each group of point cloud segments to determine a group of principal plane points of the group of point clouds to obtain multiple groups of principal plane points, wherein in each iterative process, plane fitting is performed based on a current group of points greater than a target point number threshold, and built-in points are removed from the current group of points until the group of principal plane points less than the target point number threshold is obtained after the multiple iterative processes.

9. The method according to claim 1, wherein The segmenting of the semantic point cloud and performing parameterized processing to obtain corresponding types of parameterized semantic features include: Determining the parameterized semantic feature based on at least one of a linear fitting parameter, a confidence level, a center point position of a bounding box of the semantic point cloud segment, and a type of the semantic point cloud; Among them, the linear fitting parameters of the semantic point cloud segment are obtained by performing linear fitting on the semantic point cloud segment, the confidence of the semantic point cloud segment is determined based on the ratio between the number of linear fitting built-in points of the semantic point cloud segment and the total number of points of the semantic point cloud, and the position of the center point of the bounding box of the semantic point cloud segment is determined based on the coordinates of the center point of the bounding box of the semantic point cloud segment in the laser sensor coordinate system.

10. The method according to claim 9, wherein: The performing line-to-line and surface-to-surface feature matching based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar that collects the laser point cloud frame includes: Searching the center point position of the bounding box using a proximity search algorithm to determine line-to-line feature matching points and face-to-face feature matching points between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein, when searching based on the proximity search algorithm, a weight of a residual function is determined based on the number of times the point-to-point feature matching points and the face-to-face feature matching points are searched, such that the weight of the residual function is positively correlated with the number of times; Transformation processing is performed based on the point-to-point feature matching points and the surface-to-surface feature matching points to determine the position and posture of the laser radar that collects the laser point cloud frame.

11. The method according to claim 1, wherein The performing line-to-line and surface-to-surface feature matching based at least on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar that collects the laser point cloud frame includes: Determining a parameterized semantic feature matching constraint and a geometric feature matching constraint between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraint indicates a matching relationship between the line-to-line and surface-to-surface feature matching; Based on the parameterized semantic feature matching constraint and the geometric feature matching constraint, the position and posture of the laser radar that collects the laser point cloud frame is determined.

12. The method according to claim 1, wherein The method further comprises: Based on the position of the lidar, the current parameterized semantic feature frame is added to the historical parameterized semantic feature subgraph through coordinate transformation, and the first parameterized semantic feature frame in the historical parameterized semantic feature subgraph is removed to update the historical parameterized semantic feature subgraph.

13. A laser point cloud parameterized semantic feature matching device, comprising: The acquisition module obtains the semantic point cloud for parameterization processing from the laser point cloud frame; a segmentation module, which segments the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; A parameterized processing module, which performs parameterized processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; A first combining module combines the corresponding types of parameterized semantic features into a current parameterized semantic feature frame; The second combination module combines multiple historical parameterized semantic feature frames into a parameterized semantic feature subgraph; The matching module performs line-to-line and surface-to-surface feature matching based on at least the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the position and posture of the laser radar that collects the laser point cloud frame.

14. An electronic device, comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, where the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 12.

15. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images