A point cloud registration method, device and computer readable storage medium

Abstract

The application provides a point cloud registration method, device and computer readable storage medium. The method comprises: parsing first point cloud data and second point cloud data to obtain first semantic information and second semantic information, wherein the first semantic information comprises a plurality of semantics with coordinates, and the semantics in the first semantic information indicate the types of elements in the first point cloud data; the second semantic information comprises a plurality of semantics with coordinates, and the semantics in the second semantic information indicate the types of elements in the second point cloud data; matching the semantics in the first semantic information with the semantics in the second semantic information to obtain a matched semantic pair; and registering the first point cloud data and the second point cloud data according to the matched semantic pair. The point cloud registration method provided by the application can reduce the amount of point cloud data required for point cloud registration according to the position information of the matched semantic pair, thereby improving the accuracy of point cloud registration.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically relating to a point cloud registration method, apparatus, and computer-readable storage medium. Background Technology

[0002] Laser point cloud mapping technology plays a crucial role in fields such as 3D modeling, robot navigation, autonomous driving, and urban planning. Point cloud registration is one of the core technologies within this field. This technology is essential in both LiDAR odometry and point cloud fusion. Point cloud registration typically involves registering a source point cloud onto a target point cloud to obtain the pose transformation relationship between the two point clouds.

[0003] Current common point cloud registration methods mainly include Iterative Closest Point (ICP) and Normal Distributions Transform (NDT) methods and their variants. ICP is a commonly used point cloud registration algorithm that iteratively optimizes the matching error between two point clouds until the error is minimized. NDT transforms point cloud data into a probability density distribution and finds the optimal transformation parameters by optimizing this distribution. ICP uses a large amount of point cloud data and is heavily influenced by initial iteration values, making it prone to getting trapped in local optima. NDT's performance is significantly affected by the choice of cell size and assumptions about the probability density distribution, making it difficult to balance accuracy and efficiency. Furthermore, existing registration methods, regardless of whether they employ prior convergence thresholds, maximum number of iterations, mean squared error, gradient changes, etc., only guarantee the convergence of the algorithm itself, not the accuracy of the result.

[0004] In reality, there are many scenarios where satellite navigation and positioning are unavailable. Because of the lack of an absolute reference with sufficient accuracy (which can reach centimeter-level accuracy when satellite navigation is available), multiple point cloud maps built from different acquisition devices, routes, and time periods are prone to misalignment and cannot be directly merged. In such cases, to maintain global consistency, point cloud registration must be used for point cloud fusion. Often, in these situations, point clouds with the same name are far apart. Using common algorithms can easily lead to mismatches, resulting in significant deviations in the results.

[0005] Therefore, improving the accuracy of point cloud registration is an urgent problem to be solved. Summary of the Invention

[0006] To address the problems existing in the prior art, a point cloud registration method, apparatus, and computer-readable storage medium are proposed. Using this method, apparatus, and computer-readable storage medium, the accuracy of point cloud registration can be improved.

[0007] This application provides the following solutions.

[0008] Firstly, this application provides a point cloud registration method, including:

[0009] The first point cloud data and the second point cloud data are parsed to obtain the first semantic information and the second semantic information. The first semantic information includes multiple semantics with coordinates. The semantics in the first semantic information indicate the type of the element in the first point cloud data. The second semantic information includes multiple semantics with coordinates. The semantics in the second semantic information indicate the type of the element in the second point cloud data.

[0010] Match the semantics in the first semantic information with the semantics in the second semantic information to obtain matching semantic pairs;

[0011] The first point cloud data and the second point cloud data are registered based on the matching semantic pairs.

[0012] In some possible embodiments, the multiple semantics in the first semantic information include point semantics, line semantics, and surface semantics, and the multiple semantics in the first point cloud data include point elements, line elements, and surface elements;

[0013] The point semantics in the first semantic information indicate the type of point elements in the first point cloud data; the line semantics in the first semantic information indicate the type of line semantics in the first point cloud data; and the surface semantics in the first semantic information indicate the type of surface elements in the first point cloud data.

[0014] The second semantic information includes point semantics, line semantics, and surface semantics; the second point cloud data includes point elements, line elements, and surface elements.

[0015] The point semantics in the second semantic information indicate the type of point elements in the second point cloud data; the line semantics in the second semantic information indicate the type of line semantics in the second point cloud data; and the surface semantics in the second semantic information indicate the type of surface elements in the second point cloud data.

[0016] In some possible embodiments, the semantics in the first semantic information and the semantics in the second semantic information are matched to obtain matching semantic pairs, including:

[0017] The first semantic information is divided into multiple grids based on the semantic coordinates, and the second semantic information is also divided into multiple grids.

[0018] The first grid of the target location in the first semantic information is matched with the target location and multiple second grids around the target location in the second semantic information, and the target grid with the highest matching degree with the first grid is determined among the multiple second grids;

[0019] Match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs.

[0020] In some possible embodiments, it also includes:

[0021] Calculate the degree of misalignment between the first point cloud data and the second point cloud data;

[0022] The number of grids in multiple second grids is determined based on the degree of misalignment; the higher the degree of misalignment, the more grids are in multiple second grids.

[0023] In some possible embodiments, the degree of misalignment between the first point cloud data and the second point cloud data is calculated using the following formula. :

[0024]

[0025]

[0026] in, Indicates the type within the first grid m and the second grid n. semantic similarity, Indicates the type within the first grid m The total number of semantics, Indicates the type within the second grid n The total number of semantics, Representation type The weights.

[0027] In some possible embodiments, the plurality of second grids includes the grid of the target location in the second semantic information, and eight grids adjacent to the grid of the target location.

[0028] In some possible embodiments, the semantics in the first grid are matched with the semantics in the target grid to obtain at least two matching semantic pairs, including:

[0029] Match the geometric relationships between semantics in the target grid with the geometric relationships between semantics in the first grid to obtain at least two matching semantic pairs.

[0030] In some possible embodiments, the semantics in the first grid and the corresponding semantics in the second grid are determined to form a matching semantic pair when the semantics in the first grid and the semantics in the target grid satisfy the following conditions:

[0031] Less than °and Less than rice;

[0032] Less than °and Less than rice;

[0033] in, and All of these are pre-set constants. , representing the set of angles between the normal vectors of the face semantics and the line semantics in the first mesh { The i-th element in} The set of angles between the surface semantic normal vector and the line semantic normal vector in the target mesh { The j-th element in} differences and The corresponding semantic indicators are of the same type;

[0034] , represents the set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the first grid. The i-th element in} The set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type;

[0035] , representing the set of angles between the semantic center point of the face, the line connecting the points, and the semantic normal vector of the face in the first grid. The i-th element in} The set of angles between the semantic center point of the surface in the target mesh and the line connecting the point semantic center point and the surface semantic normal vector { The j-th element in} differences and The corresponding semantic indicators are of the same type;

[0036] , represents the set of semantic geometric distances between the semantic center points of faces and the points in the first grid. The i-th element in} The set of semantic geometric distances between the semantic center point of the surface and the point semantic geometric distance in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0037] In some possible embodiments, registration of the first point cloud data and the second point cloud data is performed based on matching semantic pairs, including:

[0038] Extract the matching data corresponding to the positions of the matching semantic pairs in the first and second point cloud data;

[0039] Based on the matching data, the first point cloud data and the second point cloud data are registered.

[0040] In some possible embodiments, extracting matching data corresponding to the positions of matching semantic pairs in the first point cloud data and the second point cloud data includes:

[0041] The coordinates in the first point cloud data and the second point cloud data are... to Data within the specified range is extracted as matching data;

[0042]

[0043]

[0044] X, Y, and Z represent the sets of coordinate points in the x, y, and z directions of the matching semantic pair in the local coordinate system, respectively. Let X be the minimum value in set X. The minimum value in set Y. This represents the minimum value in set Z. The maximum value in set X. The maximum value in set Y The maximum value in set Z. This is the preset error distance.

[0045] In some possible embodiments, registration of the first point cloud data and the second point cloud data is performed based on the matching data pair, including:

[0046] Based on the matching data, the first point cloud data and the second point cloud data are registered using an iterative nearest point algorithm.

[0047] In some possible embodiments, it also includes:

[0048] Extract validation data from the first and second point cloud data, excluding the locations of matching semantic pairs;

[0049] The first and second point cloud data after registration were verified based on the verification data.

[0050] Secondly, this application provides a point cloud registration device, comprising:

[0051] The parsing module is used to parse the first point cloud data and the second point cloud data to obtain first semantic information and second semantic information. The first semantic information includes multiple semantics with coordinates, and the semantics in the first semantic information indicate the type of elements in the first point cloud data. The second semantic information includes multiple semantics with coordinates, and the semantics in the second semantic information indicate the type of elements in the second point cloud data.

[0052] The matching module is used to match the semantics in the first semantic information and the semantics in the second semantic information to obtain matching semantic pairs;

[0053] The registration module is used to register the first point cloud data and the second point cloud data according to the matching semantic pairs.

[0054] In some possible embodiments, the multiple semantics in the first semantic information include point semantics, line semantics, and surface semantics, and the multiple semantics in the first point cloud data include point elements, line elements, and surface elements;

[0055] The point semantics in the first semantic information indicate the type of point elements in the first point cloud data; the line semantics in the first semantic information indicate the type of line semantics in the first point cloud data; and the surface semantics in the first semantic information indicate the type of surface elements in the first point cloud data.

[0056] The second semantic information includes point semantics, line semantics, and surface semantics; the second point cloud data includes point elements, line elements, and surface elements.

[0057] The point semantics in the second semantic information indicate the type of point elements in the second point cloud data; the line semantics in the second semantic information indicate the type of line semantics in the second point cloud data; and the surface semantics in the second semantic information indicate the type of surface elements in the second point cloud data.

[0058] In some possible embodiments, the matching module includes:

[0059] The partitioning submodule is used to divide the first semantic information into multiple grids based on the semantic coordinates, and to divide the second semantic information into multiple grids.

[0060] The first matching submodule is used to match the first grid of the target location in the first semantic information with the target location and multiple second grids around the target location in the second semantic information, and to determine the target grid with the highest matching degree with the first grid among the multiple second grids;

[0061] The second matching submodule is used to match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs.

[0062] In some possible embodiments, it also includes:

[0063] The calculation module is used to calculate the degree of misalignment between the first point cloud data and the second point cloud data;

[0064] The quantity determination module is used to determine the number of grids in multiple second grids based on the degree of misalignment. The higher the degree of misalignment, the more grids are in the multiple second grids.

[0065] In some possible embodiments, the degree of misalignment between the first point cloud data and the second point cloud data is calculated using the following formula. :

[0066]

[0067]

[0068] in, Indicates the type within the first grid m and the second grid n. semantic similarity, Indicates the type within the first grid m The total number of semantics, Indicates the type within the second grid n The total number of semantics, Representation type The weights.

[0069] In some possible embodiments, the plurality of second grids includes the grid of the target location in the second semantic information, and eight grids adjacent to the grid of the target location.

[0070] In some possible embodiments, the second matching submodule is used to match the geometric relationships between semantics in the target grid with the geometric relationships between semantics in the first grid to obtain at least two matching semantic pairs.

[0071] In some possible embodiments, the semantics in the first grid and the corresponding semantics in the second grid are determined to form a matching semantic pair when the semantics in the first grid and the semantics in the target grid satisfy the following conditions:

[0072] Less than °and Less than rice;

[0073] Less than °and Less than rice;

[0074] in, and All of these are pre-set constants. , representing the set of angles between the normal vectors of the face semantics and the line semantics in the first mesh { The i-th element in} The set of angles between the surface semantic normal vector and the line semantic normal vector in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type;

[0075] , represents the set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the first grid. The i-th element in} The set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type;

[0076] , representing the set of angles between the semantic center point of the face, the line connecting the points, and the semantic normal vector of the face in the first grid. The i-th element in} The set of angles between the semantic center point of the surface in the target mesh and the line connecting the point semantic center point and the surface semantic normal vector { The j-th element in} Differences and The corresponding semantic indicators are of the same type;

[0077] , represents the set of semantic geometric distances between the semantic center points of faces and the points in the first grid. The i-th element in} The set of semantic geometric distances between the semantic center point of the surface and the point semantic geometric distance in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0078] In some possible embodiments, the registration module is used to extract matching data corresponding to the positions of matching semantic pairs in the first point cloud data and the second point cloud data;

[0079] Based on the matching data, the first point cloud data and the second point cloud data are registered.

[0080] In some possible embodiments, the registration module is used to register the coordinates in the first point cloud data and the second point cloud data. to Data within the specified range is extracted as matching data;

[0081]

[0082]

[0083] X, Y, and Z represent the sets of coordinate points in the x, y, and z directions of the matching semantic pair in the local coordinate system, respectively. Let X be the minimum value in set X. The minimum value in set Y. This represents the minimum value in set Z. The maximum value in set X. The maximum value in set Y The maximum value in set Z. This is the preset error distance.

[0084] In some possible embodiments, the registration module is used to register the first point cloud data and the second point cloud data according to the matching data using an iterative nearest-point algorithm.

[0085] In some possible embodiments, it also includes:

[0086] The verification data extraction module is used to extract verification data other than the positions of matching semantic pairs in the first point cloud data and the second point cloud data.

[0087] The verification module is used to verify the first and second point cloud data after registration based on the verification data.

[0088] Thirdly, embodiments of this application provide a computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to perform the aforementioned point cloud registration method.

[0089] Therefore, the point cloud registration method provided in this application can be understood to obtain first semantic information and second semantic information by parsing first point cloud data and second point cloud data. Matching semantic pairs can be obtained by matching the first semantic information and second semantic information. The amount of point cloud data required for point cloud registration can be reduced according to the location information of the matching semantic pairs, thereby improving the accuracy of point cloud registration.

[0090] Other advantages of this application will be explained in more detail with reference to the following description and figures.

[0091] It should be understood that the above description is merely an overview of the technical solution of this application, so as to enable a clearer understanding of the technical means of this application and thus allow for its implementation in accordance with the contents of the specification. To make the above and other objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application are illustrated below. Attached Figure Description

[0092] By reading the detailed description of the exemplary embodiments below, those skilled in the art will understand the advantages and benefits described herein, as well as other advantages and benefits. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. In the drawings:

[0093] Figure 1 A flowchart illustrating a point cloud registration method provided in an embodiment of this application;

[0094] Figure 2 A flowchart illustrating a semantic matching method provided in an embodiment of this application;

[0095] Figure 3 A schematic diagram of a grid position provided for an embodiment of this application;

[0096] Figure 4 This is a schematic diagram of a point cloud registration device provided in an embodiment of this application.

[0097] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0098] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0099] In the description of embodiments of this application, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of the disclosed features, figures, steps, behaviors, components, portions or combinations thereof in this specification, and do not exclude the possibility of the presence of one or more other features, figures, steps, behaviors, components, portions or combinations thereof.

[0100] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B at the same time, and B alone.

[0101] The terms "first," "second," etc., are used only for ease of description to distinguish identical or similar technical features and should not be construed as indicating or implying the relative importance or number of these technical features. Therefore, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, the term "multiple" means two or more.

[0102] It should also be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0103] See Figure 1 The figure is a flowchart of a point cloud registration method provided in an embodiment of this application.

[0104] like Figure 1 As shown, the point cloud registration method provided in this application includes:

[0105] S101: Parse the first point cloud data and the second point cloud data to obtain first semantic information and second semantic information. The first semantic information includes multiple semantics with coordinates. The semantics in the first semantic information indicate the type of the element in the first point cloud data. The second semantic information includes multiple semantics with coordinates. The semantics in the second semantic information indicate the type of the element in the second point cloud data.

[0106] The types of elements in the embodiments of this application may include streetlights, road signs, traffic lights, lane lines, guardrails, utility poles, ground symbols, etc., and are not limited thereto in this application. Based on the geometric shape characteristics of the elements, this application may also classify the elements into point elements, line elements, and surface elements.

[0107] It should be noted that the multiple semantics in the first semantic information in this application embodiment may include point semantics, line semantics, and area semantics, and the multiple semantics in the first point cloud data may include point elements, line elements, and area elements. Specifically, the point semantics in the first semantic information indicates the type of point elements in the first point cloud data, the line semantics in the first semantic information indicates the type of line semantics in the first point cloud data, and the area semantics in the first semantic information indicates the type of area elements in the first point cloud data; the multiple semantics in the second semantic information include point semantics, line semantics, and area semantics, and the multiple semantics in the second point cloud data include point elements, line elements, and area elements; the point semantics in the second semantic information indicates the type of point elements in the second point cloud data, the line semantics in the second semantic information indicates the type of line semantics in the second point cloud data, and the area semantics in the second semantic information indicates the type of area elements in the second point cloud data.

[0108] S102: Match the semantics in the first semantic information with the semantics in the second semantic information to obtain a matching semantic pair.

[0109] In this embodiment of the application, in order to improve the matching speed, the first semantic information and the second semantic information to be matched can be divided into multiple grids before matching, and then the semantics in the successfully matched grids can be matched. Specific implementation methods may include the following steps S201-S203:

[0110] S201: Divide the first semantic information into multiple grids according to the semantic coordinates, and divide the second semantic information into multiple grids.

[0111] In practical applications, the number of second grids can be determined based on the degree of misalignment between the first and second point cloud data. This application first calculates the degree of misalignment between the first and second point cloud data. Then, it determines the number of grids in the multiple second grids based on the degree of misalignment. It should be noted that the higher the degree of misalignment between the first and second point cloud data, the more grids are in the multiple second grids. As an example, such as... Figure 3 As shown, the multiple second grids may include nine second grids. Specifically, the nine second grids include grid 5, which represents the target location in the second semantic information, and eight grids adjacent to the grid representing the target location, namely grid 1, grid 2, grid 3, grid 4, grid 6, grid 7, grid 8, and grid 9.

[0112] As one possible implementation, this application can calculate the degree of misalignment between the first point cloud data and the second point cloud data using the following formula. :

[0113]

[0114]

[0115] in, Indicates the type within the first grid m and the second grid n. semantic similarity, Indicates the type within the first grid m The total number of semantics, Indicates the type within the second grid n The total number of semantics, Representation type The weights.

[0116] S202: Match the first grid of the target location in the first semantic information with the target location and multiple second grids around the target location in the second semantic information, and determine the target grid with the highest matching degree with the first grid among the multiple second grids.

[0117] As one possible implementation, this application can calculate the similarity between the first grid and multiple second grids respectively, and then take the second grid with the highest similarity among the multiple second grids as the target grid.

[0118] S203: Match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs.

[0119] As one possible implementation, this application can match the geometric relationships between semantics in the target grid with the geometric relationships between semantics in the first grid to obtain at least two matching semantic pairs. As an example, when the semantics in the first grid and the semantics in the target grid satisfy the following conditions, it is determined that the semantics in the first grid and the corresponding semantics in the second grid form a matching semantic pair:

[0120] Less than °and Less than rice;

[0121] Less than °and Less than rice.

[0122] in, and All of these are pre-set constants. , representing the set of angles between the normal vectors of the face semantics and the line semantics in the first mesh { The i-th element in} The set of angles between the surface semantic normal vector and the line semantic normal vector in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0123] , represents the set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the first grid. The i-th element in} The set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0124] , representing the set of angles between the semantic center point of the face, the line connecting the points, and the semantic normal vector of the face in the first grid. The i-th element in} The set of angles between the semantic center point of the surface in the target mesh and the line connecting the point semantic center point and the surface semantic normal vector { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0125] , represents the set of semantic geometric distances between the semantic center points of faces and the points in the first grid. The i-th element in} The set of semantic geometric distances between the semantic center point of the surface and the point semantic geometric distance in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0126] S103: Register the first point cloud data and the second point cloud data according to the matching semantic pair.

[0127] In this embodiment of the application, matching data corresponding to the positions of matching semantic pairs in the first point cloud data and the second point cloud data can be extracted; then, the first point cloud data and the second point cloud data can be registered based on the matching data pairs.

[0128] In practical applications, this application can convert the coordinates in the first point cloud data and the second point cloud data into... to Data within the specified range is extracted as matching data;

[0129]

[0130]

[0131] X, Y, and Z represent the sets of coordinate points in the x, y, and z directions of the matching semantic pair in the local coordinate system, respectively. Let X be the minimum value in set X. The minimum value in set Y. This represents the minimum value in set Z. The maximum value in set X. The maximum value in set Y The maximum value in set Z. This is the preset error distance.

[0132] A 60-second data collection session in a certain industrial park yielded 603 frames of laser point cloud data, totaling 23,723,132 scan points. The extracted semantics consist of 8 point semantics, 1 line semantics, and 9 area semantics. The selection criteria are as follows: Using a 0.5-meter reference distance, the z-direction range for point semantics is set separately to 3 meters. After filtering all semantics, 577 frames remain, totaling 1,423,678 scans. The data compression ratio is 1,423,678 / 23,723,132 ≈ 0.06. Therefore, it is evident that the filtering method in this embodiment can significantly reduce the number of input points involved in subsequent ICP fine registration, thus greatly improving ICP registration efficiency.

[0133] As one possible implementation, this application can register the first point cloud data and the second point cloud data using the Iterative Nearest Point (ICP) algorithm based on the matching data. As an example, this application can construct parallel constraints between faces and overlap constraints between points, as described in detail below:

[0134] (1) Constrained observation equations for points

[0135] Assume that the semantic point of the matching data in the second semantic information is the source semantic point. The semantic points in the first semantic information that match the data are the target semantic points. The corresponding rotation matrices of the source point cloud and the target point cloud are: The translation relationship is Then the point after the source point transformation is:

[0136]

[0137] The corresponding cost function is

[0138]

[0139] Using the left perturbation model on the special Euclidean group SE(3), the corresponding Jacobian matrix is:

[0140]

[0141] in, The complete expression for an antisymmetric matrix is:

[0142]

[0143] (2) The constrained observation equations for the surface are:

[0144] First, select three points within the plane of the surface semantics to form two non-parallel vectors. Then, solve for the normal vector of this plane and normalize it. Assume the unit normal vectors corresponding to the source semantics and the target semantics are... and When calculating the normal vector, the chosen directions must correspond to avoid reversed normal vectors. Assume the source and target semantics are specified as a set of corresponding vector edge unit vectors. and After transforming the source semantic vector, we can obtain:

[0145]

[0146]

[0147] The core idea of ​​the constraint is that the angle between the normal and edge vectors of the source semantics and the normal and edge vectors of the target semantics after transformation should be as small as possible. This is an angle constraint. To convert the angle constraint to a distance constraint, it is easier to optimize together with the point constraint. The constraint becomes: the magnitude of the difference between the relevant vector of the source semantics and the relevant vector of the target semantics after adjustment should be as small as possible. Since the influence of the angle is proportional to the length of the vector, the length of the vector needs to be considered. Therefore, the corresponding cost function is...

[0148]

[0149]

[0150] Using the left perturbation model, the corresponding Jacobian matrix is:

[0151]

[0152]

[0153] in, This represents the length of the longest side of the semantic object.

[0154] (3) Optimization method

[0155] After integrating point semantics and surface semantics observations, the final optimization conditions are designed as follows:

[0156]

[0157]

[0158] In the formula, This represents the minimization operator. The parameters in the equation represent, in order, roll angle, pitch angle, yaw angle, x-axis translation, y-axis translation, and z-axis translation. This is a nonlinear optimization problem, which can be solved using open-source mathematical libraries such as Ceres (Ceres Non-linear Optimization Library), GTSAM (Georgia Tech Smoothing and Mapping), and g2o (General Graph Optimization). It should be noted that although these open-source libraries come from different sources, their underlying principles are the same; they are all implementations of nonlinear optimization mathematical algorithms. Therefore, this application does not impose any restrictions on the open-source mathematical libraries used.

[0159] Finally, in this embodiment of the application, the coarse registration result obtained above can be used as the initial iteration value to perform standard ICP algorithm calculation on the selected source point cloud and target point cloud to obtain fine registration result.

[0160] Furthermore, this application can also extract verification data beyond the positions of matching semantic pairs in the first and second point cloud data; then, based on the verification data, the first and second point cloud data after registration are verified using the aforementioned Iterative Nearest Point (ICP) algorithm. The application of the ICP algorithm can be referred to the description of processing the matching data above, and will not be repeated here in the embodiments of this application. Based on the existing convergence judgment conditions, this application adds a method for constraint self-checking through redundant semantic observations, making the point cloud registration results more reliable.

[0161] This application utilizes the aforementioned observation equations to calculate residuals and verifies each residual. If all residuals are less than a preset value, such as 0.1m, the fine registration result is considered reliable. Otherwise, the point cloud selection range is... Expand to Then, step S103 is executed again. At this time, the coordinate range of the matching data in the retained first point cloud data and second point cloud data is... to .

[0162] It should be noted that the semantic information in this embodiment is clearer and more precise in meaning than feature points, lines, and surfaces. Matching based on the geometric relationships between semantic elements utilizes the high relative accuracy of laser point cloud measurements, is unaffected by absolute positional errors, and is more robust than methods that match semantic points according to the nearest neighbor principle.

[0163] In summary, the point cloud registration method provided in this application, by parsing the first and second point cloud data to obtain first and second semantic information, can obtain matching semantic pairs through matching the first and second semantic information. This allows for narrowing the scope of point cloud registration based on the location information of the matching semantic pairs, reducing the amount of data required for point cloud registration, and improving both the accuracy and efficiency of point cloud registration. Furthermore, this application rapidly calculates the initial transformation matrix through semantic constraints, which significantly accelerates the efficiency of point cloud registration.

[0164] In the description of this specification, references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this application, and the aforementioned terms do not necessarily refer to the same implementation or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0165] The method flowcharts for embodiments of this application describe certain operations as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, or some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., by software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0166] Those skilled in the art will understand that in the methods described in the above specific embodiments, the order in which the steps are written does not imply a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.

[0167] Based on the point cloud registration method provided in the above embodiments, this application also provides a point cloud registration device.

[0168] like Figure 4 As shown, the point cloud registration device provided in this application embodiment includes:

[0169] The parsing module 100 is used to parse the first point cloud data and the second point cloud data to obtain first semantic information and second semantic information. The first semantic information includes multiple semantics with coordinates. The semantics in the first semantic information indicate the type of the element in the first point cloud data. The second semantic information includes multiple semantics with coordinates. The semantics in the second semantic information indicate the type of the element in the second point cloud data.

[0170] The matching module 200 is used to match the semantics in the first semantic information and the semantics in the second semantic information to obtain matching semantic pairs;

[0171] The registration module 300 is used to register the first point cloud data and the second point cloud data according to the matching semantic pairs.

[0172] As one possible implementation, the multiple semantics in the first semantic information include point semantics, line semantics, and area semantics; the multiple semantics in the first point cloud data include point elements, line elements, and area elements; the point semantics in the first semantic information indicate the type of point elements in the first point cloud data; the line semantics in the first semantic information indicate the type of line semantics in the first point cloud data; and the area semantics in the first semantic information indicate the type of area elements in the first point cloud data. Similarly, the multiple semantics in the second semantic information include point semantics, line semantics, and area semantics; the multiple semantics in the second point cloud data include point elements, line elements, and area elements; the point semantics in the second semantic information indicate the type of point elements in the second point cloud data; the line semantics in the second semantic information indicate the type of line semantics in the second point cloud data; and the area semantics in the second semantic information indicate the type of area elements in the second point cloud data.

[0173] As one possible implementation, the matching module includes: a partitioning submodule, used to divide the first semantic information into multiple grids according to the semantic coordinates, and to divide the second semantic information into multiple grids; a first matching submodule, used to match the first grid of the target position in the first semantic information with the target position and multiple second grids around the target position in the second semantic information, and to determine the target grid with the highest matching degree with the first grid among the multiple second grids; and a second matching submodule, used to match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs.

[0174] As one possible implementation, it also includes: a calculation module for calculating the degree of misalignment between the first point cloud data and the second point cloud data; and a quantity determination module for determining the number of grids in the multiple second grids based on the degree of misalignment, wherein the higher the degree of misalignment, the more grids are in the multiple second grids.

[0175] As one possible implementation, the degree of misalignment between the first point cloud data and the second point cloud data is calculated using the following formula. :

[0176]

[0177]

[0178] in, Indicates the type within the first grid m and the second grid n. semantic similarity, Indicates the type within the first grid m The total number of semantics, Indicates the type within the second grid n The total number of semantics, Representation type The weights. As one possible implementation, the multiple second grids include the grid representing the target location in the second semantic information, and eight grids adjacent to the grid representing the target location.

[0179] As one possible implementation, the second matching submodule is used to match the geometric relationships between semantics in the target grid with the geometric relationships between semantics in the first grid to obtain at least two matching semantic pairs.

[0180] As one possible implementation, when the semantics in the first grid and the semantics in the target grid satisfy the following conditions, it is determined that the semantics in the first grid and the corresponding semantics in the second grid form a matching semantic pair:

[0181] Less than °and Less than rice;

[0182] Less than °and Less than rice;

[0183] in, and All of these are pre-set constants. , representing the set of angles between the normal vectors of the face semantics and the line semantics in the first mesh { The i-th element in} The set of angles between the surface semantic normal vector and the line semantic normal vector in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , represents the set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the first grid. The i-th element in} The set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , representing the set of angles between the semantic center point of the face, the line connecting the points, and the semantic normal vector of the face in the first grid. The i-th element in} The set of angles between the semantic center point of the surface in the target mesh and the line connecting the point semantic center point and the surface semantic normal vector { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , represents the set of semantic geometric distances between the semantic center points of faces and the points in the first grid. The i-th element in} The set of semantic geometric distances between the semantic center point of the surface and the point semantic geometric distance in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

[0184] As one possible implementation, the registration module is used to extract matching data corresponding to the positions of matching semantic pairs in the first point cloud data and the second point cloud data; and to register the first point cloud data and the second point cloud data according to the matching data pairs.

[0185] As one possible implementation, a registration module is used to register the coordinates in the first point cloud data and the second point cloud data. to Data within the specified range is extracted as matching data;

[0186]

[0187]

[0188] X, Y, and Z represent the sets of coordinate points in the x, y, and z directions of the matching semantic pair in the local coordinate system, respectively. Let X be the minimum value in set X. The minimum value in set Y. This represents the minimum value in set Z. The maximum value in set X. The maximum value in set Y The maximum value in set Z. This is the preset error distance.

[0189] In one possible implementation, a registration module is used to register the first point cloud data and the second point cloud data using an iterative nearest-point algorithm based on the matching data. In another possible implementation, it further includes: a verification data extraction module, used to extract verification data from the first point cloud data and the second point cloud data excluding the positions of the matching semantic pairs; and a verification module, used to verify the registered first point cloud data and the second point cloud data based on the verification data.

[0190] It should be noted that the point cloud registration device in the embodiments of this application can implement each process of the aforementioned point cloud registration method embodiments and achieve the same effect and function, which will not be repeated here.

[0191] According to the point cloud registration method and point cloud registration apparatus in the above embodiments, a non-volatile computer storage medium for point cloud registration method is provided, which stores computer-executable instructions configured to be executed by a processor when running: the method described in the above embodiments.

[0192] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. Furthermore, although the operations of the methods of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally, certain steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple sub-steps.

[0193] While the spirit and principles of this application have been described above with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A point cloud registration method, characterized in that, include: The first point cloud data and the second point cloud data are parsed to obtain first semantic information and second semantic information. The first semantic information includes multiple semantics with coordinates, and the semantics in the first semantic information indicate the type of the element in the first point cloud data. The second semantic information includes multiple semantics with coordinates, and the semantics in the second semantic information indicate the type of the element in the second point cloud data. The first semantic information is divided into multiple grids based on the semantic coordinates, and the second semantic information is also divided into multiple grids. The first grid of the target location in the first semantic information is matched with the target location and multiple second grids around the target location in the second semantic information, and the target grid with the highest matching degree with the first grid is determined among the multiple second grids; Match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs; The first point cloud data and the second point cloud data are registered according to the matching semantic pair.

2. The method according to claim 1, characterized in that, The semantics in the first semantic information include point semantics, line semantics, and surface semantics, and the semantics in the first point cloud data include point elements, line elements, and surface elements; The point semantics in the first semantic information indicate the type of point elements in the first point cloud data; the line semantics in the first semantic information indicate the type of line semantics in the first point cloud data; and the surface semantics in the first semantic information indicate the type of surface elements in the first point cloud data. The second semantic information includes point semantics, line semantics, and surface semantics; the second point cloud data includes point elements, line elements, and surface elements. The point semantics in the second semantic information indicate the type of point elements in the second point cloud data; the line semantics in the second semantic information indicate the type of line semantics in the second point cloud data; and the area semantics in the second semantic information indicate the type of area elements in the second point cloud data.

3. The method according to claim 1, characterized in that, Also includes: Calculate the degree of misalignment between the first point cloud data and the second point cloud data; The number of grids in the plurality of second grids is determined based on the degree of misalignment; the higher the degree of misalignment, the more grids are in the plurality of second grids.

4. The method according to claim 3, characterized in that, The degree of misalignment between the first point cloud data and the second point cloud data is calculated using the following formula. : in, Indicates the type within the first grid m and the second grid n. semantic similarity, Indicates the type within the first grid m The total number of semantics, Indicates the type within the second grid n The total number of semantics, Representation type The weights.

5. The method according to claim 1, characterized in that, The plurality of second grids includes the grid of the target location in the second semantic information, and eight grids adjacent to the grid of the target location.

6. The method according to claim 1, characterized in that, The step of matching the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs includes: The geometric relationships between semantics in the target grid are matched with the geometric relationships between semantics in the first grid to obtain at least two matching semantic pairs.

7. The method according to claim 6, characterized in that, When the semantics in the first grid and the semantics in the target grid satisfy the following conditions, it is determined that the semantics in the first grid and the corresponding semantics in the second grid form the matching semantic pair: Less than °and Less than rice; Less than °and Less than rice; in, and All of these are pre-set constants. , representing the set of angles between the normal vectors of the face semantics and the line semantics in the first mesh { The i-th element in} The set of angles between the surface semantic normal vector and the line semantic normal vector in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , representing the set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the first grid. The i-th element in} The set of geometric distances between the semantic center points of surfaces and the semantic center points of lines in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , representing the set of angles between the semantic center point of the surface and the semantic line connecting the points and the semantic normal vector of the surface in the first grid. The i-th element in} The set of angles between the face semantic center point and the point semantic line connecting the face semantic center point and the face semantic normal vector in the target mesh { The j-th element in} Differences and The corresponding semantic indicators are of the same type; , representing the set of semantic geometric distances between the face semantic center point and the point semantic geometric distance in the first grid. The i-th element in} The set of semantic geometric distances between the semantic center point of the surface and the point semantic geometric distance in the target mesh { The j-th element in} Differences and The corresponding semantic indicators have the same type.

8. The method according to claim 1, characterized in that, The registration of the first point cloud data and the second point cloud data according to the matching semantic pair includes: Extract the matching data corresponding to the position of the matching semantic pair in the first point cloud data and the second point cloud data; The first point cloud data and the second point cloud data are registered based on the matching data.

9. The method according to claim 8, characterized in that, The step of extracting the matching data corresponding to the position of the matching semantic pair in the first point cloud data and the second point cloud data includes: The coordinates in the first point cloud data and the second point cloud data are... to Data within the specified range is extracted as matching data; X, Y, and Z represent the sets of coordinate points in the x, y, and z directions of the matching semantic pair in the local coordinate system, respectively. Let X be the minimum value in set X. The minimum value in set Y. This represents the minimum value in set Z. The maximum value in set X. The maximum value in set Y The maximum value in set Z. This is the preset error distance.

10. The method according to claim 8, characterized in that, The step of registering the first point cloud data and the second point cloud data according to the matching data includes: Based on the matching data, the first point cloud data and the second point cloud data are registered using an iterative nearest point algorithm.

11. The method according to claim 8, characterized in that, Also includes: Extract verification data other than the positions of the matching semantic pairs in the first point cloud data and the second point cloud data; The first and second point cloud data after registration are verified based on the verification data.

12. A point cloud registration device, characterized in that, include: The parsing module is used to parse the first point cloud data and the second point cloud data to obtain first semantic information and second semantic information. The first semantic information includes multiple semantics with coordinates, and the semantics in the first semantic information indicate the type of the element in the first point cloud data. The second semantic information includes multiple semantics with coordinates, and the semantics in the second semantic information indicate the type of the element in the second point cloud data. The partitioning submodule is used to divide the first semantic information into multiple grids based on the semantic coordinates, and to divide the second semantic information into multiple grids. The first matching submodule is used to match the first grid of the target location in the first semantic information with the target location and multiple second grids around the target location in the second semantic information, and to determine the target grid with the highest matching degree with the first grid among the multiple second grids; The second matching submodule is used to match the semantics in the first grid with the semantics in the target grid to obtain at least two matching semantic pairs; The registration module is used to register the first point cloud data and the second point cloud data according to the matching semantic pair.

13. A computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to perform the method as described in any one of claims 1-11.

Images