Method, apparatus, device, and storage medium for processing high-precision map point cloud data

Abstract

The present disclosure provides a method, apparatus, device, and storage medium for processing high-precision map point cloud data, which relates to the field of data processing, and particularly relates to technical fields such as autonomous driving, high-precision maps, and three-dimensional reconstruction. The specific implementation solution is as follows: obtaining multiple frames of first point cloud data collected by a collection device during the movement in a first location; determining a first relative pose corresponding to each frame of the first point cloud data according to the multiple frames of first point cloud data; generating a first voxel map according to the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively, and updating the first relative poses corresponding to the multiple frames of first point cloud data according to the first voxel map to obtain second relative poses corresponding to the multiple frames of first point cloud data respectively; performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively to obtain a three-dimensional map corresponding to the first location. The solution of the present disclosure can improve the accuracy of the three-dimensional map.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular, to a method, apparatus, device, and storage medium for processing high-precision map point cloud data, which can be used in technical fields such as autonomous driving, high-precision maps, and 3D reconstruction. Background Art

[0002] A high-precision map, also known as a high-accuracy map, can be used by autonomous vehicles. A high-precision map has accurate vehicle position information and rich road element data information, which can help a vehicle predict complex road surface information, such as slope, curvature, heading, etc., and better avoid potential risks. Currently, 3D reconstruction technology can be used in high-precision map applications. Exemplarily, multiple frames of point cloud data can be collected for a certain place, and 3D reconstruction can be performed on the multiple frames of point cloud data to obtain a 3D map of the place.

[0003] When performing map reconstruction for an outdoor place, the global pose corresponding to each frame of point cloud data can be obtained by using the Global Positioning System (GPS). Furthermore, multiple frames of point cloud data can be registered according to the global pose corresponding to each frame of point cloud data to obtain a 3D map. For an indoor place or a place without GPS coverage, since the global pose corresponding to each frame of point cloud data cannot be obtained by using GPS, it is necessary to first determine the relative pose corresponding to each frame of point cloud data, and then register the multiple frames of point cloud data according to the relative pose to obtain a 3D map.

[0004] However, in practical applications, the accuracy of the 3D map constructed for an indoor place or a place without GPS coverage by using the above method is not high. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, device, and storage medium for processing high-precision map point cloud data.

[0006] According to a first aspect of the present disclosure, there is provided a method for processing high-precision map point cloud data, including:

[0007] Obtaining multiple frames of first point cloud data collected by a collection device during the movement in a first place;

[0008] Determining a first relative pose corresponding to each frame of the first point cloud data according to the multiple frames of the first point cloud data, where the first relative pose is used to indicate the pose change information of the collection device when collecting this frame of the first point cloud data relative to the previous frame of the first point cloud data when collecting this frame of the first point cloud data;

[0009] Generate a first voxel map based on the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively, and update the first relative poses corresponding to the multi-frame first point cloud data according to the first voxel map to obtain the second relative poses corresponding to the multi-frame first point cloud data respectively;

[0010] Perform three-dimensional reconstruction on the multi-frame first point cloud data according to the second relative poses corresponding to the multi-frame first point cloud data respectively to obtain the three-dimensional map corresponding to the first site.

[0011] According to a second aspect of the present disclosure, there is provided a processing device for high-precision map point cloud data, including:

[0012] An acquisition module for acquiring multi-frame first point cloud data collected by an acquisition device during movement in a first site;

[0013] A determination module for determining, according to the multi-frame first point cloud data, a first relative pose corresponding to each frame of the first point cloud data, where the first relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of the first point cloud data relative to the previous frame of the first point cloud data;

[0014] An update module for generating a first voxel map based on the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively, and updating the first relative poses corresponding to the multi-frame first point cloud data according to the first voxel map to obtain the second relative poses corresponding to the multi-frame first point cloud data respectively;

[0015] A three-dimensional reconstruction module for performing three-dimensional reconstruction on the multi-frame first point cloud data according to the second relative poses corresponding to the multi-frame first point cloud data respectively to obtain the three-dimensional map corresponding to the first site.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0020] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0021] According to a fifth aspect of the present disclosure, there is provided a computer program product, which includes: a computer program stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to execute the method described in the first aspect.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0023] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0024] Figure 1 It is a schematic diagram of an application scenario provided for an embodiment of the present disclosure;

[0025] Figure 2 It is a schematic flowchart of a method for processing high-precision map point cloud data provided for an embodiment of the present disclosure;

[0026] Figure 3 It is a schematic diagram of the generation process of a voxel map provided for an embodiment of the present disclosure;

[0027] Figure 4 It is a schematic flowchart of another method for processing high-precision map point cloud data provided for an embodiment of the present disclosure;

[0028] Figure 5 It is a schematic flowchart of yet another method for processing high-precision map point cloud data provided for an embodiment of the present disclosure;

[0029] Figure 6 It is a schematic diagram of the layering of the first point cloud data provided for an embodiment of the present disclosure;

[0030] Figure 7 It is a schematic diagram of a process for processing high-precision map point cloud data provided for an embodiment of the present disclosure;

[0031] Figure 8 It is a schematic structural diagram of a device for processing high-precision map point cloud data provided for an embodiment of the present disclosure;

[0032] Figure 9 It is a schematic structural diagram of an electronic device provided for an embodiment of the present disclosure. Detailed Embodiments

[0033] The exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0034] First, to facilitate understanding of the technical solution of the present disclosure, concepts and terms related to the present disclosure will be explained first.

[0035] Point cloud data: By measuring the spatial coordinates of each sampling point on the surface of an object with a measuring instrument, a set of points is obtained, which is called a point cloud. The point cloud data includes the three-dimensional coordinates of each point and may also include other information, such as laser reflection intensity information, color information, etc. In this embodiment, the above-mentioned measuring instrument may be a lidar.

[0036] Pose: It refers to position and attitude. Generally speaking, the position can be represented by parameters such as coordinates and translation matrices, and the attitude can be represented by parameters such as angles and rotation matrices.

[0037] Voxel map: A voxel refers to the smallest unit in the three-dimensional space segmentation. A voxel map may include multiple voxels.

[0038] K-Dimension tree (KD-tree): It is a data structure for spatial partitioning and is often used for searches in high-dimensional spaces, such as range searches and nearest neighbor searches. The construction of the KD-tree is to arrange the unordered point cloud in an ordered manner according to a certain order to facilitate fast and efficient retrieval.

[0039] The following will be combined with Figure 1 Describe the application scenarios of the present disclosure.

[0040] Figure 1 It is a schematic diagram of an application scenario provided for the embodiments of the present disclosure. As Figure 1 shown, this application scenario exemplifies two stages, namely the map generation stage and the map usage stage.

[0041] In the map generation stage, the map generation device can generate a three-dimensional map corresponding to Place A. Refer to Figure 1, the map generation device can obtain multiple frames of point cloud data. In some examples, the above point cloud data is collected by the collection device during the movement in Place A. Exemplarily, the collection device can be a lidar, or other devices capable of collecting point cloud data. Taking the lidar as an example, the above multiple frames of point cloud data can be collected in the following way: A vehicle / robot is equipped with a lidar. During the movement of the vehicle / robot in Place A, the lidar collects data on Place A to obtain multiple frames of point cloud data.

[0042] Continue to refer to Figure 1 , the map generation device performs three-dimensional reconstruction processing on the multiple frames of point cloud data to generate a three-dimensional map corresponding to Place A. The three-dimensional map generated by the map generation device can be stored in the cloud server.

[0043] Furthermore, continue to refer to Figure 1 , in the map usage stage, the terminal device can obtain the three-dimensional map from the cloud server and use it in the navigation application of the terminal device. Exemplarily, when the user carries the terminal device and moves (such as walking or driving) in Place A, the terminal device can display the three-dimensional map through the navigation application and plan a movement path for the user according to the user's target address, so that the user can reach the target address according to the movement path.

[0044] Among them, Figure 1 the map generation device shown can be a device in the cloud server. For example, the map generation device can be a processor, chip, chip module, module or unit in the cloud server. The map generation device can also be a device independent of the cloud server. For example, the map generation device can be a computer, computer, server, etc. with certain computing capabilities.

[0045] In some scenarios, the above Place A is an outdoor place. For example, Place A is an outdoor parking lot, amusement park, zoo, etc. These places can be covered by GPS. When collecting point cloud data, the global pose corresponding to each frame of point cloud data can be obtained by using GPS. Among them, the global pose corresponding to one frame of point cloud data refers to the pose of the lidar in the world coordinate system when the lidar collects this frame of point cloud data. Therefore, the multiple frames of point cloud data can be registered according to the global poses corresponding to each frame of point cloud data to obtain a three-dimensional map.

[0046] In other scenarios, the above Place A is an indoor place or a place without GPS coverage. For example, Place A is an underground parking lot, indoor parking lot, large shopping mall, etc. Since the global pose corresponding to each frame of point cloud data cannot be obtained by using GPS, it is necessary to first obtain the relative pose corresponding to each frame of point cloud data, and then register the multiple frames of point cloud data according to the relative pose to obtain a three-dimensional map.

[0047] However, in practical applications, for indoor venues or venues without GPS coverage, the accuracy of the three-dimensional map constructed by the above method is not high.

[0048] The present disclosure provides a method, an apparatus, a device, and a storage medium for processing high-precision map point cloud data, which are applied to technical fields such as autonomous driving, high-precision maps, and three-dimensional reconstruction in the field of data processing, and can be used to construct a three-dimensional map corresponding to an indoor venue or a venue without GPS coverage, so as to improve the accuracy of the three-dimensional map.

[0049] In the solution of the present disclosure, after determining the relative poses corresponding to each frame of point cloud data, the relative poses corresponding to each frame of point cloud data can be globally optimized according to each frame of point cloud data to obtain the optimized relative poses of each frame of point cloud data. Furthermore, according to the optimized relative poses of each frame of point cloud data, three-dimensional reconstruction processing is performed on each frame of point cloud data to obtain a three-dimensional map. In the above process, since the relative poses corresponding to each frame of point cloud data are globally optimized, the relative poses of each frame of point cloud data are more accurate. Therefore, the accuracy of the three-dimensional map can be improved.

[0050] The technical solution of the present disclosure will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0051] Figure 2 FIG. is a schematic flowchart of a method for processing high-precision map point cloud data provided by an embodiment of the present disclosure. The method of this embodiment can be executed by Figure 1 the map generation device in. As Figure 2 shown, the method of this embodiment includes:

[0052] S201: Obtain multiple frames of first point cloud data collected by a collection device during the movement in a first venue.

[0053] Wherein, the first venue is an indoor venue or a venue without GPS coverage. For example, the first venue can be an underground garage, an indoor parking lot, a large shopping mall, etc. The collection device can be a lidar or other device for collecting point cloud data. The collection device can be set in a vehicle or in a robot, and the collection device moves with the vehicle / robot in the first venue and collects multiple frames of first point cloud data.

[0054] It can be understood that in the application scenario of the present disclosure, the first venue is an indoor venue or a venue without GPS coverage, and the global pose corresponding to the first point cloud data cannot be obtained through GPS. Therefore, in this embodiment, the global pose corresponding to the first point cloud data is unknown.

[0055] S202: Determine a first relative pose corresponding to each frame of the first point cloud data according to the multi-frame first point cloud data, where the first relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of the first point cloud data relative to the previous frame of the first point cloud data when acquiring this frame of the first point cloud data.

[0056] That is to say, the first relative pose corresponding to the i-th frame of point cloud data indicates the pose change information between the pose of the acquisition device when acquiring the i-th frame of point cloud data and the pose when acquiring the (i - 1)-th frame of point cloud data, where i is a natural number greater than 1.

[0057] The pose in the embodiments of the present disclosure refers to position and attitude. The global pose can be represented by the absolute coordinates and orientation of the lidar. The relative pose can be represented by a translation matrix and a rotation matrix.

[0058] It should be noted that this embodiment does not make specific limitations on the method for determining the first relative pose corresponding to each frame of the first point cloud data. Exemplarily, various odometry techniques can be used to determine the first relative pose corresponding to each frame of the first point cloud data.

[0059] S203: Generate a first voxel map according to the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively, and update the first relative poses corresponding to the multi-frame first point cloud data according to the first voxel map to obtain the second relative poses corresponding to the multi-frame first point cloud data respectively.

[0060] In this embodiment, S203 is used to optimize the first relative poses corresponding to each frame of the first point cloud data determined in S202. When optimizing the first relative poses in this embodiment, the idea of global optimization is adopted, that is, when performing relative pose optimization, all frames of point cloud data in the multi-frame first point cloud data and the first relative poses corresponding to all frames of point cloud data are referred to. By globally optimizing the first relative poses corresponding to each frame of the first point cloud data, the accuracy of the relative poses of each frame of the first point cloud data can be improved.

[0061] In this embodiment, the implementation manner of optimizing the first relative poses corresponding to each frame of the first point cloud data is as follows: First, generate a first voxel map according to the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively. Among them, the first voxel map includes a plurality of voxels, each voxel can be regarded as a cube, and the feature points with the same or similar features in the multi-frame point cloud data are associated with a voxel. Each voxel has a voxel feature, and the voxel feature can be calculated according to the feature points associated with this voxel. It can be seen that in this embodiment, the first voxel map indicates the similarity relationship between the feature points in the multi-frame first point cloud data.

[0062] It should be understood that in this embodiment, the first voxel map is generated based on the first point cloud data of all frames. The first voxel map can also be referred to as a global voxel map, which can reflect the distribution of the feature points of the first point cloud data of all frames.

[0063] In some possible implementation manners, for each frame of the first point cloud data, according to the first relative pose corresponding to the frame of the first point cloud data, the frame of the first point cloud data is registered with each voxel in the first voxel map, so that each feature point in the frame of the first point cloud data is respectively associated with a certain voxel in the first voxel map. After performing the above-mentioned association processing on the multiple frames of the first point cloud data respectively, a first voxel map is generated.

[0064] It should be understood that during the process of generating the first voxel map, if a certain voxel newly associates with a feature point, the voxel feature of the voxel needs to be updated. Specifically, according to all the feature points associated with the voxel, the voxel feature of the voxel is calculated. If the voxel feature does not meet the preset conditions, the voxel is split, that is, the voxel is split into 8 voxels.

[0065] Figure 3 It is a schematic diagram of the generation process of the voxel map provided by the embodiment of the present disclosure. As Figure 3 shown in (a) therein, it is assumed that the original voxel map includes a cube. After the splitting process, 8 cubes are obtained, as Figure 3 shown in (b) therein. After the re-splitting process, Figure 3 some of the cubes in (b) are split into 8 cubes, as Figure 3 shown in (c) therein.

[0066] Furthermore, after the first voxel map is generated, the first relative poses corresponding to the multiple frames of the first point cloud data are updated according to the first voxel map, and the second relative poses corresponding to the multiple frames of the first point cloud data are obtained. Exemplarily, for each voxel in the first voxel map, according to the feature points associated with the voxel, a constraint relationship is constructed, and according to the above constraint relationship, calculations are performed to obtain the second relative poses corresponding to each frame of the first point cloud data respectively.

[0067] S204: According to the second relative poses corresponding to the multiple frames of the first point cloud data, perform three-dimensional reconstruction on the multiple frames of the first point cloud data to obtain the three-dimensional map corresponding to the first site.

[0068] It should be noted that the processing method for performing three-dimensional reconstruction on the multiple frames of the first point cloud data is not specifically limited. For example, various existing registration techniques can be used to implement it.

[0069] The method for processing high-precision map point cloud data provided in this embodiment includes: obtaining multiple frames of first point cloud data collected by a collection device during movement in a first place; determining a first relative pose corresponding to each frame of the first point cloud data according to the multiple frames of first point cloud data; generating a first voxel map according to the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively, and updating the first relative poses corresponding to the multiple frames of first point cloud data according to the first voxel map to obtain second relative poses corresponding to the multiple frames of first point cloud data respectively; performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively to obtain a three-dimensional map corresponding to the first place. In the above process, since the second relative poses corresponding to each frame of the first point cloud data are globally optimized according to the first voxel map, the second relative poses corresponding to each frame of the first point cloud data are more accurate. Therefore, performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to each frame of the first point cloud data can improve the accuracy of the reconstructed three-dimensional map.

[0070] Based on the above embodiment, a specific embodiment is combined below to detail how to globally optimize the first relative pose corresponding to the first point cloud data.

[0071] Figure 4 The flowchart of another method for processing high-precision map point cloud data provided in an embodiment of the present disclosure. The method of this embodiment can be used as Figure 2 a possible implementation manner of S203 in Figure 4 As shown, the method of this embodiment includes:

[0072] S401: Determine the line feature points and surface feature points in the multiple frames of first point cloud data.

[0073] Exemplarily, for each frame of point cloud data, the line feature points and surface feature points in this frame of point cloud data can be determined respectively. Specifically, each frame of point cloud data includes multiple points, and this frame of point cloud data can be regarded as a set of multiple points. It can be determined from this set which points are line feature points and which points are surface feature points.

[0074] In a possible implementation manner, taking the i-th frame of point cloud data as an example, the line feature points and surface feature points in the i-th frame of point cloud data can be determined in the following manner.

[0075] For each point in the i-th frame of point cloud data, obtain multiple (e.g., 20) neighboring points of this point in the i-th frame of point cloud data. For example, the k-nearest neighbor algorithm can be used to determine multiple neighboring points of this point. Perform Singular Value Decomposition (SVD) calculation on the multiple neighboring points to obtain eigenvalues λ1 > λ2 > λ3.

[0076] Use the following formula (1) to obtain the line eigenvalue σ of this point 1D :

[0077]

[0078] Use the following formula (2) to obtain the surface eigenvalue σ of this point 2D :

[0079]

[0080] According to the line eigenvalue σ of each point in the i-th frame of point cloud data 1D , perform non-maximum suppression processing on each point in the i-th frame of point cloud data to determine the line feature points in the i-th frame of point cloud data. In a possible implementation, sort each point in the i-th frame of point cloud data in descending order according to the line eigenvalue σ 1D to obtain a sorting result. Construct a KD-tree for the sorting result. The line eigenvalue σ of the first point in the KD-tree 1D is the largest. Select the first point from the KD-tree as the line feature point, and start radius search from the first point according to the structure of the KD-tree, and delete all the points found. Through the above process, the line feature points in the i-th frame of point cloud data can be determined.

[0081] According to the surface eigenvalue σ of each point in the i-th frame of point cloud data 2D , perform non-maximum suppression processing on each point in the i-th frame of point cloud data to determine the surface feature points in the i-th frame of point cloud data. In a possible implementation, sort each point in the i-th frame of point cloud data in descending order according to the surface eigenvalue σ 2D to obtain a sorting result. Construct a KD-tree for the sorting result. The surface eigenvalue σ of the first point in the KD-tree 2D is the largest. Select the first point from the KD-tree as the surface feature point, and start radius search from the first point according to the structure of the KD-tree, and delete all the points found. Through the above process, the surface feature points in the i-th frame of point cloud data can be determined.

[0082] S402: Generate the first voxel map according to the line feature points in the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively.

[0083] In this embodiment, since the first voxel map is generated according to all frames of the first point cloud data, the first voxel map can be referred to as a global voxel map.

[0084] In a possible implementation, for each frame of the first point cloud data, adjust the line feature points in the frame of the first point cloud data according to the first relative pose corresponding to the frame of the first point cloud data to obtain the adjusted line feature points. Perform registration processing on the adjusted line feature points in each frame of the first point cloud data to obtain the registration result of the line feature points. Furthermore, generate the first voxel map according to the registration result of the line feature points.

[0085] Exemplarily, the generation process of the first voxel map is as follows: Divide the space of the first site into voxels of 1*1*1 meter. Traverse each line feature point in the registration result of the line feature points, associate the line feature point to a certain voxel, and perform update processing on the voxel. The update processing process is as follows:

[0086] (a1) Perform SVD calculation on all the line feature points associated with the voxel to obtain the eigenvalues λ1 > λ2 > λ3 and the corresponding eigenvectors v 1 , v 2 , v 3 . Calculate the line feature σ of the voxel using the above formula (1) 1D .

[0087] (b1) If the line feature σ of the voxel 1D is greater than or equal to the first threshold, update the voxel feature of the voxel according to all the line feature points associated with the voxel, and use and v 1 to represent the voxel feature of the voxel. In this case, there is no need to perform segmentation processing on the voxel. Among them, is the average value of all the feature points associated with the voxel.

[0088] (c1) If the line feature σ of the voxel 1D is less than the first threshold, perform octree segmentation processing on the voxel, that is, divide the voxel into 8 new voxels.

[0089] For each new voxel, repeat the above step (a1) until the line feature σ of the new voxel 1D is greater than or equal to the first threshold and there is no need to segment; or, until the segmentation times of the voxel are greater than the preset times and no longer segment.

[0090] S403: Generate the first surface voxel map based on the surface feature points in the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively.

[0091] In this embodiment, since the first surface voxel map is generated based on all frames of the first point cloud data, the first surface voxel map can be referred to as a global surface voxel map.

[0092] In a possible implementation, for each frame of the first point cloud data, adjust the surface feature points in the frame of the first point cloud data according to the first relative pose corresponding to the frame of the first point cloud data to obtain the adjusted surface feature points. Perform registration processing on the adjusted surface feature points in each frame of the first point cloud data to obtain the surface feature point registration result. Furthermore, generate the first surface voxel map according to the surface feature point registration result.

[0093] Exemplarily, the generation process of the first surface voxel map is as follows: Divide the space of the first site into voxels of 1*1*1 meters. Traverse each surface feature point in the surface feature point registration result, associate the surface feature point with a certain voxel, and perform update processing on the voxel. The update processing process is as follows:

[0094] (a2) Perform SVD calculation on all surface feature points associated with the voxel to obtain eigenvalues λ1 > λ2 > λ3 and corresponding eigenvectors v 1 , v 2 , v 3 . Calculate the surface feature σ of the voxel using the above formula (2) 2D .

[0095] (b2) If the surface feature σ of the voxel 2D is greater than or equal to the second threshold, update the voxel feature of the voxel according to all surface feature points associated with the voxel, and use and v 3 to represent the voxel feature of the voxel. In this case, there is no need to perform segmentation processing on the voxel. Among them, is the average value of all feature points associated with the voxel.

[0096] (c2) If the surface feature σ of the voxel 2D is less than the second threshold, perform octree segmentation processing on the voxel, that is, divide the voxel into 8 new voxels.

[0097] For each new voxel, repeat the above step (a2) until the surface feature σ of the new voxel 2D is greater than or equal to the second threshold and there is no need to segment; or, until the segmentation times of the voxel are greater than the preset times and no longer segment.

[0098] S404: Determine the first point-line constraint relationship based on the line feature points in the multi-frame first point cloud data and the first voxel map.

[0099] In this embodiment, for each voxel in the first voxel map, the first point-line constraint relationship can be constructed according to the distances between the line feature points associated with the voxel and the voxel feature of the voxel.

[0100] Exemplarily, the first point-line constraint relationship can be shown as the following formula (3):

[0101]

[0102] Wherein, in the above formula (3), q is the line feature of a certain voxel in the first voxel map, p i is the i-th line feature point associated with the voxel, N is the number of line feature points associated with the voxel, n is the line direction corresponding to the voxel, T is the pose, q * 、n * 、T * respectively represent the optimized values of q, n, and T. A is the covariance matrix, u 3 is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix A, is the average value of all feature points associated with the voxel.

[0103] S405: Determine the first point-plane constraint relationship based on the plane feature points in the multi-frame first point cloud data and the first plane voxel map.

[0104] In this embodiment, for each voxel in the first plane voxel map, the first point-plane constraint relationship can be constructed according to the distances between the plane feature points associated with the voxel and the voxel feature of the voxel.

[0105] Exemplarily, the first point-plane constraint relationship can be shown as the following formula (4):

[0106]

[0107] Wherein, in the above formula (4), q is the plane feature of a certain voxel in the first plane voxel map, p i is the i-th line feature point associated with the voxel, N is the number of line feature points associated with the voxel, n is the normal vector of the voxel plane feature, T is the pose, q * 、n * 、T * respectively represent the optimized values of q, n, and T. A is the covariance matrix, Tr(A) represents the rank of the covariance matrix A, u 1 is the eigenvector corresponding to the maximum eigenvalue of the covariance matrix A, is the average value of all feature points associated with the voxel, and I is the identity matrix.

[0108] S406: Update the first relative pose corresponding to each of the multiple frames of first point cloud data according to the first point-line constraint relationship and the first point-plane constraint relationship to obtain the second relative pose corresponding to each of the multiple frames of first point cloud data.

[0109] Exemplarily, linear optimization can be performed according to the first point-line constraint relationship shown in the above formula (3) and the first point-plane constraint relationship shown in the above formula (4) to obtain the second relative pose corresponding to each of the multiple frames of first point cloud data.

[0110] In this embodiment, by generating a first line voxel map and a first plane voxel map based on all frames of first point cloud data, and then updating the first relative pose corresponding to each of the multiple frames of first point cloud data according to the first line voxel map and the first plane voxel map to obtain the second relative pose corresponding to each of the multiple frames of first point cloud data, global optimization of the phase pose corresponding to the multiple frames of first point cloud data is realized, and the accuracy of the phase pose corresponding to the multiple frames of first point cloud data is improved.

[0111] Based on any of the above embodiments, the technical solution of the present disclosure will be described in more detail below in combination with a more specific embodiment.

[0112] Figure 5 It is a schematic flowchart of another method for processing high-precision map point cloud data provided by an embodiment of the present disclosure. As Figure 5 shown, the method of this embodiment includes:

[0113] S501: Obtain multiple frames of first point cloud data collected during the movement of the acquisition device in the first place.

[0114] It should be understood that the specific implementation manner of S501 is similar to that of Figure 2 S201, and details are not described here.

[0115] S502: Determine the third relative pose corresponding to each frame of first point cloud data according to the multiple frames of first point cloud data, where the third relative pose is used to indicate the pose change information of the acquisition device when collecting this frame of first point cloud data relative to the previous frame of first point cloud data when collecting this frame of first point cloud data.

[0116] In this embodiment, an odometry technique can be used to determine the third relative pose corresponding to each frame of first point cloud data. It should be noted that there are various odometry techniques, and this embodiment does not limit this. Here, an odometry technique based on scan to map will be taken as an example for description.

[0117] Exemplarily, the third relative pose corresponding to the first point cloud data of the i-th frame can be determined based on the first point cloud data of the i-th frame and the second voxel map, where the second voxel map is generated based on a preset number of first point cloud data before the i-th frame, and i takes 1, 2, 3, …, N in sequence, and N is the total number of frames of the multi-frame first point cloud data.

[0118] For example, assuming that the preset number is 20, when determining the third relative pose corresponding to the first point cloud data of the i-th frame, the second voxel map generated from the 20 frames of first point cloud data before the i-th frame can be obtained first, and the third relative pose corresponding to the first point cloud data of the i-th frame can be determined by registering the first point cloud data of the i-th frame with the second voxel map.

[0119] The above-mentioned second voxel map is generated based on a preset number of frames of first point cloud data before the i-th frame, rather than based on all frames of first point cloud data. Therefore, the second voxel map can also be called a local voxel map. By registering each frame of first point cloud data with the local voxel map to determine the third relative pose corresponding to the first point cloud data of this frame, the processing efficiency can be improved.

[0120] In a possible implementation, the second voxel map includes: a second line voxel map and a second surface voxel map. The following registration method can be used to determine the third relative pose corresponding to the first point cloud data of the i-th frame:

[0121] (1) Determine the line feature points and surface feature points in the first point cloud data of the i-th frame based on the first point cloud data of the i-th frame.

[0122] It should be understood that the implementation method of step (1) can refer to S401 in the above embodiment, and will not be elaborated here.

[0123] (2) Determine the second point-line constraint relationship based on the line feature points in the first point cloud data of the i-th frame and the second line voxel map.

[0124] (3) Determine the second point-surface constraint relationship based on the surface feature points in the first point cloud data of the i-th frame and the second surface voxel map.

[0125] It should be understood that the implementation methods of steps (2) and (3) can refer to S404 and S405 in the above embodiment, and will not be elaborated here.

[0126] (4) Determine the third relative pose corresponding to the first point cloud data of the i-th frame based on the second point-line constraint relationship and the second point-surface constraint relationship.

[0127] Exemplarily, according to the above second point-line constraint relationship and second point-surface constraint relationship, linear optimization is performed to obtain the third relative pose corresponding to the first point cloud data of the i-th frame.

[0128] It can be understood that during the process of registering the i-th frame of the first point cloud data using the second voxel map, when the feature points associated with the voxels in the second voxel map change, the second voxel map needs to be updated. The process of updating the second voxel map is similar to the process of updating the first voxel map described in S402 in the above embodiment, and will not be elaborated here.

[0129] In some application scenarios, the first venue may include multiple layers of space. For example, the first venue is a multi-story shopping mall or a multi-story garage, etc. When the first venue includes multiple layers of space, a three-dimensional map needs to be constructed for each layer of space respectively. The following S503 is used to perform hierarchical processing on multiple frames of the first point cloud data to determine which frames are located on the first layer, which frames are located on the second layer, etc. The following S504 is used to optimize the third relative pose of the first point cloud data in each layer of space to obtain the first relative pose of the first point cloud data.

[0130] It should be understood that when the first venue is a single-layer space, S503 does not need to be executed, and S504 can be directly executed.

[0131] S503: According to the third relative poses respectively corresponding to the multiple frames of the first point cloud data, determine the first point cloud data corresponding to each layer of space in the first venue from the multiple frames of the first point cloud data.

[0132] Exemplarily, Figure 6 is a schematic diagram of the hierarchical processing of the first point cloud data provided by the embodiment of the present disclosure. Taking the first venue including two layers of space as an example, as Figure 6 shown, the black dots exemplify the third relative poses corresponding to each frame of the first point cloud data determined in S502. A demarcation line can be determined in the middle of the passage between the first layer of space and the second layer of space. In this way, the movement trajectory of the acquisition device is divided into several segments by the demarcation line. Those located above the demarcation line belong to the upper layer of space, and those located below the demarcation line belong to the lower layer of space.

[0133] S504: According to the first point cloud data corresponding to each layer of space, update the third relative poses respectively corresponding to the first point cloud data in this layer of space to obtain the first relative poses respectively corresponding to the first point cloud data in this layer of space.

[0134] In this embodiment, each layer of space can be used as the processing granularity to optimize the third relative poses corresponding to the first point cloud data within this layer of space, further improving the accuracy of the relative pose.

[0135] In a possible implementation manner, the following method can be used to optimize the third relative pose to the first relative pose:

[0136] (1) Determine at least one trajectory loop for the movement of the acquisition device in each layer of space based on the first point cloud data corresponding to each layer of space. Each trajectory loop includes the first point cloud data of the starting frame, the first point cloud data of the ending frame, and at least one first point cloud data of the intermediate frames.

[0137] In one example, a trajectory loop can be determined in the following manner: If there is the first point cloud data of the k-th frame and the first point cloud data of the (k + p)-th frame in the first point cloud data corresponding to this layer of space, then determine the first point cloud data of the k-th frame as the first point cloud data of the starting frame in the trajectory loop, determine the first point cloud data of the (k + p)-th frame as the first point cloud data of the ending frame in the trajectory loop, and determine the first point cloud data between the k-th frame and the (k + p)-th frame as the first point cloud data of the intermediate frames in the trajectory loop, where k and p are natural numbers. Among them, the distance between the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose corresponding to the first point cloud data of the (k + p)-th frame is less than or equal to the first threshold, and the difference between the acquisition time of the first point cloud data of the (k + p)-th frame and the acquisition time of the first point cloud data of the k-th frame is greater than or equal to the second threshold.

[0138] In another example, a trajectory loop can be determined in the following manner: If there is the first point cloud data of the k-th frame and the first point cloud data of the (k + p)-th frame in the first point cloud data corresponding to this layer of space, then determine the first point cloud data of the k-th frame as the first point cloud data of the starting frame in the trajectory loop, determine the first point cloud data of the (k + p)-th frame as the first point cloud data of the ending frame in the trajectory loop, and determine the first point cloud data between the k-th frame and the (k + p)-th frame as the first point cloud data of the intermediate frames in the trajectory loop, where k and p are natural numbers; among them, the distance between the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose corresponding to the first point cloud data of the (k + p)-th frame is less than or equal to the first threshold, and the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose corresponding to the first point cloud data of the (k + p)-th frame are located on the boundary line of this layer of space.

[0139] Optionally, before determining the trajectory loop, the third relative poses corresponding to multiple frames of first point cloud data can also be filtered. For example, if the distance between the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose between the first point cloud data of the i-th frame is less than the preset threshold, then the third relative pose corresponding to the first point cloud data of the k-th frame can be deleted. In this way, when determining the trajectory loop, the calculation amount can be reduced and the processing efficiency can be improved.

[0140] (2) Update the third relative pose corresponding to each piece of first point cloud data in each trajectory loop based on the first point cloud data of the starting frame and the first point cloud data of the ending frame in that trajectory loop, to obtain the first relative pose corresponding to each piece of first point cloud data in that trajectory loop.

[0141] Exemplarily, when performing relative pose update, an optimization method based on a pose graph can be adopted.

[0142] Optionally, when multiple trajectory loops are determined in step (1), registration processing can also be performed on the starting frame and the ending frame of each trajectory loop to obtain a registration score. If the registration score is less than a preset threshold, it indicates that this trajectory loop is not a real trajectory loop, and this trajectory loop can be deleted. If the registration score is greater than or equal to the preset threshold, it indicates that this trajectory loop is a real trajectory loop, and this trajectory loop is retained. In this way, only the retained trajectory loops need to be processed in step (2), which can improve the processing efficiency on the one hand and ensure the accuracy of the pose optimization result on the other hand.

[0143] S505: Generate a first voxel map based on the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively, and update the first relative poses corresponding to the multiple frames of first point cloud data according to the first voxel map, to obtain the second relative poses corresponding to the multiple frames of first point cloud data respectively.

[0144] It should be understood that the specific implementation manner of S505 can refer to Figure 3 S303 in Figure 4 or the detailed description of the embodiments shown in

[0145] S506: Obtain the global poses corresponding to multiple frames of second point cloud data, where the second point cloud data is collected during the movement of the acquisition device within a preset range near the entrance and / or exit of the first venue.

[0146] In the application scenario of this embodiment, there is GPS coverage within a preset range near the entrance and / or exit of the first venue. When the acquisition device moves within a preset range near the entrance and / or exit of the first venue, the second point cloud data is collected, and the global pose corresponding to the second point cloud data can be obtained by using GPS. Therefore, in this embodiment, the global poses corresponding to each frame of first point cloud data can be determined by using the global pose corresponding to the second point cloud data.

[0147] S507: Determine the global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively and the global poses corresponding to the multiple frames of second point cloud data.

[0148] In a possible implementation, the following method can be used to determine the global poses corresponding to multiple frames of first point cloud data: determine the first movement trajectory of the acquisition device according to the second relative poses corresponding to the multiple frames of first point cloud data, and determine the second movement trajectory of the acquisition device according to the global poses corresponding to the multiple frames of second point cloud data; perform registration processing on the first movement trajectory and the second movement trajectory to obtain a registration result, where the registration result is used to indicate the registration relationship between at least part of the first point cloud data and at least part of the second point cloud data; determine the global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data, the global poses corresponding to the multiple frames of second point cloud data, and the registration result.

[0149] Exemplarily, in the specific implementation process, several frames of first point cloud data close to the above time stamps can be found in each frame of first point cloud data according to the time stamps corresponding to each frame of second point cloud data. Interpolate the first movement trajectory according to the second relative poses corresponding to the selected several frames of first point cloud data, and interpolate the second movement trajectory according to the global poses corresponding to each frame of second point cloud data. Furthermore, by performing registration processing on the first movement trajectory and the second movement trajectory, the registration relationship between at least part of the first point cloud data and at least part of the second point cloud data can be obtained. In this way, based on this registration relationship, the global pose corresponding to each frame of first point cloud data can be determined.

[0150] S508: Perform three-dimensional reconstruction on the multiple frames of first point cloud data according to the global poses corresponding to the multiple frames of first point cloud data to obtain the three-dimensional map corresponding to the first site.

[0151] In this embodiment, after determining the global poses corresponding to each frame of first point cloud data, three-dimensional reconstruction can be performed on each frame of first point cloud data according to the global poses to obtain the three-dimensional map corresponding to the first site. The specific reconstruction method is similar to the three-dimensional reconstruction method of the outdoor scene and will not be elaborated here.

[0152] Figure 7 This is a schematic diagram of a processing process for high-precision map point cloud data provided by an embodiment of the present disclosure. As Figure 7 shown, the processing process of high-precision map point cloud data in this embodiment includes the following three processes:

[0153] Process 1: Determine the relative poses corresponding to each frame of first point cloud data according to multiple frames of first point cloud data.

[0154] Specifically, taking the first point cloud data of the i-th frame as an example, feature points are extracted from the first point cloud data of the i-th frame, and the feature points and the local voxel map are registered to obtain the relative pose of the first point cloud data of the i-th frame. Further, according to the registration result, the local voxel map is updated, and the relative pose corresponding to the first point cloud data is optimized based on the local voxel map. The optimization here adopts the optimization method based on bundle adjustment. In Process 1, the local voxel map is the second voxel map in the above embodiment. For example, it may include a second line voxel map and a second surface voxel map. Correspondingly, the feature points may include line feature points and surface feature points.

[0155] In this embodiment, Process 1 may correspond to Figure 5 S502 in the embodiment shown. The relative pose determined by Process 1 may correspond to Figure 5 the third relative pose in the embodiment shown.

[0156] Process 2: Optimize the relative poses corresponding to the first point cloud data of each frame.

[0157] Specifically, when the first site includes multiple layers of space, the relative poses corresponding to the first point cloud data of each frame are processed in layers to obtain the relative poses in each layer of space. Then, for each layer of space, a trajectory loop is determined, and then registration processing is performed according to the trajectory loop. According to the registration result and the pose graph, the relative poses are optimized.

[0158] In this embodiment, Process 2 may correspond to Figure 5 S503 - S504 in the embodiment shown. The relative poses optimized by Process 2 may correspond to Figure 5 the first relative pose in the embodiment shown.

[0159] Process 3: Globally optimize the relative poses corresponding to the first point cloud data of each frame and convert them into global poses.

[0160] Specifically, according to all the first point cloud data of each frame, a global voxel map is generated, and global optimization processing is performed on the relative poses corresponding to the first point cloud data of each frame based on the global voxel map. Further, using the second point cloud data with global poses (for example, the point cloud data collected within a preset range near the entrance and / or exit of the first site), and the registration relationship between the second point cloud data and the first point cloud data, the relative poses of the first point cloud data of each frame are converted into global poses. In Process 3, the global voxel map is the first voxel map in the above embodiment. For example, it may include a first line voxel map and a first surface voxel map.

[0161] In this embodiment, Process 3 may correspond to Figure 5S505 - S507 in the illustrated embodiment. Among them, the relative pose optimized using the global voxel map can correspond to Figure 5 the second relative pose in the illustrated embodiment, and then convert the second relative pose into a global pose.

[0162] Process 4: According to the global poses corresponding to each frame of the first point cloud data, perform three - dimensional reconstruction on each frame of the first point cloud data to obtain a three - dimensional map corresponding to the first site.

[0163] In this embodiment, Process 4 can correspond to Figure 5 S508 in the illustrated embodiment.

[0164] In this embodiment, in Process 1, by optimizing the relative pose corresponding to the first point cloud data using the local voxel map, the accuracy of the determined relative pose is improved. In Process 2, by performing hierarchical processing on the relative pose, a three - dimensional map can be reconstructed for each layer of space respectively, so that it can be used for mapping a site including multiple layers of space. Additionally, in Process 3, by globally optimizing the relative poses corresponding to each frame of the first point cloud data using the global voxel map, the cumulative error caused by the odometry method is avoided, the accuracy of the relative pose of the first point cloud data is improved, and further, the accuracy of the constructed three - dimensional map can be improved.

[0165] Figure 8 It is a schematic structural diagram of a processing device for high - precision map point cloud data provided by an embodiment of the present disclosure. The device in this embodiment can be in the form of software and / or hardware. As Figure 8 shown, the processing device 800 for high - precision map point cloud data provided in this embodiment includes: an acquisition module 801, a determination module 802, an update module 803, and a three - dimensional reconstruction module 804. Among them,

[0166] The acquisition module 801 is configured to acquire multiple frames of first point cloud data collected by the acquisition device during the movement in the first site;

[0167] The determination module 802 is configured to determine, according to the multiple frames of first point cloud data, a first relative pose corresponding to each frame of the first point cloud data, where the first relative pose is used to indicate the pose change information of the acquisition device when collecting this frame of the first point cloud data relative to the previous frame of the first point cloud data when collecting this frame of the first point cloud data;

[0168] The update module 803 is configured to generate a first voxel map according to the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively, and update the first relative poses corresponding to the multiple frames of first point cloud data according to the first voxel map to obtain second relative poses corresponding to the multiple frames of first point cloud data respectively;

[0169] A 3D reconstruction module 804, configured to perform 3D reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data, so as to obtain a 3D map corresponding to the first location.

[0170] In a possible implementation, the first voxel map includes: a first line voxel map and a first surface voxel map; the update module 803 includes:

[0171] A first determination unit, configured to determine line feature points and surface feature points in the multiple frames of first point cloud data;

[0172] A first generation unit, configured to generate the first line voxel map according to the line feature points in the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data;

[0173] A second generation unit, configured to generate the first surface voxel map according to the surface feature points in the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data.

[0174] In a possible implementation, the update module 803 further includes:

[0175] A second determination unit, configured to determine a first point-line constraint relationship according to the line feature points in the multiple frames of first point cloud data and the first line voxel map;

[0176] A third determination unit, configured to determine a first point-surface constraint relationship according to the surface feature points in the multiple frames of first point cloud data and the first surface voxel map;

[0177] A first update unit, configured to update the first relative poses corresponding to the multiple frames of first point cloud data according to the first point-line constraint relationship and the first point-surface constraint relationship, so as to obtain the second relative poses corresponding to the multiple frames of first point cloud data.

[0178] In a possible implementation, the 3D reconstruction module 804 includes:

[0179] An acquisition unit, configured to acquire the global poses corresponding to multiple frames of second point cloud data, where the second point cloud data is acquired during the movement of the acquisition device within a preset range near the entrance and / or exit of the first location;

[0180] A fourth determination unit, configured to determine the global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data and the global poses corresponding to the multiple frames of second point cloud data;

[0181] A 3D reconstruction unit, configured to perform 3D reconstruction on the multiple frames of first point cloud data according to the respective global poses corresponding to the multiple frames of first point cloud data, so as to obtain a 3D map corresponding to the first site.

[0182] In a possible implementation manner, the fourth determination unit includes:

[0183] A first determination subunit, configured to determine a first movement trajectory of the acquisition device according to the respective second relative poses corresponding to the multiple frames of first point cloud data, and determine a second movement trajectory of the acquisition device according to the respective global poses corresponding to the multiple frames of second point cloud data;

[0184] A registration subunit, configured to perform registration processing on the first movement trajectory and the second movement trajectory to obtain a registration result, where the registration result is used to indicate the registration relationship between at least part of the first point cloud data and at least part of the second point cloud data;

[0185] A second determination subunit, configured to determine the respective global poses corresponding to the multiple frames of first point cloud data according to the respective second relative poses corresponding to the multiple frames of first point cloud data, the respective global poses corresponding to the multiple frames of second point cloud data, and the registration result.

[0186] In a possible implementation manner, the first site includes multiple layers of space; the determination module 802 includes:

[0187] A fifth determination unit, configured to determine a third relative pose corresponding to each frame of first point cloud data according to the multiple frames of first point cloud data, where the third relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of first point cloud data relative to the previous frame of first point cloud data when acquiring this frame of first point cloud data;

[0188] A sixth determination unit, configured to respectively determine the first point cloud data corresponding to each layer of space of the first site from the multiple frames of first point cloud data according to the respective third relative poses corresponding to the multiple frames of first point cloud data;

[0189] A second update unit, configured to update the respective third relative poses corresponding to the first point cloud data in each layer of space according to the first point cloud data corresponding to each layer of space, so as to obtain the respective first relative poses corresponding to the first point cloud data in each layer of space.

[0190] In a possible implementation manner, the second update unit includes:

[0191] A third determination subunit, configured to determine at least one trajectory loop for the acquisition device to move in the space of each layer according to the first point cloud data corresponding to the space of each layer. Each trajectory loop includes first point cloud data of a starting frame, first point cloud data of an ending frame, and at least one intermediate frame of first point cloud data;

[0192] An update subunit, configured to update the third relative pose corresponding to each first point cloud data in the trajectory loop according to the first point cloud data of the starting frame and the first point cloud data of the ending frame in each trajectory loop, so as to obtain the first relative pose corresponding to each first point cloud data in the trajectory loop.

[0193] In a possible implementation, the third determination subunit is specifically configured to:

[0194] If there is k-th frame first point cloud data and (k + p)-th frame first point cloud data in the first point cloud data corresponding to the space of this layer, then determine the k-th frame first point cloud data as the first point cloud data of the starting frame in the trajectory loop, determine the (k + p)-th frame first point cloud data as the first point cloud data of the ending frame in the trajectory loop, and determine the first point cloud data between the k-th frame and the (k + p)-th frame as the intermediate frame first point cloud data in the trajectory loop, where k and p are natural numbers;

[0195] Wherein, the distance between the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data is less than or equal to a first threshold, and the difference between the acquisition time of the (k + p)-th frame first point cloud data and the acquisition time of the k-th frame first point cloud data is greater than or equal to a second threshold; or,

[0196] The distance between the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data is less than or equal to a first threshold, and the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data are located on the boundary line of the space of this layer.

[0197] In a possible implementation, the fifth determination unit is specifically configured to:

[0198] Determine the third relative pose corresponding to the i-th frame first point cloud data according to the i-th frame first point cloud data and the second voxel map; wherein, the second voxel map is generated according to a preset number of first point cloud data before the i-th frame, and i sequentially takes 1, 2, 3,..., N, and N is the total number of frames of the multi-frame first point cloud data.

[0199] In a possible implementation, the second voxel map includes: a second line voxel map and a second surface voxel map; the fifth determination unit is specifically configured to:

[0200] Determine the line feature points and plane feature points in the first point cloud data of the i-th frame according to the first point cloud data of the i-th frame;

[0201] Determine the second point-line constraint relationship according to the line feature points in the first point cloud data of the i-th frame and the second line voxel map;

[0202] Determine the second point-plane constraint relationship according to the plane feature points in the first point cloud data of the i-th frame and the second plane voxel map;

[0203] Determine the third relative pose corresponding to the first point cloud data of the i-th frame according to the second point-line constraint relationship and the second point-plane constraint relationship.

[0204] The high-precision map point cloud data processing device provided in this embodiment can be used to execute the high-precision map point cloud data processing method provided in any of the above method embodiments. The implementation principle and technical effect are similar and will not be elaborated here.

[0205] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0206] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0207] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, and at least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program to enable the electronic device to execute the solution provided in any of the above embodiments.

[0208] Figure 9 FIG. shows a schematic block diagram of an exemplary electronic device 900 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0209] As Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of device 900 can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0210] Multiple components in device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, optical disc, etc.; and a communication unit 909, such as a network card, modem, wireless communication transceiver, etc. The communication unit 909 allows device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0211] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 executes the various methods and processes described above, such as the method for processing high-precision map point cloud data. For example, in some embodiments, the method for processing high-precision map point cloud data can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method for processing high-precision map point cloud data described above can be executed. Alternatively, in other embodiments, the computing unit 901 can be configured to execute the method for processing high-precision map point cloud data in any other appropriate way (e.g., by means of firmware).

[0212] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0213] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0214] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0215] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0216] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0217] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0218] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is made herein.

[0219] The above specific embodiments do not constitute a limitation to the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for processing high-precision map point cloud data, including: Obtaining multiple frames of first point cloud data collected by a collection device during movement in a first location; According to the multiple frames of first point cloud data, determining a first relative pose corresponding to each frame of first point cloud data, where the first relative pose is used to indicate the pose change information of the collection device when collecting this frame of first point cloud data relative to the previous frame of first point cloud data when collecting this frame of first point cloud data; Determining line feature points and surface feature points in the multiple frames of first point cloud data; Generating a first line voxel map according to the line feature points in the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively; Generating a first surface voxel map according to the surface feature points in the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data respectively; Determining a first point-line constraint relationship according to the line feature points in the multiple frames of first point cloud data and the first line voxel map; Determining a first point-surface constraint relationship according to the surface feature points in the multiple frames of first point cloud data and the first surface voxel map; Updating the first relative poses corresponding to the multiple frames of first point cloud data according to the first point-line constraint relationship and the first point-surface constraint relationship to obtain second relative poses corresponding to the multiple frames of first point cloud data respectively; Performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively to obtain a three-dimensional map corresponding to the first location.

2. The method according to claim 1, wherein, Performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively to obtain a three-dimensional map corresponding to the first location, including: Obtaining global poses corresponding to multiple frames of second point cloud data, where the second point cloud data is collected by the collection device during movement within a preset range near the entrance and / or exit of the first location; Determining global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively and the global poses corresponding to the multiple frames of second point cloud data; Performing three-dimensional reconstruction on the multiple frames of first point cloud data according to the global poses corresponding to the multiple frames of first point cloud data respectively to obtain a three-dimensional map corresponding to the first location.

3. The method according to claim 2, wherein, Determining global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data respectively and the global poses corresponding to the multiple frames of second point cloud data, including: Determining a first movement trajectory of the collection device according to the second relative poses corresponding to the multiple frames of first point cloud data respectively, and determining a second movement trajectory of the collection device according to the global poses corresponding to the multiple frames of second point cloud data respectively; Performing registration processing on the first movement trajectory and the second movement trajectory to obtain a registration result, where the registration result is used to indicate the registration relationship between at least part of the first point cloud data and at least part of the second point cloud data; Determine the global pose corresponding to each frame of the multi-frame first point cloud data according to the second relative pose corresponding to each frame of the multi-frame first point cloud data, the global pose corresponding to each frame of the multi-frame second point cloud data, and the registration result.

4. The method according to any one of claims 1 to 3, wherein, the first site includes multiple layers of space; Determining the first relative pose corresponding to each frame of the first point cloud data according to the multi-frame first point cloud data includes: Determining the third relative pose corresponding to each frame of the first point cloud data according to the multi-frame first point cloud data, where the third relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of the first point cloud data relative to the previous frame of the first point cloud data acquired for this frame; Determining the first point cloud data corresponding to each layer of space in the first site from the multi-frame first point cloud data according to the third relative pose corresponding to each frame of the multi-frame first point cloud data; Updating the third relative pose corresponding to each frame of the first point cloud data in each layer of space according to the first point cloud data corresponding to each layer of space to obtain the first relative pose corresponding to each frame of the first point cloud data in each layer of space.

5. The method according to claim 4, wherein, Updating the third relative pose corresponding to each frame of the first point cloud data in each layer of space according to the first point cloud data corresponding to each layer of space to obtain the first relative pose corresponding to each frame of the first point cloud data in each layer of space includes: Determining at least one trajectory loop of the acquisition device moving in each layer of space according to the first point cloud data corresponding to each layer of space, and each trajectory loop includes a starting frame of the first point cloud data, an ending frame of the first point cloud data, and at least one intermediate frame of the first point cloud data; Updating the third relative pose corresponding to each frame of the first point cloud data in the trajectory loop according to the starting frame of the first point cloud data and the ending frame of the first point cloud data in each trajectory loop to obtain the first relative pose corresponding to each frame of the first point cloud data in the trajectory loop.

6. The method according to claim 5, wherein, Determining at least one trajectory loop of the acquisition device moving in each layer of space according to the first point cloud data corresponding to each layer of space includes: If there is a k-th frame of the first point cloud data and a (k + p)-th frame of the first point cloud data in the first point cloud data corresponding to each layer of space, then determining the k-th frame of the first point cloud data as the starting frame of the first point cloud data in the trajectory loop, determining the (k + p)-th frame of the first point cloud data as the ending frame of the first point cloud data in the trajectory loop, and determining the first point cloud data between the k-th frame and the (k + p)-th frame as the intermediate frame of the first point cloud data in the trajectory loop, where k and p are natural numbers; wherein the distance between the third relative pose corresponding to the k-th frame of the first point cloud data and the third relative pose corresponding to the (k + p)-th frame of the first point cloud data is less than or equal to a first threshold, and the difference between the acquisition time of the (k + p)-th frame of the first point cloud data and the acquisition time of the k-th frame of the first point cloud data is greater than or equal to a second threshold; or, The distance between the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose corresponding to the first point cloud data of the (k + p)-th frame is less than or equal to the first threshold, and the third relative pose corresponding to the first point cloud data of the k-th frame and the third relative pose corresponding to the first point cloud data of the (k + p)-th frame are located on the boundary line of this layer of space.

7. The method according to claim 4, wherein, determining the third relative pose corresponding to each frame of the first point cloud data according to the multi-frame first point cloud data includes: determining the third relative pose corresponding to the i-th frame of the first point cloud data according to the i-th frame of the first point cloud data and the second voxel map; wherein, the second voxel map is generated according to a preset number of first point cloud data before the i-th frame, and i sequentially takes 1, 2, 3,..., N, and N is the total number of frames of the multi-frame first point cloud data.

8. The method according to claim 7, wherein, the second voxel map includes: a second line voxel map and a second surface voxel map; determining the third relative pose corresponding to the i-th frame of the first point cloud data according to the i-th frame of the first point cloud data and the second voxel map includes: determining the line feature points and surface feature points in the i-th frame of the first point cloud data according to the i-th frame of the first point cloud data; determining a second point-line constraint relationship according to the line feature points in the i-th frame of the first point cloud data and the second line voxel map; determining a second point-surface constraint relationship according to the surface feature points in the i-th frame of the first point cloud data and the second surface voxel map; determining the third relative pose corresponding to the i-th frame of the first point cloud data according to the second point-line constraint relationship and the second point-surface constraint relationship.

9. A processing device for high-precision map point cloud data, comprising: an acquisition module, configured to acquire multi-frame first point cloud data collected by an acquisition device during movement in a first place; a determination module, configured to determine the first relative pose corresponding to each frame of the first point cloud data according to the multi-frame first point cloud data, where the first relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of the first point cloud data relative to the previous frame of the first point cloud data when acquiring this frame of the first point cloud data; an update module, configured to generate a first voxel map according to the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively, and update the first relative poses corresponding to the multi-frame first point cloud data according to the first voxel map to obtain the second relative poses corresponding to the multi-frame first point cloud data respectively; a three-dimensional reconstruction module, configured to perform three-dimensional reconstruction on the multi-frame first point cloud data according to the second relative poses corresponding to the multi-frame first point cloud data respectively to obtain a three-dimensional map corresponding to the first place; wherein, the first voxel map includes: a first line voxel map and a first surface voxel map; the update module includes: a first determination unit, configured to determine the line feature points and surface feature points in the multi-frame first point cloud data; a first generation unit, configured to generate the first line voxel map according to the line feature points in the multi-frame first point cloud data and the first relative poses corresponding to the multi-frame first point cloud data respectively; A second generation unit, configured to generate the first surface voxel map according to the surface feature points in the multiple frames of first point cloud data and the first relative poses corresponding to the multiple frames of first point cloud data; Wherein, the update module further includes: A second determination unit, configured to determine a first point-line constraint relationship according to the line feature points in the multiple frames of first point cloud data and the first line voxel map; A third determination unit, configured to determine a first point-surface constraint relationship according to the surface feature points in the multiple frames of first point cloud data and the first surface voxel map; A first update unit, configured to update the first relative poses corresponding to the multiple frames of first point cloud data according to the first point-line constraint relationship and the first point-surface constraint relationship, so as to obtain the second relative poses corresponding to the multiple frames of first point cloud data.

10. The apparatus according to claim 9, Wherein, The three-dimensional reconstruction module includes: An acquisition unit, configured to acquire the global poses corresponding to multiple frames of second point cloud data, where the second point cloud data is acquired during the movement of the acquisition device within a preset range near the entrance and / or exit of the first location; A fourth determination unit, configured to determine the global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data and the global poses corresponding to the multiple frames of second point cloud data; A three-dimensional reconstruction unit, configured to perform three-dimensional reconstruction on the multiple frames of first point cloud data according to the global poses corresponding to the multiple frames of first point cloud data, so as to obtain a three-dimensional map corresponding to the first location.

11. The apparatus according to claim 10, Wherein, The fourth determination unit includes: A first determination subunit, configured to determine a first movement trajectory of the acquisition device according to the second relative poses corresponding to the multiple frames of first point cloud data, and determine a second movement trajectory of the acquisition device according to the global poses corresponding to the multiple frames of second point cloud data; A registration subunit, configured to perform registration processing on the first movement trajectory and the second movement trajectory to obtain a registration result, where the registration result is used to indicate the registration relationship between at least part of the first point cloud data and at least part of the second point cloud data; A second determination subunit, configured to determine the global poses corresponding to the multiple frames of first point cloud data according to the second relative poses corresponding to the multiple frames of first point cloud data, the global poses corresponding to the multiple frames of second point cloud data, and the registration result.

12. The apparatus according to any one of claims 9 to 11, Wherein, The first location includes multiple floors of space; The determination module includes: A fifth determination unit, configured to determine a third relative pose corresponding to each frame of first point cloud data according to the multiple frames of first point cloud data, where the third relative pose is used to indicate the pose change information of the acquisition device when acquiring this frame of first point cloud data relative to the previous frame of first point cloud data acquired by the acquisition device; A sixth determination unit, configured to respectively determine, from the multiple frames of first point cloud data, the first point cloud data corresponding to each layer of space of the first site according to the third relative poses corresponding to the multiple frames of first point cloud data; A second update unit, configured to update the third relative poses corresponding to the first point cloud data in each layer of space according to the first point cloud data corresponding to each layer of space, to obtain the first relative poses corresponding to the first point cloud data in each layer of space.

13. The apparatus according to claim 12, wherein, the second update unit includes: A third determination subunit, configured to determine, according to the first point cloud data corresponding to each layer of space, at least one trajectory loop for the acquisition device to move in this layer of space, and each trajectory loop includes a starting frame first point cloud data, an ending frame first point cloud data, and at least one intermediate frame first point cloud data; An update subunit, configured to update the third relative poses corresponding to the first point cloud data in this trajectory loop according to the starting frame first point cloud data and the ending frame first point cloud data in each trajectory loop, to obtain the first relative poses corresponding to the first point cloud data in this trajectory loop.

14. The apparatus according to claim 13, wherein, the third determination subunit is specifically configured to: If there is a k-th frame first point cloud data and a (k + p)-th frame first point cloud data in the first point cloud data corresponding to this layer of space, then determine the k-th frame first point cloud data as the starting frame first point cloud data in this trajectory loop, determine the (k + p)-th frame first point cloud data as the ending frame first point cloud data in this trajectory loop, and determine the first point cloud data between the k-th frame and the (k + p)-th frame as the intermediate frame first point cloud data in this trajectory loop, where k and p are natural numbers; wherein, the distance between the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data is less than or equal to a first threshold, and the difference between the acquisition time of the (k + p)-th frame first point cloud data and the acquisition time of the k-th frame first point cloud data is greater than or equal to a second threshold; or, the distance between the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data is less than or equal to a first threshold, and the third relative pose corresponding to the k-th frame first point cloud data and the third relative pose corresponding to the (k + p)-th frame first point cloud data are located on the boundary line of this layer of space.

15. The apparatus according to any one of claim 12, wherein, the fifth determination unit is specifically configured to: Determine the third relative pose corresponding to the i-th frame first point cloud data according to the i-th frame first point cloud data and the second voxel map; wherein, the second voxel map is generated according to a preset number of first point cloud data before the i-th frame, and i sequentially takes 1, 2, 3,..., N, and N is the total number of frames of the multiple frames of first point cloud data.

16. The apparatus according to claim 15, wherein, the second voxel map includes: a second line voxel map and a second surface voxel map; the fifth determination unit is specifically configured to: Determine line feature points and surface feature points in the first point cloud data of the i-th frame according to the first point cloud data of the i-th frame; Determine a second point-line constraint relationship according to the line feature points in the first point cloud data of the i-th frame and the second line voxel map; Determine a second point-surface constraint relationship according to the surface feature points in the first point cloud data of the i-th frame and the second surface voxel map; Determine a third relative pose corresponding to the first point cloud data of the i-th frame according to the second point-line constraint relationship and the second point-surface constraint relationship.

17. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images