Point cloud map generation method, apparatus, electronic device and storage medium

Abstract

This disclosure relates to a method, apparatus, electronic device, and storage medium for generating point cloud maps. The method includes: generating a first point cloud map based on point cloud data to be processed; acquiring a target feature region in the first point cloud map; determining an error in the target feature region based on its region type and the pose information of at least a portion of the point clouds contained within it; optimizing the pose information of the target frame point clouds in the target feature region using the error; and generating a second point cloud map based on the pose-optimized point cloud data to be processed. In this embodiment, because pose optimization is performed on point clouds in the point cloud data to be processed where the confidence level of the pose information is less than a confidence threshold, the accuracy of the second point cloud map generated based on the optimized point cloud data is higher.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of point cloud, and particularly relates to a point cloud map generation method and device, an electronic device and a storage medium. BACKGROUND

[0002] A high-precision point cloud map is mainly generated by point cloud data, which is collected by a vehicle-mounted laser radar and the like. For example, when a vehicle is running, the pose information of the vehicle at each time is recorded by combining an inertial navigation system, and the point cloud data at each time is collected by a laser radar, and then the collected point cloud data is spliced based on the above-posed information, thereby generating a high-precision point cloud map.

[0003] However, when the pose information deviates greatly, it is difficult to accurately splice the collected point cloud data, thereby resulting in a poor precision of the constructed point cloud map. SUMMARY

[0004] The present disclosure provides a point cloud map generation method, device, electronic device and storage medium.

[0005] According to a first aspect of the present disclosure, a point cloud map generation method is provided, and the method comprises:

[0006] generating a first point cloud map based on to-be-processed point cloud data;

[0007] obtaining a target feature region in the first point cloud map; the first point cloud map is pre-divided into a plurality of feature regions according to feature attributes, and different feature regions correspond to different region types;

[0008] determining an error of the target feature region based on a region type of the target feature region and pose information of at least part of point clouds contained in the target feature region;

[0009] optimizing pose information of a target frame point cloud in the target feature region by the error of the target feature region; the confidence of the pose information of the target frame point cloud is less than a confidence threshold;

[0010] generating a second point cloud map based on the to-be-processed point cloud data after pose optimization.

[0011] According to a second aspect of the present disclosure, a point cloud map generation device is provided, and the device comprises:

[0012] a first map generation module, configured to generate a first point cloud map based on to-be-processed point cloud data;

[0013] The feature region acquisition module is used to acquire target feature regions in the first point cloud map; the first point cloud map is pre-divided into multiple feature regions according to feature attributes, and different feature regions have corresponding different region types.

[0014] An error acquisition module is used to determine the error of the target feature region based on the region type of the target feature region and the pose information of at least a portion of the point cloud contained in the target feature region.

[0015] The pose information optimization module is used to optimize the pose information of the target frame point cloud in the target feature region by means of the error in the target feature region; the confidence level of the pose information of the target frame point cloud is less than the confidence level threshold.

[0016] The second point cloud map generation module is used to generate a second point cloud map based on the pose-optimized point cloud data to be processed.

[0017] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0018] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0019] The point cloud map generation method, apparatus, electronic device, and storage medium provided in this disclosure generate a first point cloud map based on point cloud data to be processed. Based on the region type of the target feature region and the pose information of at least a portion of the point clouds contained within the target feature region, the error of the target feature region is determined. The pose information of the target frame point clouds in the target feature region with a confidence level less than a confidence threshold is optimized using the error of the target feature region. This optimizes the pose information of point clouds in the point cloud data to be processed with a confidence level less than the confidence threshold, and a second point cloud map is generated based on the optimized point cloud data. In this disclosure, because the pose information of point clouds in the point cloud data to be processed with a confidence level less than the confidence threshold is optimized, the accuracy of the second point cloud map generated based on the optimized point cloud data is higher, effectively avoiding the impact of inaccurate point cloud pose on the accuracy of the generated point cloud map. Attached Figure Description

[0020] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating point cloud map generation provided as an exemplary embodiment of this disclosure;

[0022] Figure 2 A flowchart for generating a point cloud map provided as another exemplary embodiment of this disclosure;

[0023] Figure 3 for Figure 1 Flowchart of step S140;

[0024] Figure 4 A schematic block diagram of the functional modules of a point cloud map generation apparatus provided in an exemplary embodiment of the present disclosure;

[0025] Figure 5 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0026] Figure 6 A block diagram of a computer system provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] Current high-precision point cloud map construction relies on acquisition devices such as LiDAR to obtain real-time point cloud data, as well as various positioning sensors and accurate and reliable calibration parameters. The principle is to project point cloud data from a local coordinate system onto a navigation system through projection calculations, providing input for subsequent high-precision vector maps.

[0033] However, when the pose information of the acquisition device deviates significantly, it becomes difficult to accurately stitch the acquired point cloud data, resulting in a poorly accurate point cloud map. In such cases, manual adjustment of the point cloud map is generally required, which is time-consuming, inefficient, and difficult to guarantee accuracy.

[0034] Therefore, in order to solve the above problems, the embodiments of this disclosure will optimize the point cloud data with problematic pose information, so that the accuracy of the generated point cloud map can be avoided due to the problem of pose information.

[0035] In the embodiments provided in this disclosure, reference station data, raw receiver data, ephemeris data, IMU (Inertial Measurement Unit) data, and LiDAR data are received. The received data is processed to obtain complete pose information and confidence levels. The confidence level characterizes the degree of error in the pose information of the point cloud data; point cloud data with poor confidence levels are the areas that need to be optimized subsequently. Pose information refers to the position and orientation of point cloud data acquisition devices such as LiDAR. Position can be coordinates in the device's coordinate system, and orientation can be information such as the orientation of the point cloud data acquisition device.

[0036] Specifically, the data is first processed by the reference station data, raw receiver data and ephemeris data. The positioning data of the acquisition device is solved by the PPK (Post Processed Kinematic) algorithm to obtain the low-frequency positioning solution of the receiver post-processing, which includes the pose of 6 degrees of freedom and the corresponding confidence.

[0037] Then, the SINS (Strapdown inertial navigation system) algorithm is used to process the IMU data to obtain the high-frequency positioning solution of the IMU, which specifically includes the pose of 6 degrees of freedom and the corresponding confidence level.

[0038] Finally, a Kalman filter can be used to fuse the high-frequency estimation solution of the IMU's pose with the low-frequency localization solution processed by the receiver to obtain the final fused localization solution, which specifically includes the 6-DOF pose and corresponding confidence scores. In the embodiment, a loosely coupled approach can be used for fusion to obtain the point cloud data to be processed and the confidence scores of the pose information corresponding to each frame of the point cloud.

[0039] In this embodiment, projection calculation is performed on a single frame of point cloud in the point cloud to be processed. The nearest combined navigation post-processing positioning solution is found for the timestamp in each frame of point cloud. Then, projection calculation is performed to obtain the point cloud under the navigation system. In this embodiment, a Cartesian coordinate system with a certain point as the origin can be customized. According to the timestamp generated by each frame of point cloud, the pose information of each frame of point cloud and the confidence level of the corresponding pose information are found.

[0040] In this embodiment, feature extraction is performed on each frame of the point cloud map. For example, features such as surfaces, lines, cubes, spheres, and cubic shapes in each frame of the point cloud are extracted through curvature calculation and normal vector calculation. The feature points in each frame of the point cloud are then assigned values ​​according to their attribute to characterize the feature attributes of each feature point.

[0041] A point cloud map is obtained by accumulating feature points from multiple consecutive frames of point clouds, and then the point cloud map is divided into multi-level regions. First, the feature attributes of the feature points within a larger region can be identified. If a region contains more than one feature attribute, further subdivision is performed until the finest region is reached, ensuring that all subdivided regions have the same feature attribute category. This completes the feature region division, meaning the subdivided regions contain only planar, linear, or spherical features. Therefore, the identified features can include planar, linear, or spherical features. The division method can employ spatial region partitioning, such as quadtrees or octrees, by continuously dividing the spatial region until it cannot be further divided. Alternatively, the size of the spatial region can be set to minimize the subdivision.

[0042] When a new feature point cloud frame arrives, we can first search for the nearest feature region. If no such region is found, a new feature region is constructed. For example, based on the feature attributes corresponding to the feature information of the previous few frame point clouds, if the feature belongs to a planar feature, then when the next frame point cloud is added, based on the feature attributes of the feature information of this frame point cloud, if it also belongs to a planar feature, then this frame point cloud is assigned to the planar region of the previous few frames; otherwise, if the feature attributes of the feature information corresponding to this frame point cloud do not belong to a planar feature, such as belonging to a sphere, then this frame point cloud is assigned to a newly constructed spherical region, so that when the point cloud in subsequent frames also belongs to this spherical region, it can be assigned to that spherical region as well.

[0043] Because feature regions occupy very little memory and can be persisted in memory, they can be effectively adjusted for spatially overlapping regions with large time intervals between single acquisitions when optimizing point clouds on a large scale. By dividing the feature regions according to the features in the point cloud, subsequent multi-frame point clouds, even if not adjacent to the first frame, can be processed together within the same feature region to eliminate errors. In other words, feature attributes can be identified in advance, and adjustment can be performed on multiple frames of point clouds belonging to the same feature region to eliminate errors in the point cloud frames.

[0044] For example, if the motion trajectory of the point cloud data acquisition device's LiDAR is a large loop, then the LiDAR scanning areas at the initial and final moments will have significant overlap. In this case, it is necessary to adjust the point cloud data from multiple frames in the overlapping area to optimize its pose information. If the LiDAR's motion trajectory is an arc shape, then there will be a large amount of overlap between two adjacent flight paths, and these overlapping areas also need to be effectively adjusted.

[0045] In the embodiments provided in this disclosure, feature extraction is performed on each frame of the point cloud map, and feature regions are divided based on the attribute information of their feature points. For example, multiple frames of point clouds belonging to planar features are divided into planar feature regions, and multiple frames of point clouds belonging to spherical features are divided into spherical feature regions, and so on. The embodiments establish corresponding error models for each different feature region. For example, for different feature regions, the constructed target optimization equations are: optimization equations for the thickness of the plane, the point error of the line, the point error of the sphere, etc. By solving the simultaneous equations, the fine calibration parameters are obtained. In the embodiments, the corresponding target optimization equations are used as error models.

[0046] The embodiment uses a plane as an example for illustration. In a three-dimensional coordinate system, the plane equation is:

[0047] Ax + By + Cd + d = 0 (1)

[0048] The center point of the plane is P0, and the normal vector is n(A, B, C).

[0049] The error model for planar feature regions is as follows:

[0050]

[0051] in, The error is for the target feature region; For coordinate transformation from the device coordinate system to the combined inertial navigation system, For coordinate transformation from the combined inertial navigation coordinate system to the target navigation coordinate system, P LiLet P0 be the coordinates of any 3D point in the target frame point cloud in the device coordinate system, and P0 be the coordinates of the center position of the plane in the target feature region in the target navigation coordinate system. T This represents the transpose of the normal vector.

[0052] In the embodiments provided in this disclosure, if the planar feature region contains multiple planes, the three-dimensional points on these multiple planes are transformed to the IMU coordinate system, and then to the world coordinate system. Subtracting the center point of the corresponding plane from each three-dimensional point yields a vector. This vector is then multiplied by n. T By performing a dot product operation, we can determine the distance between a 3D point and a surface. Then, we accumulate the distances. For example, if there are M points on the same surface, we divide the M points by M and average them. We then accumulate all the surfaces, resulting in N surfaces. After accumulating all the surfaces, we calculate the average again. This gives us the error of the planar feature region, i.e., the error of the planar feature region.

[0053] In the embodiment, after obtaining the error of the planar feature region, the error can be used to optimize the pose information of the target frame point cloud whose pose information confidence is lower than the confidence threshold. That is, by adjusting the pose information of the target frame point cloud, the distance from the three-dimensional point in the target frame point cloud to the corresponding plane is less than the error. In this way, all three-dimensional points in the planar feature region are pressed onto the corresponding plane as much as possible to ensure that the thickness of the plane is not too large.

[0054] In the embodiment, when optimizing the pose information of a target frame point cloud with a confidence level lower than the confidence level threshold, the confidence level of the initial solution of the single frame point cloud is used to set the threshold of the optimization degree. For example, if the confidence level threshold is 10cm, the optimization degree cannot exceed 10cm.

[0055] The embodiments provided in this disclosure, after performing pose optimization on the point cloud data to be processed, can generate a point cloud map based on the optimized point cloud data. This improves the accuracy of the generated point cloud map by optimizing the pose information in the point cloud data. Furthermore, a high-precision point cloud map is obtained through voxelization filtering, and then the point cloud map is tiled, allowing it to be uploaded to a database for subsequent use.

[0056] Therefore, based on the above embodiments, this disclosure provides a point cloud map generation method, such as... Figure 1 As shown, the method may include the following steps:

[0057] In step S110, a first point cloud map is generated based on the point cloud data to be processed.

[0058] In the embodiments provided in this disclosure, data processing can be performed on reference station data, raw receiver data, and ephemeris data as described above, along with processing of IMU data. The processed data from both methods are then fused to obtain the point cloud data to be processed and the confidence level of the pose information corresponding to each frame of the point cloud. Furthermore, the embodiments may also utilize a depth camera combined with IMU data to obtain the point cloud data to be processed, etc., and the embodiments are not limited to these methods. A first point cloud map is then generated based on the point cloud data to be processed.

[0059] Since the point cloud map generated based on the point cloud data to be processed may have insufficient accuracy due to positioning problems, thus affecting the quality of the point cloud map, it is necessary to optimize the point cloud data to be processed in order to obtain a more accurate point cloud map.

[0060] In step S120, the target feature region in the first point cloud map is obtained.

[0061] The first point cloud map is pre-divided into multiple feature regions based on feature attributes, and different feature regions have different corresponding region types.

[0062] In this embodiment, feature information of each frame of point cloud is obtained by extracting features from each frame of point cloud in the first point cloud map. Based on the feature attributes corresponding to the feature information in each frame of point cloud, the points are divided into corresponding feature regions, thus obtaining multiple feature regions. These multiple feature regions can correspond to different region types, such as planes, spheres, or lines.

[0063] In this embodiment, feature extraction can begin with the first frame of the first point cloud map. Based on the feature attributes determined by the feature information of the first frame, the type of feature region to which the first frame point cloud belongs can be determined. For example, when dividing the first frame point cloud into a planar feature region, in subsequent adjacent frame point clouds, the feature attributes determined by their feature information are also used to determine whether they can be classified into the planar feature region where the first frame point cloud is located. If not, a new feature region is constructed. In this way, all frame point clouds in the first point cloud map are classified into their corresponding feature regions.

[0064] In step S130, the error of the target feature region is determined based on the region type of the target feature region and the pose information of at least a portion of the point cloud contained in the target feature region.

[0065] In the embodiments provided in this disclosure, the first point cloud map is divided into multiple feature regions, and the error of the corresponding feature region is determined according to its region type.

[0066] In this embodiment, the error of the target feature region can be determined based on the region type of the target feature region and the pose information of at least a portion of the point clouds contained within the target feature region. For example, the error of the target feature region can be determined by the region type of the target feature region and the pose information of all the point clouds contained within the target feature region, or it can be determined by the region type of the target feature region and the pose information of a portion of the point clouds contained within the target feature region. When determining the error of the target feature region by the region type of the target feature region and the pose information of a portion of the point clouds contained within the target feature region, point clouds with a pose information confidence level lower than a confidence threshold can be excluded, thus obtaining a more accurate error for the feature region. Specifically, it can be determined whether to select the pose information of a portion or all of the point clouds contained within the target feature region, depending on the specific circumstances.

[0067] In this embodiment, the error of the target feature region varies depending on its region type. For example, if it is a planar feature region, the error is calculated based on the pose information of at least a portion of the point cloud in the planar feature region, determining the minimum average distance from each of the three-dimensional points contained in that portion of the point cloud to its respective plane, i.e., optimizing the thickness of the plane; if it is a spherical feature region, the error of the target feature region is the point error of the sphere; if it is a line feature region, the error of the target feature region is the point error of the line.

[0068] If the target region is a straight line, the points in the point cloud of the target feature region can be fitted to a straight line, and the minimum average distance from at least a portion of the points in the point cloud of the target feature region to the straight line can be obtained, with the minimum average distance to the straight line used as the error. Similarly, if the target region is a sphere, the points in the point cloud of the target feature region can be fitted to a sphere, and the minimum average distance from at least a portion of the points in the point cloud of the target feature region to the sphere can be obtained, with the minimum average distance to the sphere used as the error.

[0069] In step S140, the pose information of the target frame point cloud in the target feature region is optimized by the error of the target feature region.

[0070] Among them, the confidence level of the pose information of the target frame point cloud is less than the confidence level threshold.

[0071] For example, when the target feature region is a planar feature region, the error of the target feature region is the minimum average distance. That is, based on the pose information of at least a portion of the point cloud in the planar feature region, the minimum average distance from each 3D point contained in that portion of the point cloud to its respective plane is calculated. This is so that the pose of the point cloud with a pose information confidence level less than a confidence threshold can be optimized using this minimum average distance, ensuring that the distance from the 3D points in the optimized point cloud to the plane is not greater than the minimum average distance.

[0072] When the target feature region is a line feature region, a fitted straight line can be obtained by fitting the 3D points contained in the line feature region. The fitted straight line can be one or several lines. The error of the target feature region is the average minimum distance from at least a portion of the 3D points in the target feature region to the corresponding straight line, and this average minimum distance is used as the error. This error is used to optimize the pose of point clouds with pose information confidence levels less than a confidence threshold, ensuring that the distance from the 3D points in the optimized point cloud to the straight line is no greater than this minimum average distance, i.e., no greater than the error.

[0073] When the target feature region is a spherical feature region, we can first find the center position of the sphere and the normal vector of the sphere. We can then calculate the minimum average distance from the 3D points contained in at least a portion of the point cloud in the target feature region to the sphere as an error. This error can be used to optimize the pose of the point cloud whose pose information confidence is less than the confidence threshold, so that the distance from the 3D points in the optimized point cloud to the sphere is not greater than the minimum average distance, i.e., not greater than the error.

[0074] In step S150, a second point cloud map is generated based on the pose-optimized point cloud data to be processed.

[0075] The point cloud data to be processed after pose optimization is obtained by posing optimization of the point cloud data to be processed from the point cloud data to be processed, and the confidence of the pose information of the point cloud to be optimized is less than the confidence threshold.

[0076] In this embodiment, optimized point cloud data can be obtained by optimizing the poses of point clouds whose pose information confidence is less than a confidence threshold in the point cloud data to be processed, and a second point cloud map with higher accuracy can be generated based on the optimized point cloud data.

[0077] The point cloud map generation method provided in this disclosure generates a first point cloud map based on point cloud data to be processed. It determines the error of the target feature region based on the region type of the target feature region and the pose information of at least a portion of the point clouds contained within the target feature region. The method then optimizes the pose information of target frame point clouds in the target feature region whose confidence level is less than a confidence threshold using the error. This optimizes the pose information of point clouds in the point cloud data to be processed whose pose information confidence level is less than the confidence threshold. A second point cloud map is then generated based on the optimized point cloud data. Because this disclosure optimizes the pose information of point clouds in the point cloud data to be processed whose pose information confidence level is less than the confidence threshold, the second point cloud map generated based on the optimized point cloud data has higher accuracy, effectively avoiding the impact of inaccurate point cloud pose on the accuracy of the generated point cloud map.

[0078] Based on the above embodiments, in another embodiment provided in this disclosure, such asFigure 2 As shown, the method may further include the following steps:

[0079] In step S160, feature extraction is performed on each frame of point cloud in the first point cloud map to obtain the feature information of each frame of point cloud.

[0080] In step S170, the feature attributes of each frame point cloud are determined based on the feature information, and point clouds with the same feature attributes are divided into the same feature region to obtain multiple different feature regions.

[0081] In this embodiment, feature extraction can be performed on each frame of the point cloud in the first point cloud map to obtain the feature information of each frame. Based on the feature attributes of each frame, point clouds with the same feature attributes are grouped into the same feature region. Alternatively, feature extraction can begin from the first frame of the point cloud map. Based on the feature attributes of its feature information, feature regions can be divided. In subsequent adjacent frame point clouds, the feature attributes determined by its feature information are used to determine whether it can be included in the planar feature region where the first frame point cloud is located. If not, a new feature region is constructed. In this way, all frame point clouds in the first point cloud map are divided into corresponding feature regions.

[0082] Specifically, feature extraction can be performed on each frame of the point cloud map. For example, features such as surfaces, lines, cubes, spheres, and cubic shapes can be extracted from each frame of the point cloud through curvature calculation and normal vector calculation. Then, the feature points in each frame of the point cloud can be assigned values ​​according to the feature point attributes to represent the feature attributes to which each feature point belongs.

[0083] Therefore, in this embodiment, the target error model corresponding to the target feature region can be determined based on the region type of the target feature region. Different region types correspond to different error models. The error of the target feature region is determined based on the target error model and the pose information of at least a portion of the frame point cloud contained within the target feature region. By dividing the first point cloud map into different feature regions, error equations can be established based on the feature attributes of the feature regions to obtain the error of the corresponding feature region. This allows for pose optimization of point clouds with pose information confidence levels below a certain threshold within the corresponding feature region, avoiding inaccuracies in the generated point cloud map due to inaccurate point cloud poses.

[0084] Based on the above embodiments, in another embodiment provided in this disclosure, the coordinate system where the first point cloud map is located is the target navigation coordinate system, such as... Figure 3 As shown, step S140 above specifically includes:

[0085] In step S141, the minimum average distance from the three-dimensional points contained in at least a portion of the frame point cloud in the target feature region to the plane in the target feature region is obtained through the target error model.

[0086] In step S142, the pose information of the target frame point cloud is adjusted so that the distance from the three-dimensional point in the target frame point cloud to the plane in the target feature region is not greater than the minimum average distance.

[0087] If the target feature region is a planar feature region, then the error of the target feature region is calculated based on the pose information of at least a portion of the point cloud within the planar feature region. This involves calculating the minimum average distance from each 3D point contained in that portion of the point cloud to its respective plane. By adjusting the pose information of the target frame point cloud, the distance from each 3D point in the target frame point cloud to the plane within the target feature region is ensured to be no greater than the minimum average distance. This is equivalent to optimizing the plane thickness within the planar feature region, guaranteeing that the plane thickness does not exceed a certain value, thereby improving the accuracy of the generated point cloud map. The target error model in this case can be found in the description of the above embodiment, and will not be repeated here.

[0088] Based on the above embodiments, in another embodiment provided in this disclosure, step S150 may further include the following steps:

[0089] In S151, the confidence level of the pose information corresponding to each frame of the point cloud in the point cloud data to be processed is traversed.

[0090] In S152, the pose information of the point cloud of the frame to be optimized is optimized if the confidence level of the pose information is less than the confidence level threshold, and the pose-optimized point cloud data to be processed is obtained.

[0091] In S153, a point cloud map is generated based on the point cloud data to be processed after pose optimization, and a second point cloud map is obtained by voxelization filtering of the point cloud map to be processed.

[0092] In the embodiment provided in this disclosure, the pose-optimized point cloud data can be obtained by traversing the confidence of the pose information corresponding to each frame of point cloud in the point cloud data to be processed, and optimizing the pose information of the point cloud in the frame to be optimized whose confidence of the pose information is less than the confidence threshold. Since the pose-optimized point cloud data is the point cloud in the point cloud data to be processed, a point cloud map with higher accuracy can be obtained based on the pose-optimized point cloud data.

[0093] Based on the above embodiments, in another embodiment provided in this disclosure, step S150 may further include the following steps:

[0094] In S154, the point cloud of the frame to be optimized is obtained from the point cloud data to be processed, where the confidence level of the pose information is less than the confidence level threshold.

[0095] In S155, the pose information of the point cloud of the frame to be optimized is optimized to obtain the optimized point cloud.

[0096] In S156, a third point cloud map is generated based on the optimized frame point cloud.

[0097] In S157, the point cloud of the frame to be optimized is removed from the first point cloud map, and the third point cloud map is stitched onto the removed first point cloud map to obtain the second point cloud map.

[0098] In the embodiments provided in this disclosure, since a first point cloud map has already been generated from the point cloud data to be processed, the optimized frame point cloud can be obtained by optimizing the frame point cloud whose confidence in the pose information in the point cloud data to be processed is less than the confidence threshold. The optimized frame point cloud is then generated based on the optimized frame point cloud. In this way, the frame point cloud to be optimized in the first point cloud map is removed from the first point cloud map, and the third point cloud map is correspondingly stitched into the removed first point cloud map to obtain the third point cloud map. This not only yields a third point cloud map with higher accuracy than the first point cloud map, but also greatly reduces the computational complexity and avoids completely regenerating the point cloud map from the optimized point cloud data to be processed.

[0099] By dividing each function into corresponding functional modules, this disclosure provides a point cloud map generation device, which can be a server or a chip applied to a server. Figure 4 This is a schematic block diagram of the functional modules of a point cloud map generation apparatus provided as an exemplary embodiment of the present disclosure. Figure 4 As shown, the point cloud map generation device includes:

[0100] The first map generation module 10 is used to generate a first point cloud map based on the point cloud data to be processed.

[0101] The feature region acquisition module 20 is used to acquire target feature regions in the first point cloud map; the first point cloud map is pre-divided into multiple feature regions according to feature attributes, and different feature regions have corresponding different region types.

[0102] The error acquisition module 30 is used to determine the error of the target feature region based on the region type of the target feature region and the pose information of at least a portion of the point cloud contained in the target feature region.

[0103] The pose information optimization module 40 is used to optimize the pose information of the target frame point cloud in the target feature region by means of the error in the target feature region; the confidence level of the pose information of the target frame point cloud is less than the confidence level threshold.

[0104] The second point cloud map generation module 50 is used to generate a second point cloud map based on the pose-optimized point cloud data to be processed.

[0105] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0106] The feature extraction module is used to extract features from each frame of point cloud in the first point cloud map to obtain feature information of each frame of point cloud.

[0107] The region segmentation module is used to determine the feature attributes of each frame point cloud based on the feature information, and to divide point clouds with the same feature attributes into the same feature region to obtain the multiple different feature regions.

[0108] In yet another embodiment provided in this disclosure, it is specifically used to include:

[0109] The target error model corresponding to the target feature region is determined based on the region type of the target feature region; wherein, different region types correspond to different error models.

[0110] The error of the target feature region is determined based on the target error model and the pose information of at least a portion of the frame point cloud contained in the target feature region.

[0111] In another embodiment provided in this disclosure, the target area type is planar, and the coordinate system where the first point cloud map is located is the target navigation coordinate system; the error acquisition module is further used for:

[0112] The minimum average distance from the three-dimensional points contained in at least a portion of the point cloud in the target feature region to the plane in the target feature region is obtained through the target error model.

[0113] The pose information of the target frame point cloud is adjusted so that the distance from the three-dimensional point in the target frame point cloud to the plane in the target feature region is not greater than the minimum average distance.

[0114] In another embodiment provided in this disclosure, the target error model is:

[0115]

[0116] in, The error is the error of the target feature region; For coordinate transformation from the device coordinate system to the combined inertial navigation system, For the coordinate transformation from the combined inertial navigation coordinate system to the target navigation coordinate system, P LiLet P0 be the coordinates of any 3D point in the target frame point cloud in the device coordinate system, P0 be the center coordinates of a plane in the target feature region in the target navigation coordinate system, N be the number of planes in the target feature region, M be the number of 3D points on a plane, and n be the coordinates of any 3D point in the target frame point cloud in the device coordinate system. T This represents the transpose of the normal vector.

[0117] In another embodiment provided in this disclosure, the second point cloud map generation module is further configured to:

[0118] The confidence level of the pose information corresponding to each frame of point cloud in the point cloud data to be processed is traversed.

[0119] The pose information of the point cloud of the frame to be optimized is optimized such that the confidence level of the pose information is less than the confidence threshold, and the pose-optimized point cloud data is obtained.

[0120] A point cloud map is generated based on the pose-optimized point cloud data, and a second point cloud map is obtained by voxelizing and filtering the point cloud map.

[0121] In another embodiment provided in this disclosure, the second point cloud map generation module is further configured to:

[0122] Obtain the frame point cloud to be optimized from the point cloud data to be processed where the confidence level of the pose information is less than the confidence threshold;

[0123] The pose information of the point cloud in the frame to be optimized is optimized to obtain the optimized point cloud.

[0124] A third point cloud map is generated based on the optimized frame point cloud;

[0125] The point cloud of the frame to be optimized is removed from the first point cloud map, and the third point cloud map is stitched onto the first point cloud map after removal to obtain the second point cloud map.

[0126] Since the apparatus corresponds to the method, please refer to the description of the corresponding embodiment of the method above, which will not be repeated here.

[0127] The point cloud map generation apparatus provided in this embodiment generates a first point cloud map based on point cloud data to be processed. It determines the error of the target feature region based on the region type of the target feature region and the pose information of at least a portion of the point clouds contained within the target feature region. The apparatus then optimizes the pose information of target frame point clouds in the target feature region whose confidence level is less than a confidence threshold using the error. This optimizes the pose information of point clouds in the point cloud data to be processed whose pose information confidence level is less than the confidence threshold. A second point cloud map is then generated based on the optimized point cloud data. In this embodiment, because the pose information of point clouds in the point cloud data to be processed is optimized for pose information with a confidence level less than the confidence threshold, the second point cloud map generated based on the optimized point cloud data has higher accuracy, effectively avoiding the impact of inaccurate point cloud pose on the accuracy of the generated point cloud map.

[0128] This disclosure also provides an electronic device, including: at least one processor; a memory for storing processor-executable instructions; wherein the at least one processor is configured to execute the instructions to implement the methods disclosed in this disclosure.

[0129] Figure 5 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 5 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0130] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the method described above.

[0131] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 6 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 6 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0132] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Figure 6As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0134] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0135] The computing unit 1901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1900 via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0136] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0137] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0139] This disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this disclosure.

[0140] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0143] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0144] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0145] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for generating a point cloud map, the method comprising: The method comprises: generating a first point cloud map based on to-be-processed point cloud data; obtaining a target feature region in the first point cloud map; the first point cloud map is pre-divided into a plurality of feature regions according to feature attributes, and different feature regions correspond to different region types; determining an error of the target feature region based on a region type of the target feature region and pose information of at least part of point clouds contained in the target feature region; optimizing pose information of target frame point clouds in the target feature region through the error of the target feature region; a confidence of the pose information of the target frame point clouds is less than a confidence threshold; generating a second point cloud map based on the to-be-processed point cloud data after pose optimization; the determination of the error of the target feature region comprises: determining a target error model corresponding to the target feature region based on a region type of the target feature region; different error models correspond to feature regions of different region types; determining the error of the target feature region based on the target error model and the pose information of at least part of frame point clouds contained in the target feature region; the target error model is: wherein, is the error of the target feature region; is the coordinate transformation from the device coordinate system to the combined inertial navigation, is the coordinate transformation from the combined inertial navigation coordinate system to the target navigation coordinate system, is the coordinate of any point in the target frame point cloud in the device coordinate system, is the coordinate of the center position of a plane in the target feature region in the target navigation coordinate system, N is the number of planes in the target feature region, M is the number of points on a plane, n T denotes the normal vector transpose.

2. The method of claim 1, wherein, the method further comprises: performing feature extraction on each frame point cloud in the first point cloud map to obtain feature information of each frame point cloud; determining feature attributes of each frame point cloud based on the feature information, and dividing point clouds with the same feature attribute into the same feature region to obtain the plurality of different feature regions.

3. The method of claim 1, wherein, The target region type is a plane, and a coordinate system in which the first point cloud map is located is a target navigation coordinate system; the optimization of the pose information of the target frame point clouds in the target feature region through the error of the target feature region comprises: obtaining, through the target error model, minimum average distances of points in at least part of point clouds in the target feature region to corresponding planes in the target feature region under the target navigation coordinate system; adjusting the pose information of the target frame point clouds so that distances of points in the target frame point clouds to the planes in the target feature region are not greater than the minimum average distances.

4. The method of claim 1, wherein, The target region type is a straight line; the determination of the error of the target feature region comprises: fitting points in the point clouds in the target feature region into a straight line; obtaining minimum average distances of points in at least part of point clouds in the target feature region to the straight line, and taking the minimum average distances to the straight line as the error.

5. The method of claim 1, wherein, The target region type is a spherical surface; the determination of the error of the target feature region comprises: fitting points in the point clouds in the target feature region into a spherical surface; obtaining minimum average distances of points in at least part of point clouds in the target feature region to the spherical surface, and taking the minimum average distances to the spherical surface as the error.

6. The method of claim 1, wherein, The generation of the second point cloud map based on the to-be-processed point cloud data after pose optimization comprises: traversing confidences of pose information of each frame point cloud in the to-be-processed point cloud data; optimizing pose information of to-be-optimized frame point clouds whose confidences of pose information are less than the confidence threshold to obtain the to-be-processed point cloud data after pose optimization. generate a second point cloud map based on the to-be-processed point cloud data after pose optimization.

7. The method of claim 1, wherein, The generating a second point cloud map based on the to-be-processed point cloud data after pose optimization comprises: obtaining to-be-optimized frame point clouds in the to-be-processed point cloud data whose confidence of pose information is less than the confidence threshold; optimizing the pose information of the to-be-optimized frame point clouds to obtain optimized frame point clouds; generating a third point cloud map based on the optimized frame point clouds; removing the to-be-optimized frame point clouds from the first point cloud map, and splicing the third point cloud map into the first point cloud map after removal to obtain the second point cloud map.

8. A point cloud map generation apparatus, characterized by comprising: The apparatus comprises: a first map generation module configured to generate a first point cloud map based on to-be-processed point cloud data; a feature region acquisition module configured to acquire a target feature region in the first point cloud map; the first point cloud map is pre-divided into a plurality of feature regions according to feature attributes, and different feature regions correspond to different region types; an error acquisition module configured to determine a target error model corresponding to the target feature region based on a region type of the target feature region and pose information of at least part of point clouds contained in the target feature region; determine an error of the target feature region based on the target error model and pose information of at least part of frame point clouds contained in the target feature region; different error models correspond to different region types of feature regions; a pose information optimization module configured to optimize pose information of target frame point clouds in the target feature region through the error of the target feature region; the confidence of the pose information of the target frame point clouds is less than a confidence threshold; a second point cloud map generation module configured to generate a second point cloud map based on the to-be-processed point cloud data after pose optimization. The target error model is: wherein, is the error of the target feature region; is the coordinate transformation from the device coordinate system to the combined inertial navigation, is the coordinate transformation from the combined inertial navigation coordinate system to the target navigation coordinate system, is the coordinate of any point in the target frame point cloud in the device coordinate system, is the coordinate of the center position of a plane in the target feature region in the target navigation coordinate system, N is the number of planes in the target feature region, M is the number of points on a plane, n T denotes the normal vector transpose.

9. An electronic device, comprising: comprises: at least one processor; a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the method of any one of claims 1-7. When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the method of any one of claims 1-7.

Images