Map building method, device, equipment, storage medium, product and system

Abstract

The invention relates to a map building method, device, equipment, storage medium, product and system, and relates to the technical field of map building.The method comprises the steps that target detection data are obtained, the target detection data are latest detection data obtained by a target sensor, and the target detection data comprise multiple frames of point cloud data; the method comprises the following steps: acquiring target detection data, performing data processing on the target detection data to obtain a target point cloud, the target point cloud comprising a feature point cloud and a semantic point cloud, obtaining inter-frame poses among multiple frames of point cloud data according to the target point cloud and a target historical map, and mapping the target historical map according to historical detection data acquired by a target sensor. The target historical map comprises a feature point cloud layer and a semantic point cloud layer, and updating the feature point cloud layer and the semantic point cloud layer in the target historical map according to the target point cloud and the inter-frame pose to obtain a target map. And thus, a more accurate target map can be obtained.

Description

Technical Field

[0001] This disclosure relates to the field of map construction technology, and in particular to a map construction method, apparatus, device, storage medium, product and system. Background Technology

[0002] Autonomous valet parking systems, as an application of autonomous driving in parking scenarios, can achieve fully automated valet parking, thus replacing traditional manual valet parking and helping users save significant parking time, solving the pain point of queuing for parking during peak hours. However, autonomous valet parking systems require accurate parking lot space maps.

[0003] Among the related technologies, the main focus is on map building for underground parking lots, which can be achieved using laser acquisition or visual acquisition. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a map building method, apparatus, device, storage medium, product and system to improve the accuracy of map building for large outdoor parking lots.

[0005] According to a first aspect of the present disclosure, a map creation method is provided, comprising: Acquire target detection data, which is the latest detection data acquired by the target sensor, and the target detection data includes multi-frame point cloud data; The target detection data is processed to obtain a target point cloud, which includes a feature point cloud and a semantic point cloud. Based on the target point cloud and the target historical map, the inter-frame pose between the multi-frame point cloud data is obtained. The target historical map is constructed based on the historical detection data obtained by the target sensor. The target historical map includes a feature point cloud layer and a semantic point cloud layer. Based on the target point cloud and the inter-frame pose, the feature point cloud layer and the semantic point cloud layer in the target historical map are updated to obtain the target map.

[0006] Optionally, the step of processing the target detection data to obtain a target point cloud includes: Based on the first inertial measurement data of the target sensor, distortion is removed from each frame of point cloud data in the target detection data to obtain multiple frames of distortion-removed point cloud. Based on the second inertial measurement data of the target sensor, the multi-frame distortion-free point cloud is stitched together to obtain a local point cloud; The target point cloud is obtained by extracting feature point cloud and semantic point cloud from the local point cloud.

[0007] Optionally, before performing feature point cloud extraction and semantic point cloud extraction on the local point cloud to obtain the target point cloud, the method further includes: The local point cloud is segmented to determine the ground point cloud within the local point cloud; Feature thinning is performed on the ground point cloud in the local point cloud to obtain the key local point cloud; The step of extracting feature point cloud and semantic point cloud from the local point cloud to obtain the target point cloud includes: The key local point cloud is subjected to feature point cloud extraction and semantic point cloud extraction to obtain the target point cloud.

[0008] Optionally, obtaining the inter-frame pose between the multiple frames of point cloud data based on the target point cloud and the target historical map includes: The degradation coefficient of the target sensor is obtained based on the feature point cloud and the target historical map; Based on the degradation coefficient, determine the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud; Based on the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud, the feature point cloud and the semantic point cloud are registered to obtain the inter-frame pose between the multi-frame point cloud data.

[0009] Optionally, determining the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud based on the degradation coefficient includes: If the degradation coefficient is less than the degradation coefficient threshold, the weight of the feature point cloud is increased and the weight of the semantic point cloud is decreased to obtain the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud.

[0010] Optionally, updating the feature point cloud layer and the semantic point cloud layer in the target historical map based on the target point cloud and the inter-frame pose to obtain the target map includes: Based on the inter-frame pose, the feature point cloud in the target point cloud is stitched together to obtain a stitched feature point cloud; Based on the inter-frame pose, the semantic point cloud in the target point cloud is spliced ​​together to obtain a spliced ​​semantic point cloud. The feature point cloud layer in the target historical map is updated based on the stitched feature point cloud, and the semantic point cloud layer in the target historical map is updated based on the stitched semantic point cloud to obtain the target map.

[0011] Optionally, the method further includes: If the cumulative number of frames of acquired multi-frame point cloud data exceeds a preset number of frames threshold, the inter-frame constraints corresponding to the target detection data are determined. Based on the inter-frame constraints, the target map is globally optimized to obtain an optimized map.

[0012] Optionally, the method further includes: Based on the target detection data and the target historical map, loop closure detection is performed to obtain the loop closure detection results; If the loop closure detection result indicates the existence of loop closure data, then the loop closure constraint is obtained based on the loop closure data; Based on the lapsing constraints, the target map is globally optimized to obtain an optimized map.

[0013] According to a second aspect of the present disclosure, a map-building apparatus is provided, comprising: The acquisition module is configured to acquire target detection data, which is the latest detection data acquired by the target sensor, and the target detection data includes multi-frame point cloud data; The first acquisition module is configured to process the target detection data to obtain a target point cloud, the target point cloud including a feature point cloud and a semantic point cloud; The second acquisition module is configured to obtain the inter-frame pose between the multi-frame point cloud data based on the target point cloud and the target historical map. The target historical map is constructed based on the historical detection data obtained by the target sensor and includes a feature point cloud layer and a semantic point cloud layer. The third acquisition module is configured to update the feature point cloud layer and the semantic point cloud layer in the target historical map based on the target point cloud and the inter-frame pose, so as to obtain the target map.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the steps of the map building method provided in the first aspect of this disclosure.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the map building method provided in the first aspect of the present disclosure.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the map creation method provided in the first aspect of the present disclosure.

[0017] According to a sixth aspect of the present disclosure, a chip system is provided, the chip system including a processing unit and an interface circuit, the processing unit obtaining program instructions through the interface circuit, the program instructions being executed by the processing unit, the processing unit being used to implement the steps of the map building method provided in the first aspect of the present disclosure when executing.

[0018] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: By acquiring target detection data, which is the latest detection data obtained by the target sensor and includes multiple frames of point cloud data, and processing the target detection data to obtain target point cloud, which includes feature point cloud and semantic point cloud, and then obtaining inter-frame pose between multiple frames of point cloud data based on the target point cloud and the target historical map, the target historical map is constructed based on the historical detection data obtained by the target sensor. The target historical map includes feature point cloud layer and semantic point cloud layer, and the feature point cloud layer and semantic point cloud layer in the target historical map are updated based on the target point cloud and inter-frame pose to obtain the target map.

[0019] By processing the latest acquired target detection data and updating the target historical map, a target map with a wider scope or more accurate data can be obtained. The target historical map includes a feature point cloud layer and a semantic point cloud layer. During the processing of the target detection data, feature point clouds and semantic point clouds are also extracted from the target detection data. By combining the inter-frame poses between multiple frames of point cloud data, more accurate update data can be obtained, so as to update the feature point cloud layer and semantic point cloud layer in the target historical map more accurately. Among them, the feature point cloud layer and semantic point cloud layer can extract accurate bounding point clouds, as well as line features and surface features, thus obtaining a more accurate target map.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of a map-building method according to an exemplary embodiment.

[0023] Figure 2 This is a flowchart illustrating a map creation method according to an exemplary embodiment.

[0024] Figure 3 This is a rendering of a map-building method according to an exemplary embodiment.

[0025] Figure 4 This is a layered schematic diagram of a target historical map according to an exemplary embodiment.

[0026] Figure 5 This is a flowchart illustrating another map-building method according to an exemplary embodiment.

[0027] Figure 6 This is a block diagram illustrating a map-building apparatus according to an exemplary embodiment.

[0028] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0029] Figure 8 This is a block diagram illustrating a chip system according to an exemplary embodiment. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0032] Autonomous valet parking systems, as an application of autonomous driving in parking scenarios, can achieve fully automated valet parking, thus replacing traditional manual valet parking and helping users save significant parking time, solving the pain point of queuing for parking during peak hours. However, autonomous valet parking systems require accurate parking lot space maps.

[0033] Among related technologies, the main focus is on map construction for underground parking lots. Laser or visual data acquisition can be used for this purpose. Because underground parking lots have rich geometric features such as walls and pillars, laser technology can effectively achieve 3D reconstruction of the parking lot. Visual solutions, on the other hand, can extract features such as parking space bounding boxes and arrow markers through deep learning, converting them into a bird's-eye view. Semantic matching can then be used to construct a semantic map of the parking lot.

[0034] For large outdoor parking lots, which are relatively open and have a large area with few surrounding features, mapping requires including the parking lot itself and the roads near the entrances and exits. While laser mapping works successfully on roads, it suffers from degradation within the parking lot due to insufficient features, resulting in ghosting of parking lines and lower map accuracy. Using visual semantic mapping is also ineffective in certain road scenes lacking parking space boundaries, further reducing map accuracy.

[0035] To address the technical problem of low accuracy in the constructed maps mentioned above, this disclosure provides a map building method, apparatus, device, storage medium, product, and system. By processing the latest acquired target detection data, the target historical map is updated, resulting in a target map with a wider range or more accurate data. The target historical map includes a feature point cloud layer and a semantic point cloud layer. During the processing of the target detection data, feature point clouds and semantic point clouds are also extracted from the target detection data. By combining the inter-frame poses between multiple frames of point cloud data, more accurate update data can be obtained, enabling more accurate updates to the feature point cloud layer and semantic point cloud layer in the target historical map. The feature point cloud layer and semantic point cloud layer can extract accurate bounding point clouds, line features, and surface features, thereby obtaining a more accurate target map.

[0036] Figure 1 This is a schematic diagram illustrating an application scenario of a map-building method according to an exemplary embodiment, such as... Figure 1 As shown, it can be used to create maps of large outdoor parking lots.

[0037] Figure 2 This is a flowchart illustrating a map-building method according to an exemplary embodiment, such as... Figure 2 As shown, the following steps may be included.

[0038] In step S201, target detection data is acquired. This target detection data is the latest detection data acquired by the target sensor, and it includes multi-frame point cloud data.

[0039] In this embodiment, SLAM (Simultaneous Localization and Mapping) can be used for mapping, and the mapping object can be a large outdoor parking lot. The target sensor can be a LiDAR. Data detected by the LiDAR can be acquired periodically, and each piece of detected data can be processed. The processed data is then used to update the existing part of the map, thereby gradually building a complete map.

[0040] The target detection data can be the latest detection data acquired by the target sensor in its latest detection cycle. This target detection data can include multiple frames of point cloud data acquired by the target sensor throughout the entire cycle. During the detection process, the target sensor continuously moves. For example, the target sensor can be mounted on a vehicle, and the vehicle moves the target sensor continuously over the area requiring mapping to complete the detection of that area. Multiple frames of point cloud data can be obtained in one cycle.

[0041] In step S202, the target detection data is processed to obtain a target point cloud, which includes a feature point cloud and a semantic point cloud.

[0042] In this embodiment, feature point cloud extraction and semantic point cloud extraction can be performed on the multi-frame point cloud data in the detected target detection data to obtain feature point cloud and semantic point cloud.

[0043] In step S203, the inter-frame pose between multiple frames of point cloud data is obtained based on the target point cloud and the target historical map. The target historical map is constructed based on the historical detection data obtained by the target sensor and includes a feature point cloud layer and a semantic point cloud layer.

[0044] In this embodiment, the target point cloud data may include feature point clouds and semantic point clouds corresponding to each frame of point cloud data. Registration can be performed between the target point cloud and the target historical map to obtain inter-frame poses between multiple frames of point cloud data. These inter-frame poses can be the relative position and orientation changes between two or more consecutive frames. The target historical map is constructed based on historical detection data acquired by the target sensor, i.e., a local map already built based on the detected data. Optionally, this local map includes the latest local map constructed from point cloud data acquired in the previous cycle. Registration can be performed between the target point cloud and the latest local map to obtain inter-frame poses between multiple frames of point cloud data. The map constructed in each cycle can be managed through a sliding window to ensure that the latest local map is the most recently generated. Maps outside the sliding window are stored in a historical sub-map, which is used for loop closure detection. The target historical map includes a feature point cloud layer and a semantic point cloud layer. The feature point cloud layer can be used to extract the bounding boxes of parking spaces, and the semantic point cloud layer can be used to extract line and surface features, thereby constructing an accurate map.

[0045] In step S204, the feature point cloud layer and semantic point cloud layer in the target historical map are updated according to the target point cloud and inter-frame pose to obtain the target map.

[0046] In this embodiment, based on the feature point cloud and semantic point cloud in the target point cloud, and combined with the inter-frame pose between multiple frames of point cloud data, the data in the feature point cloud layer and semantic point cloud layer of the target historical map are updated respectively. Then, based on the feature point cloud layer and semantic point cloud layer, the data is extracted and processed separately to construct the updated target map. Specifically, the semantic point cloud layer can extract the bounding box point cloud of parking spaces through point cloud intensity or a deep learning target extraction model. The feature point cloud layer can use the mean and covariance of a local Gaussian distribution using Voxelmap technology, and utilize the distribution of covariance to extract line and surface features. Voxelmap divides the space into voxels and fits a local Gaussian distribution within each voxel, using the mean and covariance to describe the distribution characteristics of the point cloud. By analyzing the covariance matrix within the voxels, line and surface features can be extracted. For example, the shape and orientation of the covariance matrix can indicate the arrangement of the point cloud in space, thereby identifying straight lines or planar structures.

[0047] Figure 3 This is a rendering of a map-building method according to an exemplary embodiment, such as... Figure 3As shown, in this embodiment, by processing the latest acquired target detection data and updating the target historical map, a target map with a wider range or more accurate data can be obtained. The target historical map includes a feature point cloud layer and a semantic point cloud layer. During the processing of the target detection data, feature point clouds and semantic point clouds are also extracted from the target detection data. Furthermore, by combining the inter-frame poses between multiple frames of point cloud data, more accurate update data can be obtained. This allows for more accurate updates to the feature point cloud layer and semantic point cloud layer in the target historical map. Specifically, the feature point cloud layer and semantic point cloud layer can extract accurate bounding box point clouds, line features, and surface features, thereby obtaining a more accurate target map.

[0048] In one possible implementation, processing the target detection data to obtain a target point cloud may include the following steps: Based on the first inertial measurement data of the target sensor, the point cloud data of each frame in the target detection data is distorted to obtain multiple frames of distorted point cloud; based on the second inertial measurement data of the target sensor, the multiple frames of distorted point cloud are stitched together to obtain a local point cloud; feature point cloud extraction and semantic point cloud extraction are performed on the local point cloud to obtain the target point cloud.

[0049] In this embodiment, the target sensor can be detected by an inertial measurement unit (IMU) to obtain inertial measurement data. The IMU data is then processed to obtain first and second inertial measurement data. The first inertial measurement data can be the motion trajectory. Based on the first IMU data, distortion correction is performed on each frame of point cloud data in the target detection data to eliminate the effects of motion, resulting in multiple frames of distorted point clouds. The second inertial measurement data can be the pre-integrated pose of the IMU. For the multiple frames of distorted point clouds, the distorted point clouds of adjacent frames can be stitched together based on the second IMU data to synthesize a local point cloud while maintaining relative accuracy. Feature point cloud extraction and semantic point cloud extraction are then performed on the local point cloud to obtain feature point clouds and semantic point clouds, thus obtaining the target point cloud.

[0050] In one possible implementation, for large outdoor parking lots, where the open area constitutes a large proportion of the ground, the ground point cloud data can be simplified before extracting feature point clouds and semantic point clouds from the local point clouds to obtain the target point cloud. This is to avoid the problem of the ground features having too much weight, leading to an inaccurate map. Specifically, this simplification method can be as follows: The local point cloud is segmented to identify the ground point cloud within it; then, the ground point cloud within the local point cloud is thinned to obtain the key local point cloud.

[0051] In this embodiment, point cloud segmentation can be performed on the local point cloud. During point cloud segmentation, attempts can be made to distinguish different objects or surfaces in the point cloud data to determine the ground point cloud in the local point cloud. For open areas, the segmentation algorithm can identify the ground point cloud by finding a horizontal plane or minimizing the vertical distance between the point cloud and the horizontal plane. Then, feature thinning is performed on the ground point cloud in the local point cloud to reduce the number of points in the ground point cloud, retaining the point cloud corresponding to key features to obtain the key local point cloud, thereby reducing the density and weight of the ground point cloud in the overall system.

[0052] After obtaining the key local point cloud, the method for extracting feature point cloud and semantic point cloud from the local point cloud to obtain the target point cloud can be as follows: extract feature point cloud and semantic point cloud from the key local point cloud to obtain the target point cloud.

[0053] Feature point cloud extraction involves identifying point clouds with significant geometric characteristics. These point clouds typically include corner points, edge points, or center points of planar regions, representing salient structures within a scene. Feature point cloud extraction simplifies point cloud data, facilitating subsequent data processing and analysis. Various algorithms can be used to extract feature points, such as methods based on point cloud curvature, methods based on geometric properties, or machine learning-based methods.

[0054] Semantic point cloud extraction assigns semantic information to each point in a point cloud dataset and extracts the point cloud data, enabling the identification and classification of different objects or substances within the point cloud, such as vehicles, pedestrians, buildings, and roads. Semantic point cloud extraction can be implemented using deep learning or machine learning algorithms, which can be trained on large amounts of labeled data to identify and classify different categories within the point cloud.

[0055] In one possible implementation, obtaining the inter-frame pose between multiple frames of point cloud data based on the target point cloud and the target historical map may include the following steps: Based on the feature point cloud and the target historical map, the degradation coefficient of the target sensor is obtained; based on the degradation coefficient, the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud are determined; based on the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud, the feature point cloud and the semantic point cloud are registered to obtain the inter-frame pose between multiple frames of point cloud data.

[0056] In this embodiment, the feature point cloud can be registered with the target historical map to obtain the degradation coefficient of the target sensor. This degradation coefficient characterizes the degree of degradation of the target sensor; a larger coefficient indicates a higher degree of degradation. When the degradation coefficient is small, the feature point cloud can be well configured, resulting in high-precision data. A first registration weight for the feature point cloud and a second registration weight for the semantic point cloud can be determined based on the magnitude of the degradation coefficient. Specifically, the first registration weight can be positively correlated with the degradation coefficient, and the second registration weight can be negatively correlated. Then, based on the first registration weight and the second registration weight, the feature point cloud and semantic point cloud are registered to obtain the inter-frame pose between multiple frames of point cloud data. This inter-frame pose can be the relative position and attitude change between two or more consecutive frames. Specifically, based on the first registration weight, the first number of targets in the feature point cloud can be determined, and the feature point cloud can be simplified based on the first number of targets. The second registration weight of the semantic point cloud can be used to determine the number of second targets in the semantic point cloud. Then, the semantic point cloud can be simplified based on the number of second targets, and registration can be performed based on the simplified feature point cloud and the semantic point cloud. By adaptively adjusting the registration weight through the degradation coefficient, the problem of LiDAR degradation in parking lot laser mapping in open scenes can be solved, resulting in better registration effect and thus a more accurate map.

[0057] During registration, points in the feature point cloud and semantic point cloud are matched with points in the target historical map. By calculating feature descriptors and using matching algorithms, such as nearest neighbor search, corresponding feature point pairs can be found. Based on these feature point pairs, the transformation relationship between two frames is estimated, typically including rotation and translation, to obtain the inter-frame pose between multiple frames of point cloud data.

[0058] In one possible implementation, the feature point cloud can be registered with the target historical map to obtain the Jacobian matrix of the registration residual, and the degradation coefficient can be obtained based on the Jacobian matrix of the registration residual. The formula for the degradation coefficient can be: (1) (2) (3) in, For information matrix, To register the Jacobian matrix of the residuals, These are the eigenvalues ​​of the information matrix. For the eigenvectors of the information matrix, The maximum value of the eigenvalues ​​of the information matrix. It represents the minimum value of the eigenvalues ​​of the information matrix.

[0059] Based on the above formulas (1), (2) and (3), the degradation coefficient can be obtained.

[0060] In one possible implementation, the first registration weight can be positively correlated with the degradation coefficient, and the second registration weight can be negatively correlated with the degradation coefficient. That is, the larger the degradation coefficient, the smaller the first registration weight and the larger the second registration weight, and the sum of the first registration weight and the second registration weight can be 1. For example, if the degradation coefficient is 0.2, the first registration weight can be 0.8 and the second registration weight can be 0.2.

[0061] In one possible implementation, determining the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud based on the degradation coefficient includes: When the degradation coefficient is less than the degradation coefficient threshold, the weight of the feature point cloud is increased and the weight of the semantic point cloud is decreased to obtain the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud.

[0062] In this embodiment, a degradation coefficient threshold can be set. If the degradation coefficient is greater than or equal to the threshold, the target sensor is considered degraded, and the weight of the feature point cloud needs to be reduced while the weight of the semantic point cloud needs to be increased. If the degradation coefficient is less than the threshold, the target sensor is considered not degraded, and the weight of the feature point cloud can be increased while the weight of the semantic point cloud decreases. A first baseline weight corresponding to the feature point cloud and a second baseline weight corresponding to the semantic point cloud are set, allowing for increases or decreases based on the first and second baseline weights. The magnitude of the increase or decrease can be determined based on the difference between the degradation coefficient and the degradation coefficient threshold.

[0063] Figure 4 This is a layered schematic diagram of a target historical map according to an exemplary embodiment, such as... Figure 4 As shown, in one possible implementation, the feature point cloud layer and semantic point cloud layer in the target historical map are updated based on the target point cloud and inter-frame pose to obtain the target map, including: Based on the inter-frame pose, the feature point cloud in the target point cloud is stitched together to obtain the stitched feature point cloud; based on the inter-frame pose, the semantic point cloud in the target point cloud is stitched together to obtain the stitched semantic point cloud; the feature point cloud layer in the target historical map is updated according to the stitched feature point cloud, and the semantic point cloud layer in the target historical map is updated according to the stitched semantic point cloud to obtain the target map.

[0064] In this embodiment, inter-frame pose can be the relative position and orientation change between two or more consecutive frames. Based on the inter-frame pose, the feature point clouds of adjacent frames in the target point cloud can be stitched together to obtain a stitched feature point cloud. Similarly, based on the inter-frame pose, adjacent semantic point clouds in the target point cloud can be stitched together to obtain a stitched semantic point cloud. The data in the feature point cloud layer of the target historical map is then updated based on the stitched feature point cloud. The data in the semantic point cloud layer of the target historical map is also updated based on the stitched semantic point cloud. Finally, data extraction and processing are performed separately through the feature point cloud layer and the semantic point cloud layer. Figure 4 As shown, the target historical map 400 includes a feature point cloud layer 401 and a semantic point cloud layer 402. The semantic point cloud layer extracts the bounding point cloud of parking spaces, and the feature point cloud layer extracts line features and area features. Based on the extracted bounding point cloud of parking spaces, as well as the line features and area features, the target map is constructed.

[0065] In one possible implementation, after obtaining the target map, the target map can be optimized. This optimization method may be as follows: If the cumulative number of frames of acquired multi-frame point cloud data exceeds a preset threshold, the inter-frame constraints corresponding to the target detection data are determined; based on the inter-frame constraints, the target map is globally optimized to obtain an optimized map.

[0066] In this embodiment, it can be determined whether the current optimization condition is met. This optimization condition can be that the cumulative number of frames of acquired multi-frame point cloud data is greater than a preset number of frames threshold. Specifically, after each acquisition of target detection data, the cumulative number of frames is updated based on the number of point cloud data frames included in the target detection data. This cumulative number of frames is the total number of point cloud frames acquired after the last optimization. When the cumulative number of frames of acquired multi-frame point cloud data is greater than the preset number of frames threshold, it can be determined that the optimization condition is met, and the inter-frame constraints corresponding to the multi-frame point cloud data in the target detection data can be determined. Then, the target map is globally optimized based on the inter-frame constraints to obtain an optimized map. Specifically, the inter-frame constraints can be determined based on the difference between identical points in different frames. In the global optimization, these inter-frame constraints are evenly distributed across each frame to globally optimize the target map data and obtain a more accurate optimized map.

[0067] In one possible implementation, after obtaining the target map, the target map can be optimized. This optimization method may be as follows: Based on the target detection data and the target historical map, loop closure detection is performed to obtain the loop closure detection results. If the loop closure detection results indicate the existence of loop closure data, loop closure constraints are obtained based on the loop closure data. Based on the loop closure constraints, the target map is globally optimized to obtain the optimized map.

[0068] In this embodiment, to obtain a more accurate map, duplicate locations may be detected during data detection. Loop closure detection can be performed based on the target detection data and the target historical map to obtain loop closure detection results and determine whether duplicate locations exist. Specifically, the target detection data can be processed to obtain a stitched semantic point cloud and a stitched feature point cloud. Loop closure detection is then performed based on the stitched semantic point cloud and the stitched feature point cloud, combined with the target historical map. If the loop closure detection results indicate the presence of loop closure data, i.e., duplicate locations are detected, the loop closure data can be identified. This loop closure data is the detection data for the same point in both the target detection data and the target historical map. Based on the loop closure data, the difference between the target detection data and the detection data for the same point in the target historical map can be determined, and a loop closure constraint can be derived based on this difference. Then, the target map is globally optimized based on the loop closure constraint to obtain an optimized map. Specifically, in the global optimization, the loop closure constraint is evenly distributed across each frame to globally optimize the target map data and obtain a more accurate optimized map.

[0069] In one possible implementation, if the optimization conditions are met and loopback data also exists, the target map can be globally optimized based on loopback constraints and inter-frame constraints to obtain an optimized map.

[0070] Figure 5 This is a flowchart illustrating another map-building method according to an exemplary embodiment, such as... Figure 5 As shown, it includes the following steps: In step S501, multiple frames of point cloud data are acquired.

[0071] In step S502, distortion is removed and the data is stitched together to obtain a local point cloud.

[0072] In step S503, feature point cloud extraction and semantic point cloud extraction are performed.

[0073] In step S504, a local point cloud is obtained.

[0074] In step S505, the feature point cloud is used for registration and the degradation coefficient is calculated. This registration is performed using the feature point cloud and the local map from step S504. If the degradation coefficient indicates that the target sensor is not degraded, step S506 is executed; if the degradation coefficient indicates that the target sensor is degraded, step S507 is executed.

[0075] In step S506, the registration weight of the feature point cloud is increased.

[0076] In step S507, the semantic point cloud registration weight is increased.

[0077] In step S508, multi-layer registration is performed to obtain inter-frame pose.

[0078] In step S509, the feature point cloud and semantic point cloud are stitched together. Then, the map is updated based on the stitched feature point cloud and semantic point cloud to obtain a local map.

[0079] In step S510, the historical map is stored. After obtaining the local map, the historical map is stored.

[0080] In step S511, loop closure detection is performed to obtain loop closure constraints. Loop closure detection can be performed based on the stitched feature point cloud and semantic point cloud, combined with the historical map.

[0081] In step S512, it is determined whether the optimization conditions are met. If the optimization conditions are met, then step S513 is executed.

[0082] In step S513, the inter-frame constraints and lap closure constraints are globally optimized.

[0083] Figure 6 This is a block diagram illustrating a map-making apparatus according to an exemplary embodiment. (Refer to...) Figure 6 The map building device 600 includes an acquisition module 601, a first acquisition module 602, a second acquisition module 603, and a third acquisition module 604.

[0084] The acquisition module 601 is configured to acquire target detection data, which is the latest detection data acquired by the target sensor, and includes multi-frame point cloud data; The first obtaining module 602 is configured to process the target detection data to obtain a target point cloud, the target point cloud including a feature point cloud and a semantic point cloud; The second acquisition module 603 is configured to obtain the inter-frame pose between the multi-frame point cloud data based on the target point cloud and the target historical map. The target historical map is constructed based on the historical detection data acquired by the target sensor and includes a feature point cloud layer and a semantic point cloud layer. The third acquisition module 604 is configured to update the feature point cloud layer and the semantic point cloud layer in the target historical map based on the target point cloud and the inter-frame pose, so as to obtain the target map.

[0085] Optionally, the first obtaining module 602 includes: The first acquisition submodule is configured to perform distortion correction on each frame of point cloud data in the target detection data based on the first inertial measurement data of the target sensor, so as to obtain multiple frames of distortion-corrected point cloud. The second acquisition submodule is configured to stitch together the multi-frame distortion-free point cloud based on the second inertial measurement data of the target sensor to obtain a local point cloud; The third acquisition submodule is configured to extract feature point cloud and semantic point cloud from the local point cloud to obtain the target point cloud.

[0086] Optionally, the mapping device 600 further includes: The first determining module is configured to perform point cloud segmentation on the local point cloud and determine the ground point cloud in the local point cloud; The fourth obtaining module is configured to perform feature thinning on the ground point cloud in the local point cloud to obtain the key local point cloud; The third obtaining submodule includes: The first obtaining unit is configured to extract feature point cloud and semantic point cloud from the key local point cloud to obtain the target point cloud.

[0087] Optionally, the second obtaining module 603 includes: The fourth acquisition submodule is configured to obtain the degradation coefficient of the target sensor based on the feature point cloud and the target historical map; The determination submodule is configured to determine the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud based on the degradation coefficient; The fifth submodule is configured to register the feature point cloud and the semantic point cloud according to the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud to obtain the inter-frame pose between the multi-frame point cloud data.

[0088] Optionally, the determining submodule includes: The second obtaining unit is configured to increase the weight of the feature point cloud and decrease the weight of the semantic point cloud when the degradation coefficient is less than the degradation coefficient threshold, so as to obtain the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud.

[0089] Optionally, the third obtaining module 604 includes: The first stitching submodule is configured to stitch the feature point cloud in the target point cloud based on the inter-frame pose to obtain a stitched feature point cloud. The second stitching submodule is configured to stitch together the semantic point cloud in the target point cloud based on the inter-frame pose to obtain a stitched semantic point cloud. The sixth submodule is configured to update the feature point cloud layer in the target historical map based on the stitched feature point cloud, and update the semantic point cloud layer in the target historical map based on the stitched semantic point cloud, to obtain the target map.

[0090] Optionally, the mapping device 600 further includes: The second determining module is configured to determine the inter-frame constraints corresponding to the target detection data when the cumulative number of frames of acquired multi-frame point cloud data is greater than a preset number of frames threshold. The fifth acquisition module is configured to perform global optimization on the target map based on the inter-frame constraints to obtain an optimized map.

[0091] Optionally, the mapping device 600 further includes: The detection module is configured to perform loop closure detection based on the target detection data and the target historical map to obtain the loop closure detection result; The sixth obtaining module is configured to obtain loop closure constraints based on the loop closure data when the loop closure detection result indicates the existence of loop closure data; The seventh module is configured to perform global optimization on the target map based on the loop closure constraint to obtain an optimized map.

[0092] Regarding the map building device 600 in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0093] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the map-building method provided in this disclosure.

[0094] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, the electronic device 700 may be a vehicle-mounted system, a robot, a mobile phone, a computer, a messaging device, or a tablet device.

[0095] Reference Figure 7 The electronic device 700 may include one or more of the following components: a processing component 702, a first memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output interface 712, a sensor component 714, and a communication component 716.

[0096] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 702 may include one or more first processors 720 to execute instructions to complete all or part of the steps of the map-building method described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0097] The first memory 704 is configured to store various types of data to support the operation of the electronic device 700. Examples of this data include instructions for any application or method operating on the electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. The first memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0098] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0099] Multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0100] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in first memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0101] Input / output interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0102] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0103] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0104] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the map mapping method described above.

[0105] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 704 including instructions, which can be executed by a first processor 720 of an electronic device 700 to complete the map building method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0106] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the map-building method described above when executed by the programmable device.

[0107] Figure 8 This is a block diagram illustrating a chip system according to an exemplary embodiment. Some embodiments of this disclosure also provide a chip system, such as... Figure 8 As shown, the chip system includes at least one second processor 801 and at least one interface circuit 802. The second processor 801 and the interface circuit 802 are interconnected via lines. For example, the interface circuit 802 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 802 can be used to send signals to other devices (e.g., the second processor 801). Exemplarily, the interface circuit 802 can read instructions stored in the memory and send those instructions to the second processor 801. When the instructions are executed by the second processor 801, the image processing device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete components, and some embodiments of this disclosure do not specifically limit this.

[0108] In some embodiments of this disclosure, the interface circuit 802 can acquire data, program instructions, and / or information from the internal storage area of ​​the chip system; it can also acquire data, program instructions, and / or information from outside the chip system.

[0109] Optionally, the chip system also includes a second memory 803 for storing necessary computer programs and data.

[0110] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0111] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other.

[0112] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0113] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0114] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0115] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0116] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A map building method characterized by comprising: The method comprises: acquiring target detection data, the target detection data being latest detection data acquired by a target sensor, the target detection data comprising a plurality of frames of point cloud data; performing data processing on the target detection data to obtain target point cloud, the target point cloud comprising feature point cloud and semantic point cloud; obtaining inter-frame poses between the plurality of frames of point cloud data according to the target point cloud and a target historical map, the target historical map being obtained by mapping according to historical detection data acquired by the target sensor, the target historical map comprising feature point cloud layer and semantic point cloud layer; updating the feature point cloud layer and the semantic point cloud layer in the target historical map according to the target point cloud and the inter-frame poses to obtain a target map.

2. The mapping method according to claim 1, wherein the data processing on the target detection data to obtain target point cloud comprises: based on first inertial measurement data of the target sensor, performing distortion correction on each frame of point cloud data in the target detection data to obtain a plurality of frames of distortion-corrected point cloud; based on second inertial measurement data of the target sensor, performing stitching on the plurality of frames of distortion-corrected point cloud to obtain local point cloud; performing feature point cloud extraction and semantic point cloud extraction on the local point cloud to obtain the target point cloud. Before the feature point cloud extraction and semantic point cloud extraction on the local point cloud to obtain the target point cloud, the method further comprises:

3. The map building method according to claim 2, wherein, performing point cloud segmentation on the local point cloud to determine ground point cloud in the local point cloud; performing feature thinning on the ground point cloud in the local point cloud to obtain key local point cloud; the feature point cloud extraction and semantic point cloud extraction on the local point cloud to obtain the target point cloud comprises: performing feature point cloud extraction and semantic point cloud extraction on the key local point cloud to obtain the target point cloud.

4. The mapping method according to any one of claims 1-3, wherein the obtaining inter-frame poses between the plurality of frames of point cloud data according to the target point cloud and a target historical map comprises: obtaining a degeneration coefficient of the target sensor according to the feature point cloud and the target historical map; determining a first registration weight of the feature point cloud and a second registration weight of the semantic point cloud according to the degeneration coefficient; performing registration on the feature point cloud and the semantic point cloud according to the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud to obtain the inter-frame poses between the plurality of frames of point cloud data.

5. The mapping method according to claim 4, wherein the determining a first registration weight of the feature point cloud and a second registration weight of the semantic point cloud according to the degeneration coefficient comprises: in the case that the degeneration coefficient is less than a degeneration coefficient threshold, increasing the weight of the feature point cloud and decreasing the weight of the semantic point cloud to obtain the first registration weight of the feature point cloud and the second registration weight of the semantic point cloud.

6. The mapping method according to any one of claims 1-3, wherein ​ ​ ​ The updating of the feature point cloud layer and the semantic point cloud layer in the target historical map according to the target point cloud and the inter-frame pose to obtain a target map comprises: splicing the feature point cloud in the target point cloud based on the inter-frame pose to obtain a spliced feature point cloud; splicing the semantic point cloud in the target point cloud based on the inter-frame pose to obtain a spliced semantic point cloud; updating the feature point cloud layer in the target historical map according to the spliced feature point cloud and updating the semantic point cloud layer in the target historical map according to the spliced semantic point cloud to obtain the target map.

7. The map building method according to any one of claims 1 to 3, characterized by, The method further comprises: determining an inter-frame constraint corresponding to the target detection data in a case where the cumulative number of frames of the obtained multiple frames of point cloud data is greater than a preset number of frames threshold; performing global optimization on the target map according to the inter-frame constraint to obtain an optimized map.

8. The map building method according to any one of claims 1 to 3, characterized by, The method further comprises: performing loop detection according to the target detection data and the target historical map to obtain a loop detection result; obtaining a loop constraint according to the loop data in a case where the loop detection result represents that there is loop data; performing global optimization on the target map according to the loop constraint to obtain an optimized map.

9. A map building apparatus characterized by comprising: comprise: an acquisition module configured to acquire target detection data, the target detection data being the latest detection data acquired by a target sensor, the target detection data comprising multiple frames of point cloud data; a first obtaining module configured to obtain target point cloud by performing data processing on the target detection data, the target point cloud comprising feature point cloud and semantic point cloud; a second obtaining module configured to obtain an inter-frame pose between the multiple frames of point cloud data according to the target point cloud and a target historical map, the target historical map being obtained by mapping according to historical detection data acquired by the target sensor, the target historical map comprising feature point cloud layer and semantic point cloud layer; a third obtaining module configured to update the feature point cloud layer and the semantic point cloud layer in the target historical map according to the target point cloud and the inter-frame pose to obtain a target map.

10. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the map building method of any one of claims 1-8 when executed.

11. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the steps of the map building method of any one of claims 1-8.

12. A computer program product, characterised in that, The computer program comprises computer program instructions which, when executed by a processor, implement the steps of the map building method of any one of claims 1-8.

13. A chip system, characterized in that the chip system comprises a processing unit and an interface circuit, the processing unit acquires program instructions through the interface circuit, and the program instructions are executed by the processing unit, and the processing unit is configured to execute the steps of the map building method of any one of claims 1-8.

Images