A labeling method, device and electronic equipment

Abstract

Embodiments of the present application provide a labeling method and device and electronic equipment. The method comprises: obtaining a target three-dimensional point cloud of a target area; mapping the target three-dimensional point cloud to a two-dimensional grid map, wherein the two-dimensional grid map is divided into a plurality of grids; and labeling obstacles in the two-dimensional grid map according to the point cloud density of each grid in the two-dimensional grid map, wherein the point cloud density is directly proportional to the number of point clouds contained in the grid. By mapping the three-dimensional point cloud to the grid map, the point cloud density in each grid can be determined, and the point cloud density of the obstacle area is relatively higher than that of the non-obstacle area, so the point cloud density of the grid can be used to label the obstacles in the two-dimensional grid map. Since manual labeling of obstacles is not required, the efficiency of obstacle labeling is higher.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a labeling method, apparatus, and electronic device. Background Technology

[0002] "Building a map first, then performing localization based on the built map" is one of the mainstream methods in autonomous driving technology. This method breaks down localization into two steps: constructing a high-precision map of the scene and achieving high-precision localization based on the high-precision map. Therefore, the accuracy of localization depends on the accuracy of the map. Ensuring the accuracy of the high-precision map is therefore crucial to the entire localization technology.

[0003] To ensure the accuracy of high-precision maps, it is necessary to accurately mark the locations of various target objects, such as obstacles, on the map. Related technologies often employ manual annotation for obstacles, which is inefficient. Summary of the Invention

[0004] The purpose of this invention is to provide an annotation method, apparatus, and electronic device to improve annotation efficiency. The specific technical solution is as follows:

[0005] A first aspect of the present invention provides an annotation method, the method comprising:

[0006] Obtain the target 3D point cloud of the target region;

[0007] The target 3D point cloud is mapped onto a 2D raster map, wherein the 2D raster map is divided into multiple raster cells;

[0008] Obstacles are marked on the two-dimensional grid map based on the point cloud density of each grid cell, wherein the point cloud density is proportional to the number of point clouds contained in the grid cell.

[0009] A second aspect of the present invention provides a labeling device, the device comprising:

[0010] The data acquisition module is used to acquire the target 3D point cloud of the target area;

[0011] A point cloud mapping module is used to map the target 3D point cloud onto a 2D raster map, wherein the 2D raster map is divided into multiple graticles;

[0012] An obstacle labeling module is used to label obstacles in the two-dimensional grid map according to the point cloud density of each grid in the two-dimensional grid map, wherein the point cloud density is proportional to the number of point clouds contained in the grid.

[0013] A third aspect of the present invention provides an electronic device comprising:

[0014] Memory, used to store computer programs;

[0015] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0017] Beneficial effects of the embodiments of the present invention:

[0018] The annotation method, apparatus, and electronic device provided in this invention can determine the point cloud density within each grid cell by mapping a three-dimensional point cloud to a grid map. The point cloud density in obstacle areas is relatively higher than that in non-obstacle areas; therefore, obstacles can be annotated in a two-dimensional grid map based on the grid-based point cloud density. Since manual annotation of obstacles is unnecessary, the efficiency of obstacle annotation is higher.

[0019] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0021] Figure 1 A flowchart illustrating the annotation method provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram illustrating the principle of the vertical angle evaluation method applied to the annotation method provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the point cloud map filtering effect applied to the annotation method provided in an embodiment of the present invention;

[0024] Figure 4 A flowchart illustrating an obstacle grid determination method provided in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the obstacle and parking space marking structure provided in an embodiment of the present invention;

[0026] Figure 6A flowchart illustrating another annotation method provided in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the effect of the target 3D point cloud obtained by mapping according to an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of the annotation device provided in an embodiment of the present invention;

[0029] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0031] To more clearly illustrate the annotation method provided by the embodiments of the present invention, an exemplary application scenario of the annotation method provided by the embodiments of the present invention will be described below. It should be understood that the following example is only one possible application scenario of the annotation method provided by the embodiments of the present invention. In other possible embodiments, the annotation method provided by the embodiments of the present invention can also be applied to other possible application scenarios. The following example does not impose any limitations on this.

[0032] For vehicles equipped with autonomous driving capabilities (hereinafter referred to as autonomous vehicles), LiDAR can be used to scan the surrounding scene, thereby constructing a local point cloud of the area where the autonomous vehicle is located. By registering the local point cloud with a pre-established global point cloud, the autonomous vehicle can determine its own position in the global coordinate system. Combining its own position with the positions of marked obstacles in the global coordinate system, the autonomous vehicle can rationally plan its driving route to avoid collisions with obstacles.

[0033] In related technologies, obstacle labeling is often done manually. On the one hand, manual labeling is inefficient. On the other hand, manually labeling obstacles requires first creating a map, and then manually labeling the obstacles on the map before positioning can be performed based on the map. This results in a lack of seamless connection between map creation and positioning.

[0034] Based on this, embodiments of the present invention provide an annotation method, see [link to relevant documentation]. Figure 1 , Figure 1 The diagram shown is a flowchart of an annotation method provided by an embodiment of the present invention, including:

[0035] S101, acquire the target 3D point cloud of the target area.

[0036] S102 maps the target's 3D point cloud onto a 2D raster map.

[0037] The two-dimensional raster map is divided into multiple raster cells.

[0038] S103, mark obstacles in the two-dimensional grid map according to the point cloud density of each grid in the two-dimensional grid map.

[0039] Point cloud density is directly proportional to the number of point clouds contained within a raster.

[0040] Using this embodiment, the point cloud density within each grid cell can be determined by mapping the 3D point cloud to a grid map. Since the point cloud density in the area where the obstacle is located is relatively higher than in other areas, the obstacle can be marked on the 2D grid map based on the grid-based point cloud density. Because manual obstacle marking is unnecessary, the efficiency of obstacle marking is higher.

[0041] On the other hand, since the annotation method provided in this embodiment of the invention can automatically annotate obstacles based on the target 3D point cloud, it enables subsequent positioning based on the annotated obstacles, thus achieving a seamless connection between map establishment and positioning.

[0042] The following will explain S101-S103 respectively:

[0043] In S101, the method for acquiring the target 3D point cloud can vary depending on the application scenario, but the accuracy of the acquired target 3D point cloud needs to be higher than a preset accuracy threshold so that obstacles can be accurately labeled subsequently based on the target 3D point cloud. Furthermore, the target 3D point cloud can be constructed by the execution subject of the labeling method provided in this embodiment based on laser frames obtained from LiDAR scanning, or it can be constructed by other devices besides the execution subject based on laser frames; this embodiment does not impose any restrictions on this.

[0044] The following section will provide an example of how to obtain a 3D point cloud. Please refer to the relevant explanation on how to obtain a target 3D point cloud below, which will not be repeated here.

[0045] In S102, the target 3D point cloud can be regarded as a 3D map and the 2D raster map is a 2D map. Therefore, the process of mapping the target 3D point cloud to the 2D raster map can be regarded as projecting the points in the target 3D point cloud onto a 2D plane. Depending on the application scenario, the 2D plane can be a plane parallel to the ground or a plane with a certain angle to the ground. This embodiment does not impose any restrictions on this.

[0046] Furthermore, the mapping can be done by mapping all point clouds in the target 3D point cloud to a 2D raster map, or by mapping only a portion of the point clouds in the target 3D point cloud to a 2D raster map. For example, in one possible embodiment, only point clouds within the space of interest in the target 3D point cloud are mapped to the 2D raster map. The space of interest can be a space defined based on actual needs and / or experience. For instance, if someone skilled in the art can determine based on experience that only a certain area in the target 3D point cloud contains obstacles, then that area can be designated as the space of interest. In another possible embodiment, only point clouds in the target 3D point cloud that are not part of the ground are mapped to the 2D raster map. In yet another possible embodiment, only point clouds in the target 3D point cloud that belong to the space of interest but are not part of the ground are mapped to the 2D raster map.

[0047] For cases where the ground-related point clouds in the target 3D point cloud are not mapped to a 2D raster map, it is necessary to remove the ground-related point clouds from the target 3D point cloud. This process is called ground filtering. For ease of description, the target 3D point cloud after ground filtering is referred to as the ground-filtered 3D point cloud in this paper.

[0048] In one possible embodiment, the point cloud's status as belonging to the ground can be determined based on its coordinates and the planar representation of the ground, thus achieving ground filtering. In another possible embodiment, for each point cloud in each point band of the target 3D point cloud, the perpendicular angle between the point cloud and the ground point cloud in the point band is calculated. If the perpendicular angle is less than a preset angle threshold, the point cloud is determined to belong to the ground.

[0049] Here, the dot band is a set of point clouds with the same horizontal orientation within the same laser frame, and the ground point cloud is the point cloud that has been identified as belonging to the ground. For example, such as... Figure 2 As shown, Figure 2 The point clouds in the middle belong to the same point band, and this point band includes point P. a P b and P c For ease of description, P will be... a The coordinates are denoted as {x a y a , z a}, and P b The coordinates are denoted as {x b y b , z b}, P c The coordinates are denoted as {x c y c , z c}, where P a For ground point clouds, P b and P cThis is a point cloud that needs to be determined to be a ground-based element.

[0050] Then for P b Calculate the connection line P according to the following formula. a P b The perpendicular angle θ ab :

[0051]

[0052] in,

[0053]

[0054] Assuming a preset angle threshold Th θ =10.0°, if θ ab Less than Th θ Then determine P b This represents a point cloud belonging to the ground. If θ ab Greater than Th θ Then determine P b It is not a point cloud belonging to the ground.

[0055] Correspondingly, for P b For point clouds belonging to the ground, P can be connected. b and P c Two-point cloud, calculate the connecting line P b P c The perpendicular angle, thus determining P. c Is it a point cloud belonging to the ground? For P b For point clouds that do not belong to the ground, P can be connected. a and P c Two-point cloud, calculate the connecting line P a P c The perpendicular angle, thus determining P. c Is it a point cloud belonging to the ground?

[0056] For information on the effectiveness of filtration treatment, please refer to [link / reference]. Figure 3 ,in, Figure 3 The upper part is the target's 3D point cloud. Figure 3 The lower half is the three-dimensional point cloud after filtering.

[0057] In S103, the method of marking obstacles can vary depending on the application scenario, but the following conditions should be met: the higher the point cloud density of the raster, the higher the probability that it belongs to the obstacle area.

[0058] For example, in one possible embodiment, obstacles are marked in a two-dimensional grid map in the following way: Figure 4 As shown, it includes:

[0059] S1031, Based on the point cloud density of each grid in the two-dimensional grid map, determine the obstacle grids belonging to the obstacle area in the two-dimensional grid map.

[0060] In one possible embodiment, all grids with point cloud density greater than a preset threshold are identified as obstacle grids. For example, assuming that the size of each grid in the two-dimensional grid map is the same, the point cloud density can be represented by the number of point clouds included in the grid. The number of point clouds contained in each grid is proportional to the point cloud density of each grid. In this example, all grids containing at least 2 points are identified as obstacle grids.

[0061] In another possible embodiment, grids with point cloud density greater than a preset density threshold are identified in the two-dimensional grid map as occupied grids. For each occupied grid, if there is at least one occupied grid in the neighborhood of the occupied grid, the occupied grid and the grids in the neighborhood are identified as obstacle grids.

[0062] For example, assuming the density threshold of the two-dimensional grid map is 2 point clouds per grid, and the grids in the neighborhood of each grid can be the 8 grids around that grid (in other possible embodiments, the grids in the neighborhood can also be other shapes, such as four grids in the top, bottom, left, and right, or two grids in the top and bottom), then when determining the obstacle grid, it is necessary to determine all grids in the two-dimensional grid map that contain at least 2 point clouds as occupied grids. For each occupied grid, it is necessary to search whether there is an occupied grid in the grids in the neighborhood of the occupied grid, i.e., the 8 grids around it. If there is, then the occupied grid and the 8 grids around the occupied grid are determined as obstacle grids.

[0063] It is understandable that obstacles have a certain size. Therefore, if a grid cell is an obstacle grid cell, theoretically, there should also be obstacle grid cells in its neighborhood. However, if a grid cell is an obstacle grid cell and there are no obstacle grid cells in its neighborhood, it is considered that the obstacle grid cell was obtained due to the misidentification of noise in the point cloud. Therefore, by using this embodiment, the probability of misidentifying point cloud noise and incorrectly judging the point cloud density of a grid cell can be reduced by checking whether there are other occupying grid cells around the occupied grid cell. This reduces the probability of incorrectly identifying grid cells with inaccurate point cloud density as obstacle grid cells, thereby improving the accuracy of the identified obstacle grid cells and thus more accurately labeling obstacles.

[0064] S1032, divide the obstacle grid into at least one obstacle grid group.

[0065] The partitioning method can vary depending on the application scenario, but it should satisfy the following condition: the distance between any two obstacle grids within each obstacle grid group is less than a preset distance threshold. For example, in one possible embodiment, the obstacle grids can be clustered to obtain at least one obstacle grid group. In this example, each obstacle grid group contains at least N obstacle grids, and the distance between any two obstacle grids within each obstacle grid group is less than a preset distance threshold, where N is any positive integer, such as 3, 4, 6, etc.

[0066] The distance between two obstacle grids can refer to the straight-line distance between their centers, the Manhattan distance, or the number of grids separating them. Depending on the form of the distance between the obstacle grids, the preset distance threshold can vary. For example, the preset distance threshold can be 10cm, 15cm, 30cm, or it can be 1 grid, 2 grids, 3 grids, etc.

[0067] In one possible embodiment, the DBSCAN algorithm can be used to cluster the obstacle grids in the two-dimensional grid map to obtain obstacle grid groups.

[0068] S1033, for each obstacle grid group, determine the minimum convex hull region that encloses the obstacle grid group.

[0069] S1034 marks the smallest convex hull region as an obstacle.

[0070] It is understandable that the obstacle grid group obtained through division is not necessarily an obstacle. Therefore, to further improve the accuracy of annotation, prior knowledge can be used to further determine whether an obstacle grid group is an obstacle. For example, if the size of the obstacle grid group meets the preset prior size conditions, the convex hull region containing the obstacle grid group is labeled as an obstacle. As an example, the convex hull of the obstacle grid group can be drawn based on the Graham algorithm (a convex hull algorithm) to obtain the minimum convex hull region corresponding to the obstacle grid group. For example, in a scenario where the obstacles are mainly vehicles, since the horizontal dimension of a vehicle is often not less than 2m and the vertical dimension is often not less than 4m, in this example, the minimum convex hull region containing the obstacle grid group with a horizontal dimension of not less than 2m and a vertical dimension of not less than 4m is labeled as an obstacle.

[0071] For example, see Figure 5 ,in Figure 5 The convex hull area in the upper part is the obstacle area marked as perpendicular to the obstacle vehicle. Figure 5 The convex hull area in the lower half is the obstacle area marked as horizontal to the obstacle vehicle.

[0072] In the aforementioned application scenarios of autonomous vehicles, in addition to planning their routes reasonably based on marked obstacles, autonomous vehicles also need to park based on marked parking spaces in some application scenarios. Therefore, in one possible embodiment, the marking method provided by this invention is as follows: Figure 6 The diagram also includes:

[0073] S104, determine the area between two adjacent obstacles as the free area.

[0074] In this article, two adjacent obstacles refer to two obstacles where there are no other obstacles between them. For example, let's take... Figure 5 Taking the upper part as an example, the first obstacle from the left and the second obstacle from the left are adjacent obstacles, the second obstacle from the left and the third obstacle from the left are adjacent obstacles, and so on.

[0075] S105. If the size of the vacant area is smaller than the preset parking space size, mark the vacant area as a parking space.

[0076] In the aforementioned S104, the method for determining the free area may vary depending on the application scenario. For example, in one possible embodiment, the first boundary of the first obstacle and the second boundary of the second obstacle are determined, and the area between the first boundary and the second boundary is determined as the free area.

[0077] Wherein, the first obstacle and the second obstacle are any two adjacent obstacles, and the first obstacle and the second obstacle are located on different sides of the first boundary, and on different sides of the second boundary. The first boundary and the second boundary can be extracted in any way. For example, the first boundary can be the boundary line of the first obstacle obtained by fitting the boundary of the minimum convex hull region of the first obstacle near the second obstacle with the point cloud density of the grid, and the second boundary can be the boundary line of the second obstacle obtained by fitting the boundary of the minimum convex hull region of the second obstacle near the first obstacle with the point cloud density of the grid.

[0078] Specifically, during the fitting process, convergence conditions for fitting the boundary lines of obstacles can be set based on prior information about the obstacle's size. If the fitted line does not meet the convergence conditions for the obstacle, it is used as prior information in the next boundary line fitting process. If the fitted line meets the convergence conditions for the obstacle, it is designated as the obstacle boundary line. It can be understood that the boundary lines of two adjacent obstacles (the first boundary and the second boundary) can form a free area. If this free area is larger than the preset parking space size, it is marked as a parking space.

[0079] As an example, when fitting the boundary line of an obstacle, a grid grayscale image can be generated based on the obstacle grid group contained in the minimum convex hull region corresponding to the obstacle. The grayscale calculation formula for each grid is as follows:

[0080]

[0081] Among them, D i D represents the point cloud density value within the current cell of the obstacle grid group. max This represents the maximum point cloud density value of each grid cell in the entire obstacle grid group. Horizontal and vertical grayscale thresholds are set for the grayscale values ​​of each grid cell in the obstacle grid group. These thresholds can be set by those skilled in the art based on actual needs or experience; this invention does not limit this. Obstacle boundaries are cropped from the generated grayscale image based on the grayscale thresholds and the grayscale values ​​of each grid cell. For example, in the horizontal direction, if the grayscale value of a grid cell is lower than its horizontally set threshold, it indicates that the point cloud density of that grid cell is too low and may not belong to the area where the obstacle is located. Therefore, this grid cell can be cropped when cropping the obstacle boundary.

[0082] Based on the Hough transform, the previously extracted obstacle boundaries are extracted as straight lines. If the point cloud density of the grid outside the obstacle is much lower than that inside the obstacle, and the difference between the length of the extracted line and the prior values ​​of the obstacle boundary dimensions (e.g., horizontal dimension not less than 2m and vertical dimension not less than 4m) is within a certain threshold range, then the extracted line can be considered an obstacle boundary line. Otherwise, the extracted line is used as a range boundary, and the next boundary extraction will be based on this range boundary. For example, if the extracted line is used as a range boundary, then in the next obstacle boundary line extraction, a line should be extracted outside this range boundary for it to be considered an obstacle boundary line. Furthermore, as mentioned earlier, after the obstacle boundary lines are extracted, the obstacle boundary lines of two adjacent obstacles (i.e., the first boundary and the second boundary) form a free area.

[0083] In the aforementioned S105, it can be understood that since there are no other obstacles between two adjacent obstacles, if the open area between the two obstacles is larger than the preset parking space size, a vehicle can be parked in that open area, and therefore, that open area can be marked as a parking space. The marked parking spaces can be as follows: Figure 5 As shown, Figure 5 Area 1 in the upper part is the marked perpendicular parking space. Figure 5 Area 2 in the lower part is the marked horizontal parking space. Using this embodiment, parking spaces can be further marked on top of the marked obstacles, improving the applicability of the marking method.

[0084] The following is an exemplary description of how to obtain a target 3D point cloud. In one possible embodiment, the aforementioned S101 includes:

[0085] S1011, acquire multiple laser frames obtained from LiDAR scanning of the target area and the poses of each laser frame.

[0086] S1012: For each laser frame, map the point cloud in the laser frame to the global coordinate system according to the pose of the laser frame to obtain the target 3D point cloud.

[0087] In this paper, the global coordinate system can be any coordinate system that satisfies the following condition: the coordinates of any point in real space in this coordinate system do not change with the movement of LiDAR. For example, the global coordinate system can be the world coordinate system. The specific representation of the global coordinate system can be determined according to the actual calculation needs, and this application does not impose specific limitations on the embodiments.

[0088] The following will explain S1011 and S1012 respectively:

[0089] In the aforementioned S1011, the pose of each laser frame can be obtained based on the motion data of the LiDAR measured by the inertial measurement unit (IMU). It is understood that the motion data of the LiDAR measured in real time by the IMU may have some inaccuracies due to environmental factors, etc. Therefore, the pose of each laser frame obtained based on this motion data may also have some errors. Based on this, in one possible embodiment, the pose of each of the multiple laser frames is obtained in the aforementioned S1011 in the following manner:

[0090] S10111, Based on the motion data of the LiDAR obtained by the IMU, determine the pose of the LiDAR when it acquires each laser frame, and use it as the initial pose value of the laser frame.

[0091] S10112, For each laser frame, based on the initial pose value of the laser frame, map the point cloud in the laser frame to the global coordinate system to obtain the first current point cloud corresponding to the laser frame.

[0092] S10113, For each laser frame, register the first current point cloud and the local point cloud corresponding to the laser frame to obtain the pose of the laser frame.

[0093] In S10111, as mentioned above, the IMU can measure the motion data of the LiDAR when acquiring each laser frame, thereby determining the pose of each laser frame based on the motion data of the LiDAR when acquiring each laser frame, and using the pose as the initial pose value of the laser frame.

[0094] As an example, the pose information of a historical laser frame can also be measured using an IMU. Since IMU pre-integration can output the pose transformation relationship of laser frames of adjacent nodes, the initial pose value of the current laser frame can be estimated based on the pose information of historical laser frames preceding the current laser frame and the pose transformation relationship. Specifically, IMU pre-integration can output the pose transformation matrix T of adjacent laser frame nodes. imu The pose of the previous laser history frame [R] i-1 , t i-1 If [R] is known, then the initial pose value of the current laser frame can be solved using the pose transformation formula. i_init , t i_init ]:

[0095] [R i_init , t i_init ] = T imu *[R i-1 , t i-1 ]

[0096] In one possible embodiment, the aforementioned laser history frame can also be a laser history key frame. It is understood that laser history key frames are more representative among laser history frames, and using the pose information of laser history key frames to obtain the initial pose value of the current laser frame can be more accurate. Specifically, the pose transformation relationship between the current laser history frame and the previous laser history key frame can be calculated, and this can be used as a basis to determine whether the current laser history frame is a laser history key frame.

[0097] Specifically, the rotation angle transformation Δroll, pitch angle transformation Δpitch, yaw angle transformation Δyaw, and translation transformation Δtrans between the current laser historical frame and the previous laser historical keyframe are calculated, and the rotation angle transformation threshold Th is set. roll =1°, pitch angle transformation threshold Th pitch =1°, heading angle transformation threshold Th yaw =1°, translation transformation threshold Th trans =0.5m. When the transformation amount of a certain dimension in the above transformation data is greater than the threshold corresponding to that dimension, the current laser history frame can be considered as a laser history key frame. The first laser history key frame is a known key frame, which can be specified by those skilled in the art based on their needs or experience.

[0098] In S10112, the initial pose value of the laser frame is obtained based on the motion data of the LiDAR. For example, the initial pose value of the laser frame can be the position and scanning angle of the LiDAR when it acquires the laser frame. It can be understood that the coordinate system of the initial pose value is the global coordinate system. For each laser frame, each point cloud can establish a local coordinate system in the laser frame based on its distance from the LiDAR and the angle between the point cloud and the LiDAR when it scans the point cloud. Therefore, the process of mapping the point cloud of the laser frame to the global coordinate system can be to convert the local coordinates of each point cloud in the laser frame into coordinates in the global coordinate system (i.e., the positional relationship of each point cloud in the laser frame relative to the LiDAR) based on the initial pose value of the laser frame (i.e., the positional relationship of the laser frame relative to the LiDAR), thereby obtaining the first current point cloud corresponding to the laser frame.

[0099] In S10113, the local point cloud is the point cloud within a local region of the 3D point cloud constructed based on the laser frame with known pose.

[0100] Laser frames with known poses can be obtained by scanning the surrounding environment in a region with the same operating parameters but in a different geographical location than the currently operating LiDAR. For example, a laser frame with known poses can be obtained by scanning the surrounding environment from a location within 20m of the currently operating LiDAR using the same operating parameters. It is understandable that when acquiring point cloud data for a region, simply placing the LiDAR in one location for scanning will only acquire point cloud data related to that location. Therefore, when acquiring point cloud data for a region, it is usually necessary to place the LiDAR in multiple different regions to collect data separately. The surrounding environment corresponding to the point cloud data collected by the LiDAR closer to the currently operating LiDAR will have some overlap with the surrounding environment corresponding to the point cloud data collected by the currently operating LiDAR. Therefore, laser frames collected by LiDARs within a preset Euclidean distance from the currently operating LiDAR can be used as laser frames with known poses, and then registered with the point cloud data collected by the currently operating LiDAR. Since the poses of these laser frames with known poses are fixed, the point clouds of these laser frames with known poses can be mapped to the global coordinate system based on their poses to obtain the point clouds in the local regions corresponding to these laser frames with known poses, i.e., local point clouds (also known as local maps). The specific mapping process can be found in the description of S10112, which will not be repeated here. The local region in this paper should include the region where the first current point cloud is located, and the size of the local region should be less than the preset lower limit threshold.

[0101] Specifically, when registering the first current point cloud with the local point cloud to obtain the pose of the laser frame, the offset rotation of the first current point cloud relative to the local point cloud can be determined. The initial pose value is then offset and rotated according to this offset rotation to obtain the pose of the laser frame. Here, the local point cloud refers to the point cloud within a local region of the 3D point cloud constructed based on the laser frame with a known pose. It is understandable that the local point cloud and the first current point cloud are acquired from different locations. Therefore, for the same environment and object, the local point cloud data and the first current point cloud data may acquire laser frames with different poses, resulting in different point cloud data obtained based on the pose mapping of the laser frames. Therefore, when registering the first current point cloud and the local point cloud, the transformation relationship between the initial pose value of the laser frame corresponding to the first current point cloud and the pose of the laser frame with known pose corresponding to the local point cloud is determined based on the initial pose value of the laser frame with known pose. Based on this transformation relationship, the first current point cloud is mapped to the local coordinate system where the laser frame corresponding to the local point cloud is located. In this way, the newly obtained first current point cloud and local point cloud can be compared in the same coordinate system. Then, based on the position difference between the first current point cloud and the local point cloud, the offset rotation amount between the first current point cloud and the local point cloud is determined.

[0102] The difference between the initial pose value of the laser frame corresponding to the first current point cloud and the position of the laser frame with known pose corresponding to the local point cloud is determined based on the offset rotation amount. The initial pose value of the first current point cloud is offset and rotated based on the difference to obtain the pose of the registered laser frame.

[0103] As described above, the mutual registration process of point clouds involves registering the first current point cloud with all point clouds in the local point cloud. However, this registration method is inefficient. Therefore, in one possible embodiment, the feature point cloud in the first current point cloud can be registered with the local point cloud to obtain the pose of the laser frame, thereby improving the registration efficiency. The process of determining the feature point cloud of the first current point cloud can be as follows: constructing a point cloud polarization matrix for LiDAR, and determining the feature point cloud of the first current point cloud based on this polarization matrix.

[0104] Specifically, the size of the point cloud polarization matrix is ​​determined based on the LiDAR's operating parameters; the row coordinates, column coordinates, and depth values ​​of each point cloud are determined based on their positions relative to the LiDAR during acquisition; the point cloud polarization matrix is ​​constructed based on the size of the point cloud polarization matrix and the row coordinates, column coordinates, and depth values ​​of each point cloud; the feature point clouds in the first current point cloud are determined based on the point cloud polarization matrix; and the feature point clouds are registered with the local point clouds.

[0105] When constructing the point cloud polarization matrix, its size can be determined based on the LiDAR's operating parameters. For example, the number of scan lines of the LiDAR can be used as the number of rows in the point cloud polarization matrix, and the number of columns can be determined based on the horizontal angular resolution of the LiDAR's scan lines. For instance, if the horizontal angular resolution is 0.2°, then the number of columns in the point cloud polarization matrix is ​​360° / 0.2° = 1800. It is understood that different LiDARs can have different numbers of scan lines and horizontal angular resolutions; the number of scan lines can be 16 or 15, etc., and the horizontal angular resolution can be 0.2° or 0.1°, etc., and this invention is not limited in this regard.

[0106] The coordinates of each point cloud in the first current point cloud within the point cloud polarization matrix can be determined based on the position of each point cloud during acquisition. As an example, if the LiDAR of this invention has 16 scan lines and a horizontal angular resolution of 0.2°, the position of each point cloud relative to the LiDAR can be determined when acquiring the first current point cloud. Specifically, the row coordinates of each point cloud in the point cloud polarization matrix can be determined based on the number of laser lines to which it belongs during acquisition. The column coordinates of each point cloud in the point cloud polarization matrix are calculated based on the horizontal azimuth angle θ at the time of acquisition, using the formula row = θ / 0.2°. The depth value in the point cloud polarization matrix is ​​the Euclidean distance of each point cloud from the LiDAR at the time of acquisition. This determines the coordinates and depth values ​​of each point cloud in the first current point cloud. As mentioned earlier, the size of the point cloud polarization matrix can be determined based on the LiDAR's operating parameters, and the point cloud polarization matrix is ​​constructed accordingly.

[0107] In addition to corner feature points and surface feature points, the feature point cloud of the first current point cloud can also include other feature point clouds, such as line feature points, etc. The present invention does not limit which specific type of feature point cloud to be selected.

[0108] As an example, the curvature of the first current point cloud can be calculated based on the constructed point cloud polarization matrix. Corner and surface feature points in the first current point cloud can then be selected based on this curvature. Specifically, a point in the point cloud polarization matrix is ​​selected, and this selection must satisfy the following conditions: the point has an equal number of matrix points to its left and right, and these matrix points are located in the same row as the point. Based on this, the curvature c of the point cloud i corresponding to the selected point is calculated using the following point cloud curvature calculation formula. i :

[0109]

[0110] S is the set of points distributed to the left and right of point cloud i in the polarization matrix of the point cloud, || represents the number of points in the set, r j Let r be the depth value of the j-th point cloud in S within the point cloud polarization matrix. iLet be the depth value of point cloud i in the point cloud polarization matrix. Based on the characteristic that surface feature points have small curvature while corner feature points have large curvature, curvature thresholds can be set for corner and surface feature points respectively. The calculated curvature is then filtered based on these curvature thresholds to select the feature point cloud. As an example, a curvature threshold Th is set for corner feature points. edge =1.0, Surface feature point curvature threshold Th plan =0.1, which can be understood as meaning that when the curvature of the point cloud corresponding to the selected point is greater than Th. edge If the curvature of the point cloud corresponding to the selected point is less than or equal to Th, then the point cloud is a corner feature point. plan When the curvature threshold is set, the point cloud is considered a surface feature point. It is understood that the present invention does not limit the curvature thresholds for corner feature points and surface feature points; these thresholds can be set based on the experience of those skilled in the art or actual needs.

[0111] When matching the feature point cloud of the first current point cloud with the local point cloud, for corner feature points, a point-line matching method can be used. For example, if point cloud i is a corner feature point in the current laser frame, the five closest corner feature points to point cloud i are found in the local point cloud. If these five corner feature points can be fitted into a straight line, the match is successful, and these successfully matched corner feature points are retained. For area feature points, a point-area matching method can be used. For example, if point cloud j is an area feature point in the current laser frame, the five closest area feature points to point cloud j are found in the local point cloud. If these five area feature points can be fitted into a plane, the match is successful, and these successfully matched area feature points are retained. By matching all corner and surface feature points in the first current point cloud with the local point cloud, the feature point cloud in the local point cloud that matches the feature point cloud of the first current point cloud can be retained. Based on the LM algorithm (Levenberg-Marquardt algorithm) and the feature point clouds in the first current point cloud and the local point cloud that match each other, the pose of the laser frame corresponding to the first current point cloud can be solved.

[0112] By using this embodiment, the initial pose value of the laser frame can be corrected based on the registration between the local point cloud and the first current point cloud, so as to obtain a more accurate pose of the laser frame and thus obtain a more accurate target 3D point cloud.

[0113] In the aforementioned S1012, the point cloud of the laser frame can be directly mapped to the global coordinate system based on the already obtained laser frame pose to obtain the target 3D point cloud for subsequent use. It is understood that the pose acquisition of each laser frame may have certain deviations due to environmental factors, etc., and therefore the target 3D point cloud obtained based on the laser frame pose may not be accurate enough. Based on this, in one possible embodiment, the target 3D point cloud is obtained in the aforementioned S1012 in the following way:

[0114] S10121, for each of the multiple laser frames, map the point cloud of the laser frame to the global coordinate system according to the pose of the laser frame to obtain the second current point cloud corresponding to the laser frame.

[0115] S10122, For each laser frame, register the second current point cloud corresponding to the laser frame with the global point cloud to obtain the corrected pose of the laser frame.

[0116] S10123, for each laser frame, map the point cloud in the laser frame to the preset global coordinate system according to the corrected pose of the laser frame to obtain the target three-dimensional point cloud.

[0117] In S10121, this step is similar to S10112, except that the pose of the laser frame is different. For details, please refer to the description of S10112, which will not be repeated here.

[0118] In S10122, the global point cloud is all the points in the 3D point cloud constructed based on the laser frame with known pose.

[0119] As mentioned earlier, the laser frame with known pose can be a laser frame obtained by scanning the surrounding environment in a region with the same operating parameters but in a different geographical location than the currently operating LiDAR. In this embodiment, since the global point cloud and the second current point cloud are configured, it is necessary to obtain point cloud data over as large an area as possible. Therefore, an Euclidean distance threshold Th can be set. dis =20.0m, laser frames acquired by LiDARs located more than 20m away from the currently operating LiDAR are considered as laser frames with known poses. These laser frames with known poses contain more comprehensive point cloud data for the region. As an example, the Euclidean distance threshold Th is set. dis =20.0m, obtaining the Euclidean distance to the currently operating LiDAR exceeding Th dis The 25 consecutive historical keyframes are used as laser frames with known poses. Based on their poses, each laser frame with known poses is mapped to the global coordinate system to obtain the point cloud data in the global region corresponding to each laser frame with known poses, i.e., the global point cloud. The process is similar to step S10112. For details, please refer to the description of S10112. The global region in this paper should include the region where the first current point cloud is located, and the size of the global region should not be less than the preset upper limit threshold. The preset upper limit threshold is greater than the aforementioned preset lower limit threshold.

[0120] In one possible embodiment, the feature point cloud in the point cloud data corresponding to the laser frame with known pose can be mapped to a global coordinate system to obtain the global coordinates. The method for determining the feature point cloud is similar to S10113, and details can be found in the description of S10113. By using this embodiment, using the feature point cloud of the laser frame with known pose to represent all point cloud data can reduce the computational cost of mapping the point cloud while preserving the features of the laser frame with known pose, thereby improving annotation efficiency.

[0121] Specifically, when registering the global point cloud with the second current point cloud, the method described in S10113 can be used, such as the point-line matching method and the point-surface matching method, to register the global point cloud with the second current point cloud by matching angular feature points and surface feature points, and the corrected pose of the laser frame can be obtained by solving based on the mutually matched point clouds.

[0122] In one possible embodiment, the feature point cloud in the second current point cloud can be registered with the global point cloud based on the ICP method (Iterative Closest Point), and the corrected pose of the laser frame can be obtained by solving the matching point cloud.

[0123] In S10123, this step is similar to S10112, except that the pose is different. For details, please refer to the description of S10112, which will not be repeated here.

[0124] Obtaining the target 3D point cloud solely based on the corrected pose of the laser frame may still lead to inaccuracies in the subsequently obtained target 3D point cloud if the corrected pose of the laser frame contains errors. Therefore, when mapping the point cloud in the laser frame to a preset global coordinate system based on the corrected pose of the laser frame to obtain the target 3D point cloud, in addition to the method described in S10112 above, in one possible embodiment, the following can also be performed: pose smoothing is applied to the corrected pose of each laser frame to obtain an optimized pose for each laser frame; for each laser frame, the point cloud in the laser frame is mapped to a preset global coordinate system based on the optimized pose of the laser frame to obtain the target 3D point cloud.

[0125] Pose smoothing can be achieved using mean filtering or interpolation methods. In one possible embodiment, it can also involve establishing an iSAM2 (incremental smoothing and mapping method) factor graph optimizer. The laser front-end odometry results and the corrected pose of the laser frame are fed into iSAM2. Pose smoothing is then performed using iSAM2 to obtain the optimized pose of the laser frame. Based on the optimized pose of the laser frame, the point cloud data in the laser frame is mapped to the global coordinate system to obtain the target 3D point cloud. Specifically, the mapping process can be either mapping the feature point cloud in the laser frame to the global coordinate system to obtain the target 3D point cloud, or mapping all point clouds in the laser frame to the global coordinate system to obtain the target 3D point cloud. See [link to relevant documentation]. Figure 7 , Figure 7 An image of the target 3D point cloud obtained through mapping.

[0126] By using this embodiment, the pose of the laser frame can be further corrected based on the global map to obtain a more accurate pose of the laser frame, thereby obtaining a more accurate target 3D point cloud.

[0127] In actual LiDAR-acquired laser frames, each laser frame contains multiple equal-angle data. When acquiring the target 3D point cloud based on the pose of the laser frame, it is necessary to process each angle data of the laser frame. Based on this, the present invention also provides a method for acquiring a target 3D point cloud, wherein each laser frame includes multiple angle data, and the method includes:

[0128] S1021, acquire multiple laser frames obtained by LiDAR scanning the target area and the pose of each laser frame.

[0129] This step is the same as S1011, and you can refer to the description of S1011 for details. It will not be repeated here.

[0130] S1022, for each of the multiple laser frames, determine the interval between the acquisition times of adjacent sector angle data in the laser frame.

[0131] S1023, for any laser frame with an interval not greater than a preset interval threshold, map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame to obtain the target three-dimensional point cloud.

[0132] In step S1022, as mentioned earlier, different LiDARs have different operating parameters, such as operating frequency. As an example, if the selected LiDAR operates at a frequency of 10Hz, then the acquisition time for each laser frame is 100ms. Each laser frame contains 75 equal-angle data points, so under normal circumstances, the acquisition time for each angle data point is 1.33ms. Let Δt be the acquisition time interval between adjacent angle data points. Δt is calculated based on the laser frame acquisition time and the angle data points.

[0133] In S1023, the preset interval threshold can be set according to the LiDAR's operating parameters. For example, if the acquisition time for each sector corner data is 1.33ms under normal circumstances, the time interval threshold Th can be set. t =10ms, if the calculated acquisition time interval Δt between any two sector corner data in a laser frame is greater than the threshold Th t If the interval is 10ms, it indicates that some sector data in the laser frame was not fully acquired during the acquisition process, resulting in data loss. To ensure the accuracy of the acquired data, this laser frame needs to be removed. Therefore, for each laser frame containing sector data with an interval no greater than a preset interval threshold, the point cloud in the laser frame is mapped to a preset global coordinate system based on the laser frame's pose to obtain the target's 3D point cloud. The specific method for obtaining the target's 3D point cloud can be found in the description of S1012, and will not be repeated here.

[0134] By using this embodiment, abnormal laser frames acquired by LiDAR can be filtered out, improving the accuracy of the acquired laser frame data and thus improving the accuracy of the target 3D point cloud.

[0135] In one possible embodiment, the present invention also provides a method for acquiring a target three-dimensional point cloud, wherein each laser frame includes multiple sector angle data, the method comprising:

[0136] S1031, acquire multiple laser frames obtained by LiDAR scanning the target area and the pose of each laser frame.

[0137] This step is the same as S1011, and you can refer to the description of S1011 for details. It will not be repeated here.

[0138] S1032, for each laser frame, based on the motion data of the LiDAR measured by the inertial sensor IMU, determine the pose of the LiDAR when it acquires the sector angle data in the laser frame, and use it as the pose corresponding to each sector angle data.

[0139] S1033, for each laser frame, the point cloud in each sector corner data is mapped to the same coordinate system according to the pose corresponding to each sector corner data in the laser frame, so as to obtain the point cloud in the laser frame.

[0140] S1034, for each laser frame, map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame to obtain the target three-dimensional point cloud.

[0141] This step is the same as S1012, and you can refer to the description of S1012 for details. It will not be repeated here.

[0142] In step S1032, as mentioned earlier, the laser frame consists of a certain number of equal-sector angle data. The IMU can output high-frequency pose data, such as 40Hz, 50Hz, etc. For example, an IMU with an output frequency of 50Hz can be selected to measure the motion data of the LiDAR to determine the pose of each sector angle data. Specifically, the acquisition time of each sector angle in the laser frame is determined, and an acquisition timestamp for each sector angle data is generated. The IMU can measure the motion data of the LiDAR, and the pose data of the laser frame can be determined based on this motion data. Based on this, a linear interpolation algorithm is used to calculate the pose of the coordinate system where each sector angle data in the laser frame is located based on the above acquisition timestamps and the pose data of the laser frame.

[0143] In S1033, it can be understood that the positions of each equal sector angle data in a laser frame are different. Therefore, the coordinate systems of the poses calculated for different sector angle data are also different. Therefore, to facilitate the acquisition of point cloud data for the entire laser frame, it is necessary to unify the coordinate systems of the poses of multiple sector angle data. For example, the coordinate system of each sector angle data can be unified to the coordinate system of the first sector angle data starting from the Y-axis, or the coordinate system of each sector angle data can be unified to the coordinate system of the second sector angle data starting from the Y-axis. This invention is not limited in this respect.

[0144] As an example, the pose transformation relationship of each sector corner data in the laser frame relative to the first sector corner data can be determined, where the first sector corner data is a sector corner data in the laser frame; according to the pose transformation relationship, each sector corner data in the laser frame is mapped to the coordinate system to which the pose of the first sector corner data belongs, to obtain the new pose of each sector corner data; according to the new pose of each sector corner data, the point cloud in each sector corner data is mapped to the same coordinate system, to obtain the point cloud in the laser frame.

[0145] Specifically, the coordinate system of each sector angle data can be set to the coordinate system of the first sector angle data starting from the Y-axis. That is, this sector angle data is taken as the first sector angle data. Specifically, the pose of the coordinate system of the i-th sector angle data is [R]. i_Ls , t i_LsThe pose of the coordinate system containing the first sector angle data after linear interpolation is [R]. 0_Ls , t 0_Ls] Solve for the pose transformation relationship t between the two coordinate systems. 0i_Ls ,as follows:

[0146]

[0147]

[0148] The pose X of the i-th sector angle data is determined according to the following mapping formula. i Mapping the pose of the first sector angle data to the coordinate system where i is a positive integer, we obtain the new pose X′. i :

[0149] X′ i =R 0i_Ls *X i +t 0i_Ls

[0150] In this way, the poses of each sector angle data in the laser frame are mapped to the same coordinate system, thereby re-stitching the sector angle data into a complete panoramic laser frame, completing pose compensation, and obtaining the new pose X′ based on the mapped sector angle data. i The point cloud obtained after mapping the data of each sector corner is formed by the point cloud of the laser frame.

[0151] By using this embodiment, the point cloud of the laser frame can be obtained based on the fan-angle data of the laser frame, which improves the accuracy of the laser frame point cloud data and thus improves the accuracy of the target 3D point cloud.

[0152] Corresponding to the annotation method provided in the embodiments of the present invention, the embodiments of the present invention also provide an annotation device, such as... Figure 8 As shown, it includes:

[0153] Data acquisition module 801 is used to acquire the three-dimensional point cloud obtained by scanning the target area;

[0154] Point cloud mapping module 802 is used to map the three-dimensional point cloud to a two-dimensional grid map, wherein the two-dimensional grid map is divided into multiple grids;

[0155] The obstacle labeling module 803 is used to label obstacles in the two-dimensional grid map according to the point cloud density of each grid in the two-dimensional grid map.

[0156] In one possible embodiment, each laser frame includes multiple sector angle data, and the device further includes:

[0157] The free area determination module is used to determine the area between two adjacent obstacles as the free area;

[0158] The parking space determination module is used to mark the vacant area as a parking space if the size of the vacant area is larger than the preset parking space size;

[0159] The free area determination module includes:

[0160] A boundary determination unit is used to determine a first boundary of a first obstacle and a second boundary of a second obstacle, wherein the first obstacle and the second obstacle are any two adjacent obstacles, the first obstacle and the second obstacle are located on different sides of the first boundary, and the first obstacle and the second obstacle are located on different sides of the second boundary;

[0161] A free area determination unit is used to determine the area between the first boundary and the second boundary as a free area;

[0162] The obstacle labeling module 803 includes:

[0163] A grid determination unit is used to determine the obstacle grids belonging to the obstacle region in the two-dimensional grid map based on the point cloud density of each grid in the two-dimensional grid map.

[0164] A grid group determination unit is used to divide the obstacle grid into at least one obstacle grid group, wherein the distance between any two obstacle grids in the obstacle grid group is less than a preset distance threshold.

[0165] A convex hull region determination unit is used to determine, for each of the obstacle grid groups, the minimum convex hull region that surrounds the obstacle grid group.

[0166] An obstacle labeling unit is used to label the minimum convex hull region as an obstacle;

[0167] Specifically, the obstacle labeling unit is used to: if the size of the obstacle grid group meets the preset prior size condition, label the minimum convex hull region as an obstacle;

[0168] The grid group determination unit includes:

[0169] Occupy grid sub-units are used to determine grids in the two-dimensional grid map whose point cloud density is greater than a preset density threshold, and to use them as occupied grids.

[0170] An obstacle grid determination subunit is used to determine the occupied grid and the grids in the neighborhood of each occupied grid as obstacle grids if there is at least one occupied grid in the neighborhood of the occupied grid.

[0171] The point cloud mapping module 802 includes:

[0172] The filtered point cloud determination unit is used to remove points belonging to the ground from the target three-dimensional point cloud to obtain a filtered three-dimensional point cloud.

[0173] A two-dimensional raster map determination unit is used to map the filtered three-dimensional point cloud onto a two-dimensional raster map.

[0174] The ground point calculation unit is used to calculate the perpendicular angle between the point cloud and the ground point in the point band for each point cloud in each point band of the target three-dimensional point cloud. The point band is a set of point clouds with the same horizontal orientation in the same laser frame, and the ground point cloud is a point cloud that has been determined to belong to the ground.

[0175] The ground point determination unit is used to determine that the point belongs to the point cloud of the ground if the vertical angle is less than a preset angle threshold.

[0176] The data acquisition module 801 includes:

[0177] The pose determination unit is used to acquire multiple laser frames obtained by LiDAR scanning the target area and the pose of each of the multiple laser frames.

[0178] The target 3D point cloud determination unit is used to map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame for each laser frame, so as to obtain the target 3D point cloud.

[0179] The pose determination unit includes:

[0180] The pose initial value determination subunit is used to determine the pose of the LiDAR when it acquires each laser frame based on the motion data of the LiDAR obtained by the inertial sensor IMU, and use it as the pose initial value of the laser frame.

[0181] The first point cloud determination subunit is used to map the point cloud of the laser frame to the global coordinate system for each laser frame according to the initial pose value of the laser frame, so as to obtain the first current point cloud corresponding to the laser frame.

[0182] The pose determination subunit is used to register the first current point cloud and the local point cloud corresponding to each laser frame to obtain the pose of the laser frame. The local point cloud is a point cloud in a local area of ​​a three-dimensional point cloud constructed based on a laser frame with a known pose.

[0183] The pose determination subunit is specifically used for: registering the first current point cloud and the local point cloud corresponding to the laser frame, determining the offset rotation amount of the first current point cloud relative to the local point cloud, wherein the local point cloud is a point cloud in a local region of a three-dimensional point cloud constructed based on a laser frame with a known pose; and offsetting and rotating the initial pose value according to the offset rotation amount to obtain the pose of the laser frame.

[0184] The pose determination subunit is further configured to: determine the size of the point cloud polarization matrix based on the LiDAR's operating parameters; determine the row coordinates, column coordinates, and depth values ​​of each point cloud in the first current point cloud relative to the LiDAR when acquired by the LiDAR; construct the point cloud polarization matrix based on the size of the point cloud polarization matrix and the row coordinates, column coordinates, and depth values ​​of each point cloud; determine the feature point clouds in the first current point cloud based on the point cloud polarization matrix; and register the feature point clouds with the local point clouds.

[0185] The pose determination subunit is further configured to: determine the curvature c of point cloud i in the first current point cloud using the following formula. i :

[0186]

[0187] Where S is the set of points distributed on the left and right sides of point cloud i in the point cloud polarization matrix, |S| represents the number of point sets, r_j is the depth value of the j-th point cloud in S in the point cloud polarization matrix, and r_i is the depth value of point cloud i in the point cloud polarization matrix; feature point clouds in the first current point cloud are selected according to the magnitude of the curvature of each point cloud in the first current point cloud.

[0188] The target 3D point cloud determination unit includes:

[0189] The second point cloud determination subunit is used to map the point cloud of each laser frame to the global coordinate system according to the pose of the laser frame for each of the plurality of laser frames, so as to obtain the second current point cloud corresponding to the laser frame.

[0190] The pose correction determination subunit is used to register the second current point cloud corresponding to the laser frame with the global point cloud for each laser frame to obtain the corrected pose of the laser frame, wherein the global point cloud is all the point clouds in the three-dimensional point cloud constructed based on the laser frame with known pose.

[0191] The target 3D point cloud determination subunit is used to map the point cloud in the laser frame to a preset global coordinate system according to the corrected pose of the laser frame for each laser frame, so as to obtain the target 3D point cloud.

[0192] The target 3D point cloud determination subunit is specifically used for: performing pose smoothing on the corrected pose of each laser frame to obtain the optimized pose of each laser frame; and for each laser frame, mapping the point cloud in the laser frame to a preset global coordinate system according to the optimized pose of the laser frame to obtain the target 3D point cloud.

[0193] The data acquisition module 801 further includes:

[0194] The sector angle interval determination unit is used to determine the interval between the acquisition times of sector angle data with adjacent acquisition times in each of the plurality of laser frames;

[0195] The target 3D point cloud determination unit is specifically used to: for any laser frame with an interval not greater than a preset interval threshold, map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame to obtain the target 3D point cloud;

[0196] The data acquisition module 801 further includes:

[0197] The sector angle pose determination unit is used to determine the pose of the LiDAR when it acquires each sector angle data in the laser frame for each laser frame, based on the motion data of the LiDAR measured by the inertial sensor IMU, and to use the pose corresponding to each sector angle data.

[0198] The laser frame point cloud determination unit is used to map the point cloud in each sector corner data to the same coordinate system according to the pose corresponding to each sector corner data in the laser frame for each laser frame, so as to obtain the point cloud in the laser frame.

[0199] The laser frame point cloud determination unit includes:

[0200] A sector angle pose transformation subunit is used to determine the pose transformation relationship of each sector angle data in the laser frame relative to a first sector angle data, wherein the first sector angle data is a sector angle data in the laser frame.

[0201] The new pose subunit for fan corners is used to determine the new pose of each fan corner data in the laser frame according to the pose transformation relationship. The new pose is the pose of each fan corner data in the coordinate system where the pose of the first fan corner data is located.

[0202] The sector corner mapping subunit is used to map the point cloud in each sector corner data to the same coordinate system according to the new pose of each sector corner data, so as to obtain the point cloud in the laser frame.

[0203] This invention also provides an electronic device, such as... Figure 9 As shown, it includes:

[0204] Memory 901 is used to store computer programs;

[0205] When processor 902 executes a program stored in memory 901, it performs the following steps:

[0206] Obtain the 3D point cloud obtained from the scanned target area;

[0207] The three-dimensional point cloud is mapped onto a two-dimensional raster map, wherein the two-dimensional raster map is divided into multiple raster grids;

[0208] Obstacles are marked on the two-dimensional grid map based on the point cloud density of each grid cell.

[0209] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0210] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0211] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0212] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0213] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above-described annotation methods.

[0214] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the annotation methods described in the above embodiments.

[0215] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0216] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0217] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of devices, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0218] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A labeling method, characterized in that, The method includes: Obtain the target 3D point cloud of the target region; The target 3D point cloud is mapped onto a 2D raster map, wherein the 2D raster map is divided into multiple raster cells; Based on the point cloud density of each grid in the two-dimensional grid map, obstacles are marked in the two-dimensional grid map, wherein the point cloud density is proportional to the number of point clouds contained in the grid. The method further includes: Define the area between two adjacent obstacles as the free zone; If the size of the vacant area is larger than the preset parking space size, the vacant area will be marked as a parking space; The acquisition of the target 3D point cloud of the target region includes: Acquire multiple laser frames obtained by LiDAR scanning the target area and the pose of each of the multiple laser frames; For each laser frame, the point cloud in the laser frame is mapped to a preset global coordinate system according to the pose of the laser frame to obtain the target three-dimensional point cloud; The step of obtaining the pose of each of the plurality of laser frames includes: Based on the motion data of the LiDAR obtained by the inertial sensor IMU, the pose of the LiDAR when it acquires each laser frame is determined, and used as the initial pose value of the laser frame. For each laser frame, the point cloud of the laser frame is mapped to the global coordinate system according to the initial pose value of the laser frame to obtain the first current point cloud corresponding to the laser frame; For each laser frame, the first current point cloud and the local point cloud corresponding to the laser frame are registered to obtain the pose of the laser frame. The local point cloud is a point cloud in a local region of a 3D point cloud constructed based on a laser frame with a known pose. The local region includes the region where the first current point cloud is located, and the size of the local region is less than a preset lower limit threshold. For each laser frame, mapping the point cloud in the laser frame to a preset global coordinate system based on the pose of the laser frame to obtain the target 3D point cloud includes: For each of the plurality of laser frames, the point cloud of the laser frame is mapped to the global coordinate system according to the pose of the laser frame to obtain the second current point cloud corresponding to the laser frame; For each laser frame, the second current point cloud corresponding to the laser frame is registered with the global point cloud to obtain the corrected pose of the laser frame. The global point cloud is all the point clouds in the 3D point cloud constructed based on the laser frame with known pose. The global point cloud is a point cloud in a global region, which includes the region where the first current point cloud is located. The size of the global region is not less than a preset upper limit threshold, and the preset upper limit threshold is greater than a preset lower limit threshold. For each laser frame, the point cloud in the laser frame is mapped to a preset global coordinate system according to the corrected pose of the laser frame to obtain the target three-dimensional point cloud; Each laser frame includes multiple sector angle data, and the method further includes: For each laser frame, based on the motion data of the LiDAR measured by the inertial sensor IMU, the pose of the LiDAR when it acquired each sector angle data in the laser frame is determined, and is used as the pose corresponding to each sector angle data. For each laser frame, the point cloud in each sector corner data is mapped to the same coordinate system according to the pose corresponding to each sector corner data in the laser frame, so as to obtain the point cloud in the laser frame.

2. The method according to claim 1, characterized in that, The step of marking obstacles in the two-dimensional raster map based on the point cloud density of each grid cell includes: Based on the point cloud density of each grid in the two-dimensional grid map, the obstacle grids belonging to the obstacle region are determined in the two-dimensional grid map; The obstacle grid is divided into at least one obstacle grid group, wherein the distance between any two obstacle grids in the obstacle grid group is less than a preset distance threshold. For each obstacle grid group, determine the smallest convex hull region that encloses the obstacle grid group; The minimum convex hull region is marked as an obstacle.

3. The method according to claim 2, characterized in that, The step of marking the minimum convex hull region as an obstacle includes: If the size of the obstacle grid group meets the preset prior size condition, the smallest convex hull region is marked as an obstacle.

4. The method according to claim 2, characterized in that, The step of determining the obstacle grids belonging to the obstacle region in the two-dimensional grid map based on the point cloud density of each grid in the two-dimensional grid map includes: In the two-dimensional grid map, grids with point cloud density greater than a preset density threshold are identified as occupied grids. For each occupied grid, if there is at least one occupied grid in the neighborhood of the occupied grid, the occupied grid and the grids in the neighborhood are determined as obstacle grids.

5. The method according to claim 1, characterized in that, The step of determining the area between two adjacent obstacles as a free area includes: Determine a first boundary of a first obstacle and a second boundary of a second obstacle, wherein the first obstacle and the second obstacle are any two adjacent obstacles, the first obstacle and the second obstacle are located on different sides of the first boundary, and the first obstacle and the second obstacle are located on different sides of the second boundary; The area between the first boundary and the second boundary is defined as the free area.

6. The method according to claim 1, characterized in that, The step of mapping the target 3D point cloud to a 2D raster map includes: Remove points belonging to the ground from the target 3D point cloud to obtain a filtered 3D point cloud; The filtered 3D point cloud is mapped onto a 2D raster map.

7. The method according to claim 6, characterized in that, The method further includes: For each point cloud in each point band of the target 3D point cloud, calculate the perpendicular angle between the point cloud and the ground point in the point band, where the point band is a set of point clouds with the same horizontal orientation in the same laser frame, and the ground point cloud is a point cloud that has been determined to belong to the ground. If the vertical angle is less than a preset angle threshold, the point cloud is determined to be a point cloud belonging to the ground.

8. The method according to claim 1, characterized in that, The step of registering the first current point cloud and the local point cloud corresponding to the laser frame to obtain the pose of the laser frame includes: The first current point cloud and the local point cloud corresponding to the laser frame are registered to determine the offset rotation amount of the first current point cloud relative to the local point cloud. The initial pose value is offset and rotated according to the offset rotation amount to obtain the pose of the laser frame.

9. The method according to claim 1, characterized in that, The registration of the first current point cloud and the local point cloud corresponding to the laser frame includes: The size of the point cloud polarization matrix is ​​determined based on the operating parameters of LiDAR; Based on the position of each point cloud in the first current point cloud relative to the LiDAR when it is acquired by the LiDAR, determine the row coordinates, column coordinates and depth value of each point cloud; The point cloud polarization matrix is ​​constructed based on the size of the point cloud polarization matrix and the row coordinates, column coordinates, and depth values ​​of each point cloud. The feature point cloud in the first current point cloud is determined based on the point cloud polarization matrix; The feature point cloud and the local point cloud are registered.

10. The method according to claim 9, characterized in that, Determining the feature point cloud in the first current point cloud based on the point cloud polarization matrix includes: The curvature of point cloud i in the first current point cloud is determined by the following formula. : ； Where S is the set of points distributed on the left and right sides of point cloud i in the point cloud polarization matrix. Represents the number of point sets. Let be the depth value of the j-th point cloud in S within the polarization matrix of the point cloud. Let i be the depth value of point cloud i in the polarization matrix of the point cloud; Feature points in the first current point cloud are selected based on the magnitude of the curvature of each point in the first current point cloud.

11. The method according to claim 1, characterized in that, For each laser frame, mapping the point cloud in the laser frame to a preset global coordinate system based on the corrected pose of the laser frame to obtain the target 3D point cloud includes: The corrected pose of each laser frame is smoothed to obtain the optimized pose of each laser frame. For each laser frame, the point cloud in the laser frame is mapped to a preset global coordinate system according to the optimized pose of the laser frame to obtain the target three-dimensional point cloud.

12. The method according to claim 1, characterized in that, Each laser frame includes multiple sector angle data, and the method further includes: For each of the plurality of laser frames, determine the interval between the acquisition times of adjacent sector angle data in the laser frame; For each laser frame, mapping the point cloud in the laser frame to a preset global coordinate system based on the pose of the laser frame to obtain the target 3D point cloud includes: For any laser frame whose interval is not greater than a preset interval threshold, the point cloud in the laser frame is mapped to a preset global coordinate system according to the pose of the laser frame to obtain the target three-dimensional point cloud.

13. The method according to claim 1, characterized in that, For each laser frame, mapping the point cloud in each sector corner data to the same coordinate system based on the pose corresponding to each sector corner data in the laser frame to obtain the point cloud in the laser frame includes: Determine the pose transformation relationship of each sector corner data in the laser frame relative to the first sector corner data, where the first sector corner data is a sector corner data in the laser frame; Based on the pose transformation relationship, the pose of each sector corner data in the laser frame is mapped to the coordinate system where the pose of the first sector corner data is located, so as to obtain the new pose of each sector corner data. Based on the new pose of each sector corner data, the point cloud in each sector corner data is mapped to the same coordinate system to obtain the point cloud in the laser frame.

14. A labeling device, characterized in that, The device includes: The data acquisition module is used to acquire the target 3D point cloud of the target area; A point cloud mapping module is used to map the target 3D point cloud onto a 2D raster map, wherein the 2D raster map is divided into multiple graticles; An obstacle labeling module is used to label obstacles in the two-dimensional grid map according to the point cloud density of each grid in the two-dimensional grid map, wherein the point cloud density is proportional to the number of point clouds contained in the grid. The device further includes: The free area determination module is used to determine the area between two adjacent obstacles as the free area; The parking space determination module is used to mark the vacant area as a parking space if the size of the vacant area is larger than the preset parking space size; The data acquisition module includes: The pose determination unit is used to acquire multiple laser frames obtained by LiDAR scanning the target area and the pose of each of the multiple laser frames. The target 3D point cloud determination unit is used to map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame for each laser frame, so as to obtain the target 3D point cloud. The pose determination unit includes: The pose initial value determination subunit is used to determine the pose of the LiDAR when it acquires each laser frame based on the motion data of the LiDAR obtained by the inertial sensor IMU, and use it as the pose initial value of the laser frame. The first point cloud determination subunit is used to map the point cloud of the laser frame to the global coordinate system for each laser frame according to the initial pose value of the laser frame, so as to obtain the first current point cloud corresponding to the laser frame. The pose determination subunit is used to register the first current point cloud and the local point cloud corresponding to each laser frame to obtain the pose of the laser frame. The local point cloud is a point cloud in a local region of a 3D point cloud constructed based on a laser frame with a known pose. The local region includes the region where the first current point cloud is located, and the size of the local region is less than a preset lower limit threshold. The target 3D point cloud determination unit includes: The second point cloud determination subunit is used to map the point cloud of each laser frame to the global coordinate system according to the pose of the laser frame for each of the plurality of laser frames, so as to obtain the second current point cloud corresponding to the laser frame. The pose correction determination subunit is used to register the second current point cloud and the global point cloud corresponding to each laser frame to obtain the corrected pose of the laser frame. The global point cloud is all the point clouds in the three-dimensional point cloud constructed based on the laser frames with known poses. The global point cloud is a point cloud in a global region, which includes the region where the first current point cloud is located. The size of the global region is not less than a preset upper limit threshold, and the preset upper limit threshold is greater than a preset lower limit threshold. The target 3D point cloud determination subunit is used to map the point cloud in the laser frame to a preset global coordinate system according to the corrected pose of the laser frame for each laser frame, so as to obtain the target 3D point cloud. Each laser frame includes multiple sector angle data, and the data acquisition module further includes: The sector angle pose determination unit is used to determine the pose of the LiDAR when it acquires each sector angle data in the laser frame for each laser frame, based on the motion data of the LiDAR measured by the inertial sensor IMU, and to use the pose corresponding to each sector angle data. The laser frame point cloud determination unit is used to map the point cloud in each sector corner data to the same coordinate system according to the pose corresponding to each sector corner data in the laser frame for each laser frame, so as to obtain the point cloud in the laser frame.

15. The apparatus according to claim 14, characterized in that, The free area determination module includes: A boundary determination unit is used to determine a first boundary of a first obstacle and a second boundary of a second obstacle, wherein the first obstacle and the second obstacle are any two adjacent obstacles, the first obstacle and the second obstacle are located on different sides of the first boundary, and the first obstacle and the second obstacle are located on different sides of the second boundary; A free area determination unit is used to determine the area between the first boundary and the second boundary as a free area; The obstacle labeling module includes: A grid determination unit is used to determine the obstacle grids belonging to the obstacle region in the two-dimensional grid map based on the point cloud density of each grid in the two-dimensional grid map. A grid group determination unit is used to divide the obstacle grid into at least one obstacle grid group, wherein the distance between any two obstacle grids in the obstacle grid group is less than a preset distance threshold. A convex hull region determination unit is used to determine, for each of the obstacle grid groups, the minimum convex hull region that surrounds the obstacle grid group. An obstacle labeling unit is used to label the minimum convex hull region as an obstacle; Specifically, the obstacle labeling unit is used to: if the size of the obstacle grid group meets the preset prior size condition, label the minimum convex hull region as an obstacle; The grid group determination unit includes: Occupy grid sub-units are used to determine grids in the two-dimensional grid map whose point cloud density is greater than a preset density threshold, and to use them as occupied grids. An obstacle grid determination subunit is used to determine the occupied grid and the grids in the neighborhood of each occupied grid as obstacle grids if there is at least one occupied grid in the neighborhood of the occupied grid. The point cloud mapping module includes: The filtered point cloud determination unit is used to remove points belonging to the ground from the target three-dimensional point cloud to obtain a filtered three-dimensional point cloud. A two-dimensional raster map determination unit is used to map the filtered three-dimensional point cloud onto a two-dimensional raster map. The ground point calculation unit is used to calculate the perpendicular angle between the point cloud and the ground point in the point band for each point cloud in each point band of the target three-dimensional point cloud. The point band is a set of point clouds with the same horizontal orientation in the same laser frame, and the ground point cloud is a point cloud that has been determined to belong to the ground. The ground point determination unit is used to determine that the point cloud belongs to the ground if the vertical angle is less than a preset angle threshold. The pose determination subunit is specifically used for: registering the first current point cloud and the local point cloud corresponding to the laser frame, determining the offset rotation amount of the first current point cloud relative to the local point cloud, wherein the local point cloud is a point cloud in a local region of a three-dimensional point cloud constructed based on a laser frame with a known pose; and offsetting and rotating the initial pose value according to the offset rotation amount to obtain the pose of the laser frame. The pose determination subunit is further configured to: determine the size of the point cloud polarization matrix based on the LiDAR's operating parameters; determine the row coordinates, column coordinates, and depth values ​​of each point cloud in the first current point cloud relative to the LiDAR when acquired by the LiDAR; construct the point cloud polarization matrix based on the size of the point cloud polarization matrix and the row coordinates, column coordinates, and depth values ​​of each point cloud; determine the feature point clouds in the first current point cloud based on the point cloud polarization matrix; and register the feature point clouds with the local point clouds. The pose determination subunit is further configured to: determine the curvature of point cloud i in the first current point cloud using the following formula. : ； Where S is the set of points distributed on the left and right sides of point cloud i in the point cloud polarization matrix. Represents the number of point sets. Let be the depth value of the j-th point cloud in S within the polarization matrix of the point cloud. Let i be the depth value of point cloud i in the polarization matrix of the point cloud; Feature points in the first current point cloud are selected based on the magnitude of the curvature of each point cloud in the first current point cloud. The target 3D point cloud determination subunit is specifically used for: performing pose smoothing on the corrected pose of each laser frame to obtain the optimized pose of each laser frame; and for each laser frame, mapping the point cloud in the laser frame to a preset global coordinate system according to the optimized pose of the laser frame to obtain the target 3D point cloud. The data acquisition module further includes: The sector angle interval determination unit is used to determine the interval between the acquisition times of sector angle data with adjacent acquisition times in each of the plurality of laser frames; The target 3D point cloud determination unit is specifically used to: for any laser frame with an interval not greater than a preset interval threshold, map the point cloud in the laser frame to a preset global coordinate system according to the pose of the laser frame to obtain the target 3D point cloud; The laser frame point cloud determination unit includes: The sector angle pose transformation subunit is used to determine the pose transformation relationship of each sector angle data in the laser frame relative to the first sector angle data, where the first sector angle data is a sector angle data in the laser frame. The new pose subunit of the fan corner is used to map the pose of each fan corner data in the laser frame to the coordinate system where the pose of the first fan corner data is located, according to the pose transformation relationship, so as to obtain the new pose of each fan corner data. The sector corner mapping subunit is used to map the point cloud in each sector corner data to the same coordinate system according to the new pose of each sector corner data, so as to obtain the point cloud in the laser frame.

16. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-13.

Images