Enhancement Method, Device, Storage Medium and Electronic Device for Point Cloud Data

Abstract

The present application discloses a method, apparatus, storage medium and electronic device for enhancing point cloud data. The method includes: obtaining the point cloud in a target point cloud frame to obtain the original point cloud, and segmenting the original point cloud to obtain a plurality of original point cloud units; determining a plurality of ground point cloud units from the plurality of original point cloud units, and combining the plurality of ground point cloud units into a ground point cloud; dividing the ground point cloud to obtain a plurality of grid regions, and determining the ground height of each grid region; obtaining target obstacle point cloud from historical point cloud frames, and determining the obstacle height corresponding to the target obstacle point cloud; adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain the enhanced point cloud. Through the present application, the problem in the related art that the position of the newly added obstacle point cloud is inaccurate, resulting in an increase in the false detection rate of the detection model trained with the enhanced point cloud data when detecting obstacles, is improved.

Description

Technical Field

[0001] The present application relates to the field of 3D vision. Specifically, it relates to a method, device, storage medium and electronic device for enhancing point cloud data. Background Technique

[0002] In the field of 3D vision, the enhancement of point cloud data plays an important role in object detection, such as obstacle detection.

[0003] In related technologies, the methods for detecting obstacles through point cloud data enhancement include: first, performing operations such as range filtering and data cleaning on all obstacle point clouds and their GT boxes (Ground Truth, a kind of annotation data) in the original point cloud data; then classifying the GT boxes of all point cloud files according to the obstacle category, and extracting the obstacle point clouds and their corresponding GT boxes in the point cloud files to generate a database. Data enhancement is performed on the single-piece point cloud in the database through operations such as translation, flipping, and rotation, and then the number of various types of obstacle point clouds to be introduced from the outside and the corresponding number of GT boxes are determined according to the overall point cloud statistical information of this piece of point cloud; after the quantity is confirmed, the obstacle point clouds and their corresponding GT boxes are screened in the database in turn according to the obstacle category, filtering out the external point clouds that collide with the obstacle point cloud GT boxes in the original point cloud, and determining the placement position of the external point clouds.

[0004] However, in the methods of related technologies, in the case of interference between the external point clouds and the obstacles and environmental point clouds in the current frame of point cloud, although the interfering part of the original point cloud is filtered out, the external obstacle point clouds at the placement position cannot reflect the true situation of the obstacles, and the contact between the boundaries of the external obstacle point clouds and the environmental point clouds is not natural, resulting in easy misdetection at the corresponding interfering positions. It is very difficult to place the external point clouds and their corresponding GT boxes at a relatively natural position from the ground. Even considering the ground equation, due to factors such as ground undulation, it is also very difficult to place the external point cloud boxes at reasonable positions according to the ground equation in the distance, affecting the detection effect.

[0005] Aiming at the problem that the position of the newly added obstacle point clouds in related technologies is inaccurate, resulting in an increase in the misdetection rate of the detection model trained with the enhanced point cloud data for obstacle detection, no effective solution has been proposed yet. Summary of the Invention

[0006] The main purpose of the present application is to provide a method, device, storage medium and electronic device for enhancing point cloud data, so as to solve the problem that the position of the newly added obstacle point clouds in related technologies is inaccurate, resulting in an increase in the misdetection rate of the detection model trained with the enhanced point cloud data for obstacle detection.

[0007] To achieve the above object, according to one aspect of the present application, a method for enhancing point cloud data is provided. The method includes: obtaining the point cloud in the target point cloud frame to obtain the original point cloud, and segmenting the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning the target scene; determining a plurality of ground point cloud units from the plurality of original point cloud units, and combining the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; dividing the ground point cloud to obtain a plurality of grid regions, and determining the ground height of each grid region; obtaining the target obstacle point cloud from the historical point cloud frames, and determining the obstacle height corresponding to the target obstacle point cloud; and adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain the enhanced point cloud.

[0008] Optionally, adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region includes: determining the placement area of the target obstacle point cloud in the first historical point cloud frame to obtain the historical placement area, and determining the initial placement area of the target obstacle point cloud on the grid region according to the historical placement area, where the first historical point cloud frame is a point cloud frame in the plurality of historical point cloud frames; adjusting the initial placement area according to the morphological characteristics of the target obstacle point cloud to obtain the adjusted placement area; placing the target obstacle point cloud in the adjusted placement area, and obtaining the ground projection of the target obstacle point cloud; adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and adding the target obstacle point cloud with the adjusted angle to the original point cloud.

[0009] Optionally, adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and adding the target obstacle point cloud with the adjusted angle to the original point cloud includes: determining whether the ground projection of the target obstacle point cloud completely falls within the target grid region, where the target grid region is the grid region corresponding to the adjusted placement area; in the case where the ground projection of the target obstacle point cloud completely falls within the target grid region, adding the target obstacle point cloud to the original point cloud; in the case where the ground projection of the target obstacle point cloud does not completely fall within the target grid region, polling the target obstacle point cloud in the second historical point cloud frame until the ground projection of the target obstacle point cloud in the second historical point cloud frame completely falls within the target grid region after being placed in the adjusted placement area, and adding the target obstacle point cloud in the second historical point cloud frame to the original point cloud, or until all the target obstacle point clouds in the second historical point cloud frame are traversed, where the placement positions of the target obstacle point clouds in the second historical point cloud frame and the first historical point cloud frame are different.

[0010] Optionally, segmenting the original point cloud to obtain multiple original point cloud units includes: dividing the original point cloud on a target plane at a preset angle to obtain multiple sector regions, where the target plane is a plane formed by the horizontal axis and the vertical axis in a three-dimensional space; dividing each sector region circumferentially to obtain multiple original point cloud units.

[0011] Optionally, determining multiple ground point cloud units from multiple original point cloud units includes: determining the height of each point in the original point cloud unit on the vertical axis in a three-dimensional space to obtain the heights of multiple points; calculating the average value of the heights of multiple points to obtain the first average height; obtaining the points in the original point cloud unit with a height less than the first average height to obtain ground points; and combining the ground points into a ground point cloud.

[0012] Optionally, dividing the ground point cloud to obtain multiple grid regions includes: determining the smallest rectangular region containing the ground point cloud; dividing the point cloud corresponding to the smallest rectangular region according to a grid of a preset size to obtain multiple grid regions.

[0013] Optionally, determining the ground height of each grid region includes: determining the height of each point in each grid region on the vertical axis in a three-dimensional space to obtain the heights of multiple points; calculating the average value of the heights of multiple points to obtain the second average height; and determining the second average height as the ground height of the grid region.

[0014] Optionally, the original point cloud contains obstacle point clouds of multiple categories. Obtaining the target obstacle point cloud from historical point cloud frames includes: obtaining the obstacle point cloud of a category not included in the original point cloud from a database to obtain the first obstacle point cloud, where the database stores the obstacle point clouds corresponding to multiple historical point cloud frames; determining the obstacle point cloud of a category with a quantity lower than a preset quantity in the original point cloud to obtain the obstacle point cloud of the target category, and obtaining the obstacle point cloud of the target category from the database to obtain the second obstacle point cloud; and determining the first obstacle point cloud and the second obstacle point cloud as the target obstacle point cloud.

[0015] Optionally, each obstacle point cloud and the target obstacle point cloud in the original point cloud are associated with a label, where the label is used to characterize the category and morphological features of the obstacle, and the morphological features at least include the length, width, and height of the obstacle.

[0016] To achieve the above object, according to another aspect of the present application, there is provided an apparatus for enhancing point cloud data. The apparatus includes: a segmentation unit configured to obtain the point cloud in a target point cloud frame to obtain an original point cloud, and segment the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene; a determination unit configured to determine a plurality of ground point cloud units from the plurality of original point cloud units, and combine the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; a division unit configured to divide the ground point cloud to obtain a plurality of grid regions, and determine the ground height of each grid region; an acquisition unit configured to obtain a target obstacle point cloud from a historical point cloud frame, and determine the obstacle height corresponding to the target obstacle point cloud; an addition unit configured to add the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain an enhanced point cloud.

[0017] Through the present application, the following steps are adopted: obtaining the point cloud in a target point cloud frame to obtain an original point cloud, and segmenting the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene; determining a plurality of ground point cloud units from the plurality of original point cloud units, and combining the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; dividing the ground point cloud to obtain a plurality of grid regions, and determining the ground height of each grid region; obtaining a target obstacle point cloud from a historical point cloud frame, and determining the obstacle height corresponding to the target obstacle point cloud; adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain an enhanced point cloud, which improves the problem in the related art that the position of the newly added obstacle point cloud is inaccurate, resulting in an increase in the false detection rate when the detection model trained with the enhanced point cloud data detects obstacles. By obtaining the ground point cloud information and placing the newly added point cloud on the ground in the corresponding area, it is avoided that the newly added obstacle point cloud interferes with the entity part of the point cloud in this frame, thereby achieving the effect of improving the accuracy of detecting obstacles using the point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings constituting a part of the present application are used to provide a further understanding of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0019] Figure 1 is a flowchart of a method for enhancing point cloud data according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of an apparatus for enhancing point cloud data according to an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of an electronic device provided according to an embodiment of the present application. Detailed implementation manners

[0022] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data used may be interchanged under appropriate circumstances so as to describe the embodiments of the present application here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0025] For the convenience of description, some nouns or terms related to the embodiments of the present application are described below:

[0026] Point cloud: The set of point data on the surface of an object obtained by using sensor devices such as lidar is called a point cloud;

[0027] GT box: (Ground Truth), the ground truth box, a kind of labeled data.

[0028] The present invention will be described below in combination with the preferred implementation steps. Figure 1 It is a flowchart of a method for enhancing point cloud data provided according to an embodiment of the present application. As Figure 1 shown, the method includes the following steps:

[0029] Step S102, obtain the point cloud in the target point cloud frame to obtain the original point cloud, and segment the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene.

[0030] Specifically, the target point cloud frame can be a frame of point cloud data processed during the training of the 3D perception model. The original point cloud can be the point cloud directly extracted from the target point cloud frame. The original point cloud includes various types of obstacle point clouds. When processing the original point cloud, it is necessary to segment the original point cloud into multiple original point cloud units. The target scene can be the scene of X Street in City A.

[0031] Step S104: Determine multiple ground point cloud units from the multiple original point cloud units, and combine the multiple ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information.

[0032] Specifically, in addition to including various types of obstacle point clouds, the original point cloud also includes the point cloud of the terrain in the target scene. In order to distinguish the obstacle point clouds and terrain point clouds in the original point cloud, first determine multiple ground point cloud units from the multiple original point cloud units, and these ground point cloud units are combined together to form the ground point cloud in the original point cloud.

[0033] Step S106: Divide the ground point cloud to obtain multiple grid regions, and determine the ground height of each grid region.

[0034] Specifically, after determining the ground point cloud, it is also necessary to confirm the height of the terrain at each location in the original point cloud. Since the heights of different terrains in the target scene are different, the ground point cloud is divided to obtain multiple grid regions. Each grid region can encompass one type of terrain. For example, grid region M only includes the point cloud of the street area, and grid region N only includes the point cloud of the wall area. Then, determine the ground height of each grid region.

[0035] Step S108: Obtain the target obstacle point cloud from the historical point cloud frames, and determine the obstacle height corresponding to the target obstacle point cloud.

[0036] Specifically, the historical point cloud frames can be the point cloud frames obtained for the target scene at different times. Since the obstacles in the same scene change at different times, multiple point cloud frames corresponding to different times of the target scene are obtained during model training. The target obstacle point cloud can be the obstacle point cloud of a type not included in the target point cloud frame, or the obstacle point cloud with a small number of type B obstacles in the target point cloud frame. In order to place the obstacle point cloud in the original point cloud more realistically, it is also necessary to confirm the height of the target obstacle.

[0037] For example, the target scene for the target point cloud frame acquisition is X Street in City A. At the moment when the target point cloud frame is acquired, there are fewer taxis on X Street and no traffic cones are placed on X Street. In the historical point cloud frame, there are more taxis on X Street and traffic cones are placed. When processing the target point cloud frame, in order to enhance the model accuracy, the point cloud of the taxis and their GT boxes, as well as the point cloud of the traffic cones and their GT boxes, are obtained from the historical point cloud frame.

[0038] Step S110: Add the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid area to obtain the enhanced point cloud.

[0039] Specifically, the target obstacle point cloud needs to be placed at a suitable position in the original point cloud. For example, the point cloud corresponding to a taxi needs to be placed on the point cloud corresponding to the street in the original point cloud, rather than on the point cloud corresponding to a wall or a slope.

[0040] The method for enhancing point cloud data provided by the embodiments of the present application obtains the original point cloud by acquiring the point cloud in the target point cloud frame, and divides the original point cloud to obtain multiple original point cloud units. Among them, the target point cloud frame is a frame of point cloud data obtained by scanning the target scene; determines multiple ground point cloud units from the multiple original point cloud units, and combines the multiple ground point cloud units into a ground point cloud. Among them, the ground point cloud unit is a point cloud unit representing ground information; divides the ground point cloud to obtain multiple grid areas, and determines the ground height of each grid area; obtains the target obstacle point cloud from the historical point cloud frame, and determines the obstacle height corresponding to the target obstacle point cloud; adds the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid area to obtain the enhanced point cloud, which improves the problem in the related art that the position of the newly added obstacle point cloud is inaccurate, resulting in an increase in the false detection rate when the detection model trained with the enhanced point cloud data detects obstacles. By obtaining the ground point cloud information and placing the newly added point cloud on the ground in the corresponding area, it is avoided that the newly added obstacle point cloud interferes with the entity part of the point cloud in this frame, and thus the effect of improving the accuracy of detecting obstacles by using the point cloud data is achieved.

[0041] The placement of the target obstacle point cloud needs to follow physical reality. Optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid area includes: determining the placement area of the target obstacle point cloud in the first historical point cloud frame to obtain the historical placement area, and determining the initial placement area of the target obstacle point cloud on the grid area according to the historical placement area, where the first historical point cloud frame is a point cloud frame among multiple historical point cloud frames; adjusting the initial placement area according to the morphological characteristics of the target obstacle point cloud to obtain the adjusted placement area; placing the target obstacle point cloud in the adjusted placement area and obtaining the ground projection of the target obstacle point cloud; adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and adding the target obstacle point cloud with the adjusted angle to the original point cloud.

[0042] Specifically, the first historical point cloud frame can be the point cloud frame corresponding to the extracted target obstacle point cloud, and the historical placement area can be the point cloud position corresponding to the target obstacle point cloud in the first historical point cloud frame. Since the target obstacle point cloud has a corresponding placement angle in the historical point cloud frame, and the placement angle when adding the target obstacle point cloud to the target point cloud frame may not conform to physical reality, it is necessary to adjust the initial placement area. The ground projection of the target obstacle point cloud can be the projection of the target obstacle on the ground point cloud corresponding to the original point cloud. Determine the placement interval of the obstacle point cloud according to the relationship between the number of reflected point clouds of unobstructed obstacles of the same category and the distance change. At the same time, make a fine adjustment in the orientation of the obstacle according to the morphology of the obstacle point cloud to ensure that the point cloud placement conforms to physical reality as much as possible. By adjusting the placement area of the obstacle point cloud, the target obstacle point cloud is made more in line with reality, thereby avoiding problems such as point cloud interference, the target obstacle point cloud being suspended in the air, and being embedded in the ground when the target obstacle point cloud is added to the original point cloud.

[0043] For example, the point cloud corresponding to a taxi is placed on a one-way road in the first historical point cloud frame. When placing the point cloud corresponding to the taxi in the initial placement area corresponding to the original point cloud, there is a situation where the direction of the taxi is reversed. At this time, it is necessary to adjust the placement angle of the point cloud corresponding to the taxi.

[0044] After adjusting the angle, it is necessary to confirm whether to add the target obstacle point cloud. Optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud and adding the target obstacle point cloud with the adjusted angle to the original point cloud includes: determining whether the ground projection of the target obstacle point cloud completely falls within the target grid area, where the target grid area is the grid area corresponding to the adjusted placement area; in the case where the ground projection of the target obstacle point cloud completely falls within the target grid area, adding the target obstacle point cloud to the original point cloud; in the case where the ground projection of the target obstacle point cloud does not completely fall within the target grid area, polling the target obstacle point cloud in the second historical point cloud frame until the ground projection of the target obstacle point cloud in the second historical point cloud frame completely falls within the target grid area after the target obstacle point cloud in the second historical point cloud frame is placed in the adjusted placement area, adding the target obstacle point cloud in the second historical point cloud frame to the original point cloud, or until all the target obstacle point clouds in the second historical point cloud frame are traversed, where the placement positions of the target obstacle point clouds in the second historical point cloud frame and the first historical point cloud frame are different.

[0045] Specifically, the second historical point cloud frame can be a point cloud frame that contains the target obstacle point cloud but has a different placement angle of the target obstacle point cloud. Project the target obstacle point cloud onto the ground point cloud corresponding to the point cloud of the target point cloud frame, and determine whether all the target obstacle point clouds fall entirely onto the ground point cloud. In the case where the target obstacle point cloud does not fall entirely onto the ground point cloud, it indicates that there is an obstacle or a non-ground area (such as a building, a wall, a grassland, etc.) of the point cloud of the target point cloud frame at the position where the target obstacle point cloud is located. At this time, filter this obstacle point cloud and re-extract the target obstacle point cloud of the same type from the database and re-place it. In the case where the target obstacle point cloud completely falls onto the ground point cloud, determine the ground height at the position where the target obstacle point cloud is located according to the ground grid corresponding to the geometric center of the target obstacle point cloud, and adjust the target obstacle point cloud and its corresponding GT box as a whole according to this ground height, so that the bottom surfaces of the target obstacle point cloud and its GT box coincide with the ground of the grid area as much as possible. By determining whether the target obstacle point cloud completely falls onto the ground point cloud, a suitable target obstacle point cloud is selected and added to the original point cloud, thereby improving the model training accuracy.

[0046] To distinguish the ground point cloud and the obstacle point cloud, optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, segmenting the original point cloud to obtain a plurality of original point cloud units includes: dividing the original point cloud on the target plane at a preset angle to obtain a plurality of fan-shaped regions, where the target plane is the plane formed by the horizontal axis and the vertical axis in the three-dimensional space; dividing each fan-shaped region along the circumferential direction to obtain a plurality of original point cloud units.

[0047] Specifically, the preset angle can be 1 degree, so as to divide the original point cloud into 360 fan-shaped regions. The original point cloud is divided into 360 fan-shaped regions (rays) in the XY plane at a resolution of 1 radian. Each fan-shaped region is divided into several fan-shaped ring units (cells) according to a certain length, that is, the original point cloud units. By counting the height of the point cloud on the Z-axis in each cell, the height information of the point cloud plane in this region is calculated. The point cloud in the region is filtered through the ground information of each cell, and finally the segmented ground point cloud is formed to achieve the purpose of point cloud ground segmentation. By segmenting the original point cloud to obtain the original point cloud units, it is more convenient to find the ground point cloud.

[0048] After obtaining the original point cloud units, the ground point cloud can be determined. Optionally, in the point cloud data enhancement method provided in the embodiments of the present application, determining multiple ground point cloud units from multiple original point cloud units includes: determining the height of each point in the original point cloud unit on the vertical axis in three-dimensional space to obtain the heights of multiple points; calculating the average value of the heights of multiple points to obtain the first average height; obtaining the points in the original point cloud unit with a height less than the first average height to obtain the ground points; and combining the ground points into the ground point cloud.

[0049] Specifically, the vertical axis can be the Z-axis. Determine the height values of all points in the original point cloud unit on the Z-axis, calculate the average height of all points, that is, the first average height. Take the first average height as the dividing line. The points in the original point cloud unit with a height higher than the first average height are determined as obstacle points, and the points in the original point cloud unit with a height lower than the first average height are determined as ground points. Combine the ground points in all original point cloud units together to form the ground point cloud. By determining the ground point cloud, a standard reference position can be provided when placing the target obstacle point cloud on the original point cloud, thereby improving the model training accuracy.

[0050] After determining the ground point cloud, it is necessary to divide the ground point cloud into regions. Optionally, in the point cloud data enhancement method provided in the embodiments of the present application, dividing the ground point cloud to obtain multiple grid regions includes: determining the smallest rectangular region containing the ground point cloud; dividing the point cloud corresponding to the smallest rectangular region according to a grid of a preset size to obtain multiple grid regions.

[0051] Specifically, the smallest rectangular region can be the smallest rectangular range containing the ground point cloud. Divide the entire ground point cloud according to the preset scale resolution. For example, the point cloud detection range is [a, b] on the X-axis and [-c, d] on the Y-axis, and the grid resolution is set to β. Then the ground point cloud will be divided into [(b - a) / β, (d - c) / β] (note that the selected resolution should be divisible for convenient calculation) grid regions in the X-Y plane. By dividing the grid regions, the height of the ground point cloud in different terrains can be determined more specifically, so that the placement position of the target obstacle point cloud is more accurate.

[0052] After dividing into grid regions, it is necessary to determine the ground height of each grid region. Optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, determining the ground height of each grid region includes: determining the height of each point in each grid region on the vertical axis in three-dimensional space to obtain the heights of multiple points; calculating the average value of the heights of multiple points to obtain a second average height; and determining the second average height as the ground height of the grid region.

[0053] Specifically, after grid division, calculate the average ground point cloud height value of each grid region. Count the ground point cloud information in each grid region, and use the average value of the Z-axis heights of all point clouds in each grid region as the ground height value of the grid region. By determining the ground height of each grid region, the appropriate placement position of the target obstacle point cloud in the original point cloud can be judged.

[0054] In order to enhance the point cloud data, it is necessary to add target obstacle point clouds to the original point cloud. Optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, the original point cloud contains obstacle point clouds of multiple categories. Obtaining the target obstacle point cloud from historical point cloud frames includes: obtaining the obstacle point cloud of a category not included in the original point cloud from the database to obtain a first obstacle point cloud, where the database stores the obstacle point clouds corresponding to multiple historical point cloud frames; determining the obstacle point cloud of a category with a quantity lower than a preset quantity in the original point cloud to obtain the obstacle point cloud of the target category, and obtaining the obstacle point cloud of the target category from the database to obtain a second obstacle point cloud; and determining the first obstacle point cloud and the second obstacle point cloud as the target obstacle point cloud.

[0055] Specifically, the first obstacle point cloud can be the obstacle point cloud of a type not included in the target point cloud frame. For example, if there is no obstacle of the type of cone barrel in the target point cloud frame, the point cloud of the cone barrel can be extracted from the historical point cloud frame as the first obstacle point cloud. The second obstacle point cloud can be the obstacle point cloud of a type included in the target point cloud frame but with a small quantity. For example, there are only two obstacle point clouds of the taxi type in the target point cloud frame, and normally at least 5 taxis are included in the target scene taken by the target point cloud frame. At this time, it is necessary to extract 3 obstacle point clouds of the taxi type from the historical point cloud frame as the second obstacle point cloud. First, the obstacle point clouds in the database are grouped according to different categories. And complete the random extraction of a specific number of target obstacle point clouds according to the quantity requirements of different category obstacle point clouds in the target point cloud frame. Filter the extracted point clouds, and the filtering rules include: the total number of point clouds and the position of the point clouds. Filter the obstacle point clouds that do not meet the requirements of the point cloud quantity and position and re-extract them according to the category. By screening and adding appropriate target obstacle point clouds to the original point cloud, the point cloud data of the target point cloud frame can be enhanced.

[0056] Optionally, in the method for enhancing point cloud data provided in the embodiments of the present application, each obstacle point cloud and the target obstacle point cloud in the original point cloud are associated with labels, where the labels are used to characterize the category and morphological features of the obstacles, and the morphological features at least include the length, width, and height of the obstacles.

[0057] Specifically, the label can be a GT box, and the GT box is used to annotate the category, length, width, height, and external frame of the obstacle point cloud. Adding a GT box to the point cloud can make the point cloud data more complete.

[0058] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0059] The embodiments of the present application also provide an apparatus for enhancing point cloud data. It should be noted that the apparatus for enhancing point cloud data in the embodiments of the present application can be used to execute the method for enhancing point cloud data provided in the embodiments of the present application. The following introduces the apparatus for enhancing point cloud data provided in the embodiments of the present application.

[0060] Figure 2 is a schematic diagram of the apparatus for enhancing point cloud data provided in the embodiments of the present application. As Figure 2 shown, the apparatus includes:

[0061] A segmentation unit 10, configured to obtain the point cloud in the target point cloud frame to obtain the original point cloud, and segment the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene.

[0062] A determination unit 20, configured to determine a plurality of ground point cloud units from the plurality of original point cloud units, and combine the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information.

[0063] A division unit 30, configured to divide the ground point cloud to obtain a plurality of grid regions, and determine the ground height of each grid region.

[0064] An acquisition unit 40, configured to acquire the target obstacle point cloud from the historical point cloud frame, and determine the obstacle height corresponding to the target obstacle point cloud.

[0065] An addition unit 50, configured to add the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain the enhanced point cloud.

[0066] The point cloud data enhancement device provided by the embodiment of the present application obtains the point cloud in the target point cloud frame through the segmentation unit 10 to obtain the original point cloud, and segments the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning the target scene; the determination unit 20 determines a plurality of ground point cloud units from the plurality of original point cloud units, and combines the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; the division unit 30 divides the ground point cloud to obtain a plurality of grid regions and determines the ground height of each grid region; the acquisition unit 40 acquires the target obstacle point cloud from the historical point cloud frame and determines the obstacle height corresponding to the target obstacle point cloud; the addition unit 50 adds the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain the enhanced point cloud, which improves the problem in the related art that the position of the newly added obstacle point cloud is inaccurate, resulting in an increase in the false detection rate when the detection model trained with the enhanced point cloud data detects obstacles. By acquiring the ground point cloud information and placing the newly added point cloud on the ground in the corresponding area, it is avoided that the newly added obstacle point cloud interferes with the entity part of the point cloud in this frame, thereby achieving the effect of improving the accuracy of detecting obstacles using the point cloud data.

[0067] Optionally, in the device provided by the embodiment of the present application, the addition unit 50 includes: a first determination module for determining the placement area of the target obstacle point cloud in the first historical point cloud frame to obtain the historical placement area, and determining the initial placement area of the target obstacle point cloud on the grid region according to the historical placement area, where the first historical point cloud frame is a point cloud frame among the plurality of historical point cloud frames; an adjustment module for adjusting the initial placement area according to the morphological characteristics of the target obstacle point cloud to obtain the adjusted placement area; a placement module for placing the target obstacle point cloud in the adjusted placement area and obtaining the ground projection of the target obstacle point cloud; an addition module for adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud and adding the target obstacle point cloud with the adjusted angle to the original point cloud.

[0068] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the adding module includes: a judgment sub-module, configured to judge whether the ground projection of the target obstacle point cloud entirely falls within the target grid area, where the target grid area is the grid area corresponding to the adjusted placement area; an adding sub-module, configured to add the target obstacle point cloud to the original point cloud when the ground projection of the target obstacle point cloud entirely falls within the target grid area; a polling sub-module, configured to, when the ground projection of the target obstacle point cloud does not entirely fall within the target grid area, poll the target obstacle point cloud in the second historical point cloud frame until the ground projection of the target obstacle point cloud in the second historical point cloud frame entirely falls within the target grid area after the target obstacle point cloud in the second historical point cloud frame is placed in the adjusted placement area, and add the target obstacle point cloud in the second historical point cloud frame to the original point cloud, or until all the target obstacle point clouds in the second historical point cloud frame are traversed, where the placement positions of the target obstacle point clouds in the second historical point cloud frame and the first historical point cloud frame are different.

[0069] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the segmentation unit 10 includes: a first division module, configured to divide the original point cloud on the target plane at a preset angle to obtain a plurality of fan-shaped areas, where the target plane is the plane formed by the horizontal axis and the vertical axis in the three-dimensional space; a segmentation module, configured to segment each fan-shaped area circumferentially to obtain a plurality of original point cloud units.

[0070] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the determination unit 20 includes: a second determination module, configured to determine the height of each point in the original point cloud unit on the vertical axis in the three-dimensional space to obtain the heights of a plurality of points; a first calculation module, configured to calculate the average value of the heights of the plurality of points to obtain a first average height; a first acquisition module, configured to acquire the points in the original point cloud unit with a height less than the first average height to obtain ground points; a combination module, configured to combine the ground points into a ground point cloud.

[0071] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the division unit 30 includes: a third determination module, configured to determine the smallest rectangular area containing the ground point cloud; a second division module, configured to divide the point cloud corresponding to the smallest rectangular area according to a grid of a preset size to obtain a plurality of grid areas.

[0072] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the division unit 30 further includes: a fourth determination module, configured to determine the height of each point in each grid area on the vertical axis in the three-dimensional space to obtain the heights of a plurality of points; a second calculation module, configured to calculate the average value of the heights of the plurality of points to obtain a second average height; a fifth determination module, configured to determine the second average height as the ground height of the grid area.

[0073] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, the acquisition unit 40 includes: a second acquisition module, configured to acquire obstacle point clouds of categories not included in the original point cloud from a database to obtain first obstacle point clouds, where the database stores obstacle point clouds corresponding to multiple historical point cloud frames; a sixth determination module, configured to determine obstacle point clouds of categories with a quantity lower than a preset quantity in the original point cloud to obtain obstacle point clouds of a target category, and acquire obstacle point clouds of the target category from the database to obtain second obstacle point clouds; a seventh determination module, configured to determine the first obstacle point clouds and the second obstacle point clouds as target obstacle point clouds.

[0074] Optionally, in the point cloud data enhancement device provided in the embodiments of the present application, each obstacle point cloud and the target obstacle point cloud in the original point cloud are associated with a label, where the label is used to characterize the category and morphological features of the obstacle, and the morphological features at least include the length, width, and height of the obstacle.

[0075] The above-mentioned point cloud data enhancement device includes a processor and a memory. The above-mentioned segmentation unit 10, determination unit 20, division unit 30, acquisition unit 40, addition unit 50, etc. are all stored in the memory as program units, and the corresponding functions are implemented by the processor executing the above program units stored in the memory.

[0076] The processor contains a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and the model detection accuracy can be improved and the false detection can be reduced by adjusting the kernel parameters.

[0077] The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0078] The embodiments of the present invention provide a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the point cloud data enhancement method is implemented.

[0079] The embodiments of the present invention provide a processor, where the processor is used to run a program, and when the program runs, the point cloud data enhancement method is executed.

[0080] As Figure 3 shown, the embodiments of the present application also provide an electronic device. The electronic device 301 includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, the following steps are implemented: processing of state data based on a blockchain. The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0081] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0082] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0083] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0085] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0086] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0087] A computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0089] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0090] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. An enhancement method for point cloud data, characterized in that, Including: Obtain the point cloud in the target point cloud frame to get the original point cloud, and segment the original point cloud to obtain a plurality of original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene; Determine a plurality of ground point cloud units from the plurality of original point cloud units, and combine the plurality of ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; Divide the ground point cloud to obtain a plurality of grid regions, and determine the ground height of each grid region; Obtain the target obstacle point cloud from the historical point cloud frame, and determine the obstacle height corresponding to the target obstacle point cloud; According to the obstacle height and the ground height of each grid region, add the target obstacle point cloud to the original point cloud to obtain the enhanced point cloud, where adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region includes: adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and adding the target obstacle point cloud after adjusting the angle to the original point cloud; Among them, adjusting the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and adding the target obstacle point cloud after adjusting the angle to the original point cloud includes: determining whether the ground projection of the target obstacle point cloud completely falls within the target grid region, where the target grid region is the grid region corresponding to the adjusted placement area; in the case where the ground projection of the target obstacle point cloud completely falls within the target grid region, adding the target obstacle point cloud to the original point cloud; in the case where the ground projection of the target obstacle point cloud does not completely fall within the target grid region, polling the target obstacle point cloud in the second historical point cloud frame until the ground projection of the target obstacle point cloud in the second historical point cloud frame completely falls within the target grid region after the target obstacle point cloud in the second historical point cloud frame is placed in the adjusted placement area, adding the target obstacle point cloud in the second historical point cloud frame to the original point cloud, or until all the target obstacle point clouds in the second historical point cloud frame are traversed, where the placement positions of the target obstacle point clouds in the second historical point cloud frame and the first historical point cloud frame are different.

2. The method according to claim 1, wherein Adding the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region includes: Determine the placement area of the target obstacle point cloud in the first historical point cloud frame to obtain the historical placement area, and determine the initial placement area of the target obstacle point cloud on the grid region according to the historical placement area, where the first historical point cloud frame is a point cloud frame among the plurality of historical point cloud frames; Adjust the initial placement area according to the morphological characteristics of the target obstacle point cloud to obtain the adjusted placement area; Place the target obstacle point cloud in the adjusted placement area and obtain the ground projection of the target obstacle point cloud; Adjust the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud, and add the target obstacle point cloud with the adjusted angle to the original point cloud.

3. The method according to claim 1, wherein Segment the original point cloud to obtain multiple original point cloud units, including: Divide the original point cloud on the target plane at a preset angle to obtain multiple fan-shaped regions, where the target plane is a plane formed by the horizontal axis and the vertical axis in three-dimensional space; Divide each of the fan-shaped regions circumferentially to obtain multiple of the original point cloud units.

4. The method according to claim 1, characterized in that Determine multiple ground point cloud units from multiple of the original point cloud units, including: Determine the height of each point in the original point cloud unit on the vertical axis in three-dimensional space to obtain the heights of multiple points; Calculate the average value of the heights of the multiple points to obtain the first average height; Obtain the points in the original point cloud unit with a height less than the first average height to obtain ground points; Combine the ground points into the ground point cloud.

5. The method according to claim 1, wherein Divide the ground point cloud to obtain multiple grid regions, including: Determine the smallest rectangular region containing the ground point cloud; Divide the point cloud corresponding to the smallest rectangular region according to a grid of a preset size to obtain multiple of the grid regions.

6. The method according to claim 1, wherein Determine the ground height of each of the grid regions, including: Determine the height of each point in each grid region on the vertical axis in three-dimensional space to obtain the heights of multiple points; Calculate the average value of the heights of the multiple points to obtain the second average height; Determine the second average height as the ground height of the grid region.

7. The method according to claim 1, characterized in that The original point cloud contains obstacle point clouds of multiple categories. Obtaining the target obstacle point cloud from historical point cloud frames includes: Obtain the obstacle point cloud of a category not included in the original point cloud from the database to obtain the first obstacle point cloud, where the database stores the obstacle point clouds corresponding to multiple historical point cloud frames; Determine the obstacle point cloud of a category in the original point cloud with a quantity lower than a preset quantity to obtain the obstacle point cloud of the target category, and obtain the second obstacle point cloud of the target category from the database; Determine the first obstacle point cloud and the second obstacle point cloud as the target obstacle point cloud.

8. The method according to claim 1, characterized in that, Each obstacle point cloud and the target obstacle point cloud in the original point cloud are associated with a label, where the label is used to characterize the category and morphological features of the obstacle, and the morphological features at least include the length, width, and height of the obstacle.

9. An enhancement device for point cloud data, characterized in that, Including: A segmentation unit, configured to obtain the point cloud in the target point cloud frame to obtain the original point cloud, and segment the original point cloud to obtain multiple original point cloud units, where the target point cloud frame is a frame of point cloud data obtained by scanning a target scene; A determination unit, configured to determine multiple ground point cloud units from multiple of the original point cloud units, and combine the multiple ground point cloud units into a ground point cloud, where the ground point cloud unit is a point cloud unit representing ground information; A dividing unit, configured to divide the ground point cloud to obtain a plurality of grid regions and determine the ground height of each grid region; An obtaining unit, configured to obtain target obstacle point cloud from historical point cloud frames and determine the obstacle height corresponding to the target obstacle point cloud; An adding unit, configured to add the target obstacle point cloud to the original point cloud according to the obstacle height and the ground height of each grid region to obtain an enhanced point cloud, where the adding unit includes: an adding module, configured to adjust the angle of the target obstacle point cloud according to the ground projection of the target obstacle point cloud and add the target obstacle point cloud with the adjusted angle to the original point cloud; The adding module includes: a judging sub-module, configured to judge whether the ground projection of the target obstacle point cloud entirely falls within a target grid region, where the target grid region is the grid region corresponding to the adjusted placement region; an adding sub-module, configured to add the target obstacle point cloud to the original point cloud when the ground projection of the target obstacle point cloud entirely falls within the target grid region; a polling sub-module, configured to, when the ground projection of the target obstacle point cloud does not entirely fall within the target grid region, poll the target obstacle point cloud in a second historical point cloud frame until the ground projection of the target obstacle point cloud in the second historical point cloud frame entirely falls within the target grid region after the target obstacle point cloud in the second historical point cloud frame is placed in the adjusted placement region, and add the target obstacle point cloud in the second historical point cloud frame to the original point cloud, or until all the target obstacle point clouds in the second historical point cloud frame are traversed, where the placement positions of the target obstacle point cloud in the second historical point cloud frame and the first historical point cloud frame are different.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, where the program, when running, controls the device where the non-volatile storage medium is located to execute the method for enhancing point cloud data according to any one of claims 1 to 8.

11. An electronic device, characterized in that, It includes a processor and a memory, where computer-readable instructions are stored in the memory, and the processor is configured to run the computer-readable instructions, where the computer-readable instructions, when running, execute the method for enhancing point cloud data according to any one of claims 1 to 8.

Images