Laser point cloud file generation method, device, electronic device and storage medium

Abstract

The present disclosure provides a method, device, electronic device and storage medium for generating a laser point cloud file, and belongs to the field of unmanned driving technology. The method includes: adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and is not blocked by the foreground laser point cloud, and does not conflict with the non-ground laser point cloud in the initial laser point cloud file; by removing the data points that block the background laser point cloud in the adjusted first laser point cloud, a new laser point cloud file is generated. The present disclosure continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud and is not blocked by the foreground laser point cloud, and ensures that the background laser point cloud is not blocked by deleting certain data points, thereby increasing the complexity of the new laser point cloud file and improving the accuracy of the trained laser detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of unmanned driving technology, and in particular to a method, device, electronic device and storage medium for enhancing a laser point cloud file. Background Art

[0002] With the development of driverless technology, driverless cars have become the development direction of future cars and have received widespread attention from the industry. Driverless cars detect targets by inputting the laser point cloud data reflected from the surface of the target collected by the lidar into the laser detection model. Usually, the training of the laser detection model requires a large number of labeled laser point cloud files. In addition to labeling more laser point cloud files, data enhancement is another way to obtain laser point cloud files. Compared with the labeling method, the data enhancement method is more effective and can quickly and efficiently generate labeled laser point cloud files. It also has similar applications in image detection.

[0003] The related technology mainly adopts the following method when generating a laser point cloud file: obtaining a laser point cloud file, which contains annotated laser point clouds; selecting the laser point cloud to be processed from the laser point cloud file; performing processing such as rotation and stretching on the selected laser point cloud based on a preset processing method to obtain a new laser point cloud; and generating a new laser point cloud file based on the new laser point cloud.

[0004] However, the related technology does not take into account the actual position of the processed laser point cloud in the physical world, so that the new laser point cloud occludes other objects or backgrounds, and the number of point clouds on the occluded objects or backgrounds will be reduced. If the occluded point clouds are not removed, the accuracy of the trained laser detection model will be affected. Summary of the invention

[0005] The embodiments of the present disclosure provide a method, device, electronic device and storage medium for generating a laser point cloud file, which can improve the detection accuracy of the model. The technical solution is as follows:

[0006] In a first aspect, a method for generating a laser point cloud file is provided, the method comprising:

[0007] Adding the marked first laser point cloud to an initial laser point cloud file, wherein the initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud;

[0008] Adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file;

[0009] Removing data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud;

[0010] The second laser point cloud file and the initial laser point cloud file are combined into a new laser point cloud file.

[0011] In another embodiment of the present disclosure, before adding the annotated first laser point cloud to the initial laser point cloud file, the method further includes:

[0012] Obtaining the first laser point cloud from a laser point cloud file output by a trained laser detection model; and / or,

[0013] The first laser point cloud is obtained from a manually annotated laser point cloud file.

[0014] In another embodiment of the present disclosure, the adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file, comprises:

[0015] Rotating the first laser point cloud;

[0016] Adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground;

[0017] When the first laser point cloud after height adjustment is blocked by the foreground laser point cloud, the first laser point cloud is rotated and height adjusted again until the bottom surface of the readjusted first laser point cloud is located on the ground and the readjusted first laser point cloud is not blocked by the foreground laser point cloud;

[0018] When the readjusted first laser point cloud conflicts with the non-ground laser point cloud, the readjusted first laser point cloud is rotated and height-adjusted again until the bottom surface of the readjusted first laser point cloud is located on the ground, the readjusted first laser point cloud is not blocked by the foreground laser point cloud, and the readjusted first laser point cloud does not conflict with the non-ground laser point cloud.

[0019] In another embodiment of the present disclosure, the adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground includes:

[0020] Adjusting the height of the rotated first laser point cloud, and substituting the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file;

[0021] When the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground.

[0022] In another embodiment of the present disclosure, before substituting the bottom surface data of the first laser point cloud after height adjustment into the ground function equation corresponding to the initial laser point cloud file, the method further includes:

[0023] Performing ground segmentation on the initial laser point cloud file to obtain a ground laser point cloud and the non-ground laser point cloud;

[0024] The ground function equation is obtained by fitting based on the ground laser point cloud.

[0025] In another embodiment of the present disclosure, the method further includes:

[0026] When the first laser point cloud after the height adjustment has an intersection with the foreground laser point cloud, it is determined that the first laser point cloud after the height adjustment is blocked by the foreground laser point cloud.

[0027] In another embodiment of the present disclosure, the method further includes:

[0028] When there is an intersection between the readjusted first laser point cloud and the non-terrestrial laser point cloud, it is determined that there is a conflict between the readjusted first laser point cloud and the non-terrestrial laser point cloud.

[0029] In a second aspect, a device for generating a laser point cloud file is provided, the device comprising:

[0030] An adding module, used for adding the marked first laser point cloud to an initial laser point cloud file, wherein the initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud;

[0031] an adjustment module, configured to adjust the position of the first laser point cloud in the initial laser point cloud file, so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file;

[0032] A removal module, used to remove data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud;

[0033] A composition module is used to combine the second laser point cloud file and the initial laser point cloud file into a new laser point cloud file.

[0034] In another embodiment of the present disclosure, the device further comprises:

[0035] an acquisition module, used to acquire the first laser point cloud from a laser point cloud file output by a trained laser detection model; and / or,

[0036] The acquisition module is used to acquire the first laser point cloud from a manually annotated laser point cloud file.

[0037] In another embodiment of the present disclosure, the adjustment module is used to rotate the first laser point cloud; adjust the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground; when the height-adjusted first laser point cloud is blocked by the foreground laser point cloud, re-rotate and re-adjust the height of the first laser point cloud until the bottom surface of the re-adjusted first laser point cloud is located on the ground, and the re-adjusted first laser point cloud is not blocked by the foreground laser point cloud; when the re-adjusted first laser point cloud conflicts with the non-ground laser point cloud, rotate and re-adjust the height of the re-adjusted first laser point cloud again until the bottom surface of the re-adjusted first laser point cloud is located on the ground, and the re-adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the re-adjusted first laser point cloud does not conflict with the non-ground laser point cloud.

[0038] In another embodiment of the present disclosure, the adjustment module is used to adjust the height of the rotated first laser point cloud, and substitute the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file; when the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground.

[0039] In another embodiment of the present disclosure, the device further comprises:

[0040] A segmentation module, used for performing ground segmentation on the initial laser point cloud file to obtain a ground laser point cloud and the non-ground laser point cloud;

[0041] A fitting module is used to fit the ground function equation based on the ground laser point cloud.

[0042] In another embodiment of the present disclosure, the device further comprises:

[0043] The determination module is used to determine that the first laser point cloud after the height adjustment is blocked by the foreground laser point cloud when there is an intersection between the first laser point cloud after the height adjustment and the foreground laser point cloud.

[0044] In another embodiment of the present disclosure, the device further comprises:

[0045] The determination module is used to determine that there is a conflict between the readjusted first laser point cloud and the non-terrestrial laser point cloud when there is an intersection between the readjusted first laser point cloud and the non-terrestrial laser point cloud.

[0046] In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the laser point cloud file enhancement method as described in the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the method for generating a laser point cloud file as described in the first aspect.

[0048] The technical solution provided by the embodiments of the present disclosure brings the following beneficial effects:

[0049] The situation of the first laser point cloud in the actual physical world is taken into consideration. By continuously adjusting the position of the first laser point cloud in the initial laser point cloud file, the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, which increases the complexity of the generated new laser point cloud file. The laser detection model trained based on the generated new laser point cloud file has strong generalization ability and high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 is a flow chart of a method for generating a laser point cloud file provided by an embodiment of the present disclosure;

[0052] Figure 2 is a flow chart of another method for generating a laser point cloud file provided by an embodiment of the present disclosure;

[0053] Figure 3 is a flow chart of another method for generating a laser point cloud file provided by an embodiment of the present disclosure;

[0054] Figure 4 It is a schematic diagram of the structure of a device for generating a laser point cloud file provided by an embodiment of the present disclosure;

[0055] Figure 5 A structural block diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0057] It can be understood that the terms "each", "multiple", and "any" used in the embodiments of the present disclosure include two or more, each refers to each of the corresponding multiple, and any refers to any one of the corresponding multiple. For example, the multiple words include 10 words, and each word refers to each word of the 10 words, and any word refers to any one of the 10 words.

[0058] Before executing the embodiments of the present disclosure, the terms involved in the embodiments of the present disclosure are first explained.

[0059] Data augmentation is a commonly used data preprocessing method in machine learning. For example, when the amount of data at hand is too small, some meaningful data can be artificially generated for training. This data acquisition method is low-cost and effective. In addition, when the data set used for classification has skewed data, that is, one class of samples is much more than another class, this method can be used to perform data augmentation on the smaller class of samples.

[0060] Point cloud data refers to a set of vectors in a three-dimensional coordinate system. In addition to geometric positions, point cloud data may also have color information. Color information is usually obtained by acquiring color images through a camera, and then the color information (RGB) of the pixels at the corresponding position is assigned to the corresponding data points in the point cloud. Intensity information is obtained through the echo intensity collected by the laser scanner receiving device. This intensity information is related to the surface material, roughness, incident angle direction of the target object, as well as the emission energy of the instrument, laser wavelength, etc.

[0061] Overfitting means that the hypothesis becomes too strict in order to obtain a consistent hypothesis. Avoiding overfitting is a core task in classifier design. To avoid overfitting, the performance of the classifier is usually evaluated by increasing the amount of data and the test sample set.

[0062] The present disclosure provides a method for generating a laser point cloud file. Figure 1 , the method flow provided by the embodiment of the present disclosure includes:

[0063] 101. Add the marked first laser point cloud to the initial laser point cloud file.

[0064] The initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud.

[0065] 102. Adjust the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file.

[0066] 103. Remove the data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud.

[0067] 104. The second laser point cloud and the initial laser point cloud file are combined into a new laser point cloud file.

[0068] The method provided by the embodiment of the present disclosure takes into account the situation of the first laser point cloud in the actual physical world, and continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, and the complexity of the generated new laser point cloud file is increased. The laser detection model trained based on the generated new laser point cloud file has a strong generalization ability and a high detection accuracy of the model.

[0069] In another embodiment of the present disclosure, before adding the annotated first laser point cloud to the initial laser point cloud file, the method further includes:

[0070] Obtaining a first laser point cloud from a laser point cloud file output by a trained laser detection model; and / or,

[0071] A first laser point cloud is obtained from a manually annotated laser point cloud file.

[0072] In another embodiment of the present disclosure, adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file, includes:

[0073] Rotating the first laser point cloud;

[0074] Adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground;

[0075] When the first laser point cloud after height adjustment is blocked by the foreground laser point cloud, the first laser point cloud is rotated and height adjusted again until the bottom surface of the readjusted first laser point cloud is located on the ground and the readjusted first laser point cloud is not blocked by the foreground laser point cloud;

[0076] When the readjusted first laser point cloud conflicts with the non-ground laser point cloud, the readjusted first laser point cloud is rotated and height-adjusted again until the bottom surface of the readjusted first laser point cloud is located on the ground, the readjusted first laser point cloud is not blocked by the foreground laser point cloud, and the readjusted first laser point cloud does not conflict with the non-ground laser point cloud.

[0077] In another embodiment of the present disclosure, adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground includes:

[0078] Adjusting the height of the rotated first laser point cloud, and substituting the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file;

[0079] When the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground.

[0080] In another embodiment of the present disclosure, before substituting the bottom surface data of the first laser point cloud after height adjustment into the ground function equation corresponding to the initial laser point cloud file, the method further includes:

[0081] Perform ground segmentation on the initial laser point cloud file to obtain ground laser point cloud and non-ground laser point cloud;

[0082] Based on the ground laser point cloud, the ground function equation is fitted.

[0083] In another embodiment of the present disclosure, the method further includes:

[0084] When the first laser point cloud after the height adjustment has an intersection with the foreground laser point cloud, it is determined that the first laser point cloud after the height adjustment is blocked by the foreground laser point cloud.

[0085] In another embodiment of the present disclosure, the method further includes:

[0086] When there is an intersection between the readjusted first laser point cloud and the non-terrestrial laser point cloud, it is determined that there is a conflict between the readjusted first laser point cloud and the non-terrestrial laser point cloud.

[0087] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0088] The embodiment of the present disclosure provides a method for generating a laser point cloud file. Taking an electronic device executing the embodiment of the present disclosure as an example, the electronic device may be a terminal with certain computing capabilities, such as a smart phone, a laptop computer, a desktop computer, a tablet computer, etc. The electronic device may also be a server, which may be a separate physical device or a cluster or distributed system composed of multiple physical devices. Figure 2 , the method flow provided by the embodiment of the present disclosure includes:

[0089] 201. The electronic device obtains the marked first laser point cloud and the initial laser point cloud file.

[0090] The first laser point cloud is annotated with an object detection frame of the target object, the position orientation of the target object, the category, etc. In the autonomous driving scenario, the category includes pedestrians, small cars, electric cars, tricycles, etc. In the disclosed embodiment, the first laser point cloud can be referred to as instance.

[0091] Among them, the initial laser point cloud file is a laser point cloud file manually annotated by the user before the laser detection model training is performed, and the initial laser point cloud file includes at least one annotated laser point cloud. According to whether it is located on the ground, the at least one laser point cloud can be divided into a ground laser point cloud and a non-ground laser point cloud; according to whether the rotated first laser point cloud is blocked, the laser point cloud in the initial laser point cloud file is divided into a foreground laser point cloud and a background laser point cloud, wherein the laser point cloud that can block the rotated first laser point cloud is called a foreground laser point cloud, and the laser point cloud blocked by the rotated first laser point cloud is called a background laser point cloud.

[0092] When the electronic device obtains the first laser point cloud, it can adopt the following methods:

[0093] In the first method, the electronic device obtains a first laser point cloud from a laser point cloud file output by a trained laser detection model.

[0094] The electronic device obtains a laser point cloud file output by a trained laser detection model, wherein the laser point cloud file includes at least one labeled laser point cloud and the confidence of each labeled laser point cloud. The electronic device may select a labeled laser point cloud with a higher confidence as a first laser point cloud based on the confidence of each labeled laser point cloud. The electronic device may also select a category with a smaller number of laser point clouds as a target category based on the category of the labeled laser point clouds in the initial laser point cloud file, and then obtain a laser point cloud with the target category as the first laser point cloud from the laser point cloud file output by the laser detection model.

[0095] The disclosed embodiment increases the complexity of the generated new laser point cloud file by acquiring the first laser point cloud from the trained laser detection model, thereby solving the problem of repeated sampling in the initial laser point cloud file, which causes model overfitting.

[0096] In the second method, the electronic device obtains the first laser point cloud from a manually annotated laser point cloud file.

[0097] The electronic device can take a category with fewer laser point clouds as a target category according to the categories of the laser point clouds marked in the initial laser point cloud file, and then obtain a laser point cloud with the target category as the first laser point cloud from the manually marked laser point cloud file.

[0098] In a third manner, the electronic device obtains a first laser point cloud from a laser point cloud file output by a trained laser detection model and a manually annotated laser point cloud file.

[0099] For the method of obtaining the first laser point cloud from the laser point cloud file output by the trained laser detection model, the first method mentioned above can be referred to. For the method of obtaining the first laser point cloud from the manually annotated laser point cloud file, the second method mentioned above can be referred to, which will not be described in detail here. Of course, in addition to the above three methods, the electronic device can also obtain the first laser point cloud in other ways, which will not be described one by one here.

[0100] 202. The electronic device adds the marked first laser point cloud to the initial laser point cloud file.

[0101] In order to generate more laser point cloud files to realize the training of the laser detection model, the electronic device will add the first laser point cloud to the initial laser point cloud file. It should be noted here that when the electronic device adds the first laser point cloud to the initial laser point cloud file, it can add the first laser point cloud file that has not been processed to the initial laser point cloud file, and then use step 203 to perform rotation and height adjustment and other processing. The electronic device can also rotate the first laser point cloud first, and then add the rotated first laser point cloud to the initial laser point cloud file. Regardless of which adding method is used, after the first laser point cloud is added to the initial laser point cloud file, it is necessary to ensure that the first laser point cloud finally added to the initial laser point cloud file does not conflict with the non-ground laser point cloud in the initial laser point cloud file, and is not blocked by the foreground laser point cloud in the initial laser point cloud file, nor does it block the background laser point cloud in the initial laser point cloud file. If the first laser point cloud conflicts with the non-ground laser point cloud in the initial laser point cloud file or is blocked by the foreground laser point cloud in the initial laser point cloud file, the first laser point cloud needs to be rotated again; if the first laser point cloud blocks the background laser point cloud, the blocked points in the first laser point cloud need to be removed.

[0102] 203. The electronic device adjusts the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file.

[0103] When implementing this step, the following steps can be taken:

[0104] 2031. The electronic device rotates the first laser point cloud.

[0105] The electronic device obtains preset rotation parameters, and rotates each point in the first laser point cloud according to the rotation parameters to obtain a rotated first laser point cloud, wherein the rotation parameters include a rotation axis, a rotation direction, a rotation angle, and the like.

[0106] 2032. The electronic device adjusts the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground.

[0107] The electronic device adjusts the height of the rotated first laser point cloud, and substitutes the bottom surface data of the first laser point cloud with the adjusted height into the ground function equation corresponding to the initial laser point cloud file. When the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground. When the ground data does not satisfy the ground function equation, the electronic device continues to adjust the height of the first rotated laser point cloud until the bottom surface of the adjusted first laser point cloud is located on the ground.

[0108] The ground function equation is used to characterize the conditions satisfied by the coordinates of the laser point cloud located on the ground in the initial laser point cloud file. The ground function equation is obtained by fitting the ground laser point cloud in the initial laser point cloud file. The specific acquisition process of the ground function equation is:

[0109] In the first step, the electronic device performs ground segmentation on the initial laser point cloud file to obtain ground laser point cloud and non-ground laser point cloud.

[0110] The electronic device uses a ground point cloud segmentation algorithm to perform ground segmentation on the initial laser point cloud to obtain a ground-level laser point cloud and a non-ground laser point cloud. The ground point cloud segmentation algorithm includes the RanSaC algorithm, a segmentation algorithm based on neighboring information (for example, kdTree, OcTree, etc.), and a segmentation algorithm based on point cloud frequency (for example, DoN algorithm, etc.).

[0111] Furthermore, to facilitate the management of the segmented ground laser point cloud and non-ground laser point cloud, the electronic device can group the segmented ground laser point clouds into a ground laser point cloud set, and group the segmented non-ground laser point clouds into a non-ground laser point cloud set.

[0112] In the second step, the electronic device fits the ground function equation based on the ground laser point cloud.

[0113] Based on the coordinates of each point in the ground point cloud, the electronic device uses a fitting algorithm to fit each ground laser point cloud to obtain a ground function equation, wherein the fitting algorithm may be a least squares method or the like.

[0114] 2033. When the first laser point cloud after height adjustment is blocked by the foreground laser point cloud, the electronic device re-rotates and re-adjusts the height of the first laser point cloud until the bottom surface of the re-adjusted first laser point cloud is located on the ground and the re-adjusted first laser point cloud is not blocked by the foreground laser point cloud.

[0115] When the first laser point cloud after height adjustment has an intersection with the foreground laser point cloud, the electronic device determines that the first laser point cloud after height adjustment is blocked by the foreground laser point cloud. At this time, the electronic device needs to re-rotate the first laser point cloud and adjust the height of the re-rotated first laser point cloud until the bottom surface of the re-rotated first laser point cloud is located on the ground, or the number of rotations reaches a preset number of times. If the bottom surface of the first laser point cloud is not located on the ground when the number of rotations reaches the preset number of times, then the first laser point cloud is abandoned to be added to the initial laser point cloud file, wherein the preset number of times can be determined according to an empirical value. Then, the electronic device detects whether the readjusted first laser point cloud is blocked by the foreground laser point cloud. If the readjusted first laser point cloud is blocked by the foreground laser point cloud, the electronic device continues to rotate and adjust the height of the first laser point cloud until the bottom surface of the adjusted first laser point cloud is located on the ground, or the number of rotations reaches a preset number of times. If the bottom surface of the first laser point cloud is not located on the ground when the number of rotations reaches the preset number of times, then the first laser point cloud is abandoned to be added to the initial laser point cloud file. When the bottom surface of the adjusted first laser point cloud is located on the ground and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, the following step 2034 is executed to determine whether the readjusted first laser point cloud conflicts with the ground laser point cloud.

[0116] 2034. When the readjusted first laser point cloud conflicts with the non-ground laser point cloud, the electronic device rotates and adjusts the height of the readjusted first laser point cloud again until the bottom surface of the readjusted first laser point cloud is located on the ground, and the readjusted first laser point cloud is not blocked by the foreground laser point cloud, and the readjusted first laser point cloud does not conflict with the non-ground laser point cloud.

[0117] When the readjusted first laser point cloud and the non-terrestrial laser point cloud have an intersection, the electronic device determines that the readjusted first laser point cloud and the non-terrestrial laser point cloud conflict, and the electronic device rotates and adjusts the height of the readjusted first laser point cloud again until the bottom surface of the readjusted first laser point cloud is located on the ground, or the number of rotations reaches a preset number of times, if the bottom surface of the first laser point cloud is not located on the ground when the number of rotations reaches the preset number of times. When the bottom surface of the readjusted first laser point cloud is located on the ground, and the readjusted first laser point cloud is not blocked by the foreground laser point cloud, and the readjusted first laser point cloud does not conflict with the non-terrestrial laser point cloud, the electronic device executes step 204 to remove the data points in the adjusted first laser point cloud that block the background laser point cloud.

[0118] Furthermore, when the adjusted first laser point cloud ground surface is located on the bottom surface, is not blocked by the foreground laser point cloud, and does not conflict with the non-ground laser point cloud, the electronic device also adds the adjusted first laser point cloud to the non-ground point cloud set.

[0119] 204. The electronic device removes data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud.

[0120] Among them, the background laser point cloud includes background laser point clouds belonging to the ground laser point cloud set and the non-ground laser point cloud set respectively. The electronic device detects whether the adjusted first laser point cloud blocks the background laser point cloud in the non-ground laser point cloud set. If the adjusted first laser point cloud has an intersection with the background laser points in the non-ground laser point cloud set, it is determined that the adjusted first laser point cloud blocks the background laser point cloud in the non-ground laser point cloud set, and the electronic device removes the data point. The electronic device detects whether the adjusted first laser point cloud blocks the background laser point cloud in the ground laser point cloud set. If the adjusted first laser point cloud has an intersection with the background laser points in the ground laser point cloud set, it is determined that the adjusted first laser point cloud blocks the background laser point cloud in the ground laser point cloud set, and the electronic device removes the data point. By removing the data points in the adjusted first laser point cloud that block the background laser points in the ground laser point cloud set and the non-ground laser point cloud set, the electronic device obtains the second laser point cloud.

[0121] 205. The electronic device combines the second laser point cloud file and the initial laser point cloud file into a new laser point cloud file.

[0122] The electronic device merges the second laser point cloud with the initial laser point cloud file to obtain a new laser point cloud file. Of course, since the second laser point cloud is added to the non-terrestrial laser point cloud set, the electronic device can also merge the terrestrial laser point cloud set with the non-terrestrial laser point cloud set to obtain a new laser point cloud file.

[0123] The method provided by the embodiment of the present disclosure modifies the rule of instance fitting the ground, thereby ensuring that the instance has a higher probability of falling on the ground. Table 1 shows the improvement in the success rate of adding instances at various stages of 0 to 120 meters in the embodiment of the present disclosure compared with the existing ground enhancement method under the same hyperparameter conditions. Referring to Table 1, the success rate increased by 13.4% at 0 to 120 meters, 17.2% at 30 to 60 meters, 309.1% at 60 to 90 meters, and 1151.2% at 90 to 120 meters.

[0124] Table 1

[0125]

[0126]

[0127] Figure 3The process of generating the laser point cloud file provided by the embodiment of the present disclosure is shown in FIG. Figure 3 , the process mainly includes the following three steps:

[0128] Step 1: The electronic device obtains an instance, which is a labeled laser point cloud, and the instance is represented by instance. The instance can be a high-confidence instance output by a trained laser detection model, or a laser point cloud in a 3D box extracted from a manually labeled laser point cloud file. The laser point cloud includes instance information such as pedestrians and vehicles, and then the electronic device rotates the instance.

[0129] Step 2: The electronic device performs ground segmentation on the laser point cloud file to which the instance needs to be added, obtains a ground laser point cloud set and a non-ground laser point cloud set, and calculates the ground function equation based on the ground point laser point cloud set. Then, the electronic device adjusts the height of the instance rotated in the above step 1 so that the bottom surface of the rotated instance is on the ground. Next, the electronic device checks whether the instance is blocked by the foreground laser point cloud. If the instance is blocked by the foreground laser point cloud, the instance is re-rotated. If the instance is not blocked by the foreground laser point cloud, it is detected whether the instance conflicts with the non-ground laser point cloud set. If the instance conflicts with the non-ground laser point cloud set, the instance is re-rotated. If the instance does not conflict with the non-ground laser point cloud set, the instance is added to the non-ground point cloud set.

[0130] Step 3: In the non-terrestrial laser point cloud set, the electronic device removes the data points that are blocked by the instance on the background laser point cloud; in the terrestrial laser point cloud set, the electronic device removes the data points that are blocked by the instance on the background laser point cloud. Finally, the electronic device fuses the terrestrial laser point cloud set and the non-terrestrial laser point cloud set to generate a new laser point cloud file.

[0131] The method provided by the embodiment of the present disclosure takes into account the situation of the first laser point cloud in the actual physical world, and continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, and the complexity of the generated new laser point cloud file is increased. The laser detection model trained based on the generated new laser point cloud file has a strong generalization ability and a high detection accuracy of the model.

[0132] See also Figure 4 The embodiment of the present disclosure provides a device for generating a laser point cloud file, the device comprising:

[0133] An adding module 401 is used to add the marked first laser point cloud to an initial laser point cloud file, where the initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud;

[0134] An adjustment module 402 is used to adjust the position of the first laser point cloud in the initial laser point cloud file, so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file;

[0135] A removal module 403 is used to remove the data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud;

[0136] The composition module 404 is used to compose a new laser point cloud file by combining the second laser point cloud file and the initial laser point cloud file.

[0137] In another embodiment of the present disclosure, the device further comprises:

[0138] An acquisition module, used to acquire a first laser point cloud from a laser point cloud file output by a trained laser detection model; and / or,

[0139] The acquisition module is used to acquire a first laser point cloud from a manually annotated laser point cloud file.

[0140] In another embodiment of the present disclosure, the adjustment module 402 is used to rotate the first laser point cloud; adjust the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground; when the height-adjusted first laser point cloud is blocked by the foreground laser point cloud, re-rotate and adjust the height of the first laser point cloud until the bottom surface of the readjusted first laser point cloud is located on the ground, and the readjusted first laser point cloud is not blocked by the foreground laser point cloud; when the readjusted first laser point cloud conflicts with the non-ground laser point cloud, rotate and adjust the height of the readjusted first laser point cloud again until the bottom surface of the re-adjusted first laser point cloud is located on the ground, the re-adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the re-adjusted first laser point cloud does not conflict with the non-ground laser point cloud.

[0141] In another embodiment of the present disclosure, the adjustment module 402 is used to adjust the height of the rotated first laser point cloud, and substitute the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file; when the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground.

[0142] In another embodiment of the present disclosure, the device further comprises:

[0143] A segmentation module is used to perform ground segmentation on the initial laser point cloud file to obtain ground laser point cloud and non-ground laser point cloud;

[0144] The fitting module is used to fit the ground function equation based on the ground laser point cloud.

[0145] In another embodiment of the present disclosure, the device further comprises:

[0146] The determination module is used to determine that the first laser point cloud after the height adjustment is blocked by the foreground laser point cloud when there is an intersection between the first laser point cloud after the height adjustment and the foreground laser point cloud.

[0147] In another embodiment of the present disclosure, the device further comprises:

[0148] The determination module is used to determine that the readjusted first laser point cloud and the non-terrestrial laser point cloud conflict when there is an intersection between the readjusted first laser point cloud and the non-terrestrial laser point cloud.

[0149] In summary, the device provided by the embodiment of the present disclosure takes into account the situation of the first laser point cloud in the actual physical world, and continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, which increases the complexity of the generated new laser point cloud file, and the laser detection model trained based on the generated new laser point cloud file has a strong generalization ability and a high detection accuracy of the model.

[0150] Figure 5 The structure block diagram of an electronic device 500 provided by an exemplary embodiment of the present disclosure is shown. Generally, the device 500 includes: a processor 501 and a memory 502 .

[0151] The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0152] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction, which is used to be executed by the processor 501 to implement the method for generating a laser point cloud file provided in the method embodiment of the present disclosure.

[0153] In some embodiments, the electronic device 500 may further optionally include: a peripheral device interface 503 and at least one peripheral device. The processor 501, the memory 502 and the peripheral device interface 503 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 503 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: a power supply 504.

[0154] The peripheral device interface 503 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, the memory 502, and the peripheral device interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral device interface 503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0155] The power supply 504 is used to power various components in the electronic device 500. The power supply 504 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 504 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0156] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the electronic device 500, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0157] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the electronic device 500 to complete the above method for generating a laser point cloud file. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0158] The electronic device provided by the embodiment of the present disclosure takes into account the situation of the first laser point cloud in the actual physical world, and continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, thereby increasing the complexity of the generated new laser point cloud file, and the laser detection model trained based on the generated new laser point cloud file has a strong generalization ability and a high detection accuracy of the model.

[0159] The embodiment of the present disclosure provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement a method for generating a laser point cloud file. The computer-readable storage medium may be non-transitory. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] The computer-readable storage medium provided by the embodiment of the present disclosure takes into account the situation of the first laser point cloud in the actual physical world, and continuously adjusts the position of the first laser point cloud in the initial laser point cloud file so that the adjusted first laser point cloud does not conflict with the non-ground laser point cloud, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud. By deleting certain data points in the adjusted first laser point cloud, it is ensured that the background laser point cloud is not blocked by the adjusted first laser point cloud, thereby increasing the complexity of the generated new laser point cloud file, and the laser detection model trained based on the generated new laser point cloud file has a strong generalization ability and high detection accuracy of the model.

[0161] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0162] The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a laser point cloud file, characterized in that: The method comprises: Adding the marked first laser point cloud to an initial laser point cloud file, wherein the initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud; Adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file, and when the readjusted first laser point cloud and the non-ground laser point cloud have an intersection, it is determined that the readjusted first laser point cloud conflicts with the non-ground laser point cloud; Removing data points that block the background laser point cloud in the adjusted first laser point cloud to obtain a second laser point cloud; The second laser point cloud file and the initial laser point cloud file are combined into a new laser point cloud file.

2. The method according to claim 1, characterized in that Before adding the marked first laser point cloud to the initial laser point cloud file, the method further includes: The first laser point cloud is obtained from a laser point cloud file output by a trained laser detection model; and / or the first laser point cloud is obtained from a manually annotated laser point cloud file.

3. The method according to claim 1, characterized in that The adjusting the position of the first laser point cloud in the initial laser point cloud file so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file, includes: Rotating the first laser point cloud; Adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground; when the height-adjusted first laser point cloud is blocked by the foreground laser point cloud, re-rotating and re-adjusting the height of the first laser point cloud until the bottom surface of the re-adjusted first laser point cloud is located on the ground and the re-adjusted first laser point cloud is not blocked by the foreground laser point cloud; When the readjusted first laser point cloud conflicts with the non-ground laser point cloud, the readjusted first laser point cloud is rotated and height-adjusted again until the bottom surface of the readjusted first laser point cloud is located on the ground, the readjusted first laser point cloud is not blocked by the foreground laser point cloud, and the readjusted first laser point cloud does not conflict with the non-ground laser point cloud.

4. The method according to claim 3, characterized in that The step of adjusting the height of the rotated first laser point cloud so that the bottom surface of the adjusted first laser point cloud is located on the ground comprises: Adjusting the height of the rotated first laser point cloud, and substituting the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file; When the bottom surface data satisfies the ground function equation, it is determined that the bottom surface of the adjusted first laser point cloud is located on the ground.

5. The method according to claim 4, characterized in that Before substituting the bottom surface data of the first laser point cloud after the height adjustment into the ground function equation corresponding to the initial laser point cloud file, the method further includes: Performing ground segmentation on the initial laser point cloud file to obtain a ground laser point cloud and the non-ground laser point cloud; The ground function equation is obtained by fitting based on the ground laser point cloud.

6. The method according to claim 3, characterized in that The method further comprises: When the first laser point cloud after the height adjustment has an intersection with the foreground laser point cloud, it is determined that the first laser point cloud after the height adjustment is blocked by the foreground laser point cloud.

7. A device for generating a laser point cloud file, characterized in that: The device comprises: An adding module, used for adding the marked first laser point cloud to an initial laser point cloud file, wherein the initial laser point cloud file includes a foreground laser point cloud and a background laser point cloud; an adjustment module, configured to adjust the position of the first laser point cloud in the initial laser point cloud file, so that the bottom surface of the adjusted first laser point cloud is located on the ground of the initial laser point cloud file, and the adjusted first laser point cloud is not blocked by the foreground laser point cloud, and the adjusted first laser point cloud does not conflict with the non-ground laser point cloud in the initial laser point cloud file, and when the readjusted first laser point cloud and the non-ground laser point cloud have an intersection, it is determined that the readjusted first laser point cloud conflicts with the non-ground laser point cloud; a removal module, used to remove data points in the adjusted first laser point cloud that block the background laser point cloud, to obtain a second laser point cloud; A composition module is used to combine the second laser point cloud file and the initial laser point cloud file into a new laser point cloud file.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the laser point cloud file enhancement method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: At least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to implement the method for generating a laser point cloud file according to any one of claims 1 to 6.

Images