Point cloud labeling method and device, point cloud quality inspection method and device and computer storage medium

Abstract

The invention provides a point cloud labeling method, a point cloud quality inspection method, a point cloud quality inspection device and a computer storage medium. The point cloud labeling method comprises the following steps: acquiring a plurality of frames of to-be-labeled point cloud data; obtaining a pre-labeling label of each frame of to-be-labeled point cloud data; performing frame stacking on the plurality of frames of to-be-labeled point cloud data and the pre-labeled labels thereof to obtain frame stacking point cloud data; adding a refinement labeling label to the stacked frame point cloud data to generate refinement point cloud data; and splitting the refined point cloud data according to the number of data points of the to-be-labeled point cloud data to obtain labeling results of the plurality of frames of to-be-labeled point cloud data. Through the point cloud quality inspection method, manual fine trimming labeling is performed in a frame stacking state, and the labeling result of the point cloud data of each frame is obtained through splitting, so that the labeling speed can be effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of point cloud data processing, and particularly to a point cloud annotation method, a point cloud quality inspection method, a point cloud quality inspection device, and a computer storage medium. Background Art

[0002] In the field of autonomous driving, the development of autonomous driving technology depends on the improvement of environmental perception capabilities. Point cloud segmentation is one of the key links in an autonomous driving system, which directly affects the vehicle's understanding and decision-making of the surrounding environment. With the rise of artificial intelligence technology and the development of laser scanning technology, the amount of three-dimensional point cloud data is increasing rapidly. Facing the increasing amount of three-dimensional point cloud data, how to efficiently and accurately perform data segmentation annotation and data processing to improve the accuracy and efficiency of data annotation is one of the important issues that need to be considered urgently.

[0003] For point cloud segmentation data annotation, traditional manual annotation is time-consuming, laborious, and costly. For the point cloud semantic segmentation task, each data point in the point cloud needs to be labeled with a tag, which wastes a large amount of human and time costs of annotators compared with other three-dimensional point cloud annotation tasks. Summary of the Invention

[0004] To solve the above technical problems, this application proposes a point cloud annotation method, a point cloud quality inspection method, a point cloud quality inspection device, and a computer storage medium.

[0005] To solve the above technical problems, this application proposes a point cloud annotation method, which includes:

[0006] Obtain several frames of point cloud data to be annotated;

[0007] Obtain the pre-labeled tags of each frame of point cloud data to be annotated;

[0008] Stack the several frames of point cloud data to be annotated and their pre-labeled tags to obtain stacked point cloud data;

[0009] Add refined annotation tags to the stacked point cloud data to generate refined point cloud data;

[0010] Split the refined point cloud data according to the number of data points of the point cloud data to be annotated to obtain the annotation tag results of the several frames of point cloud data to be annotated.

[0011] Among them, obtaining the pre-labeled tags of each frame of point cloud data to be annotated includes:

[0012] Input each frame of point cloud data to be annotated into a three-dimensional segmentation model to obtain the pre-labeled category data points of each frame of point cloud data to be annotated;

[0013] Input each frame of the point cloud data to be labeled into a 3D detection model to obtain a pre-labeled detection box for each frame of the point cloud data to be labeled;

[0014] Fuse the pre-labeled category data points and the pre-labeled detection box to generate a pre-label for each frame of the point cloud data to be labeled.

[0015] Among them, the pre-labeled detection box is a pre-labeled detection box for dynamic obstacles;

[0016] The fusing the pre-labeled category data points and the pre-labeled detection box to generate a pre-label for each frame of the point cloud data to be labeled includes:

[0017] Refine the dynamic pre-labeled category data points in the pre-labeled category data points according to the pre-labeled category of the data points in the pre-labeled detection box to obtain a dynamic target segmentation result;

[0018] Fuse the dynamic target segmentation result with other pre-labeled category data points to generate a pre-label for each frame of the point cloud data to be labeled.

[0019] Among them, the stacking the several frames of point cloud data to be labeled and their pre-labels to obtain stacked point cloud data includes:

[0020] Obtain the pose information of each frame of the point cloud data to be labeled;

[0021] Convert the several frames of point cloud data to be labeled to the same coordinate system according to the pose information and perform stacking to obtain the stacked point cloud data.

[0022] To solve the above technical problems, the present application also proposes a point cloud quality inspection method, and the point cloud quality inspection method includes:

[0023] Obtain the prediction result of the point cloud data to be quality inspected;

[0024] Based on a pre-labeled annotation label file, obtain an annotation result;

[0025] Obtain the similarity threshold between the prediction result and the annotation result;

[0026] When the similarity threshold meets the quality inspection condition, add the point cloud data to be quality inspected to the training set.

[0027] Among them, the prediction result includes a dynamic target prediction result and a static target prediction result; the annotation result includes a dynamic target annotation result and a static target annotation result;

[0028] The obtaining the similarity threshold between the prediction result and the annotation result includes:

[0029] Obtain the intersection over union (IoU) of the dynamic object prediction result and the dynamic object annotation result;

[0030] Obtain the intersection over union (IoU) of the static object prediction result and the static object annotation result;

[0031] The point cloud quality inspection method further includes:

[0032] When the IoU of the dynamic object is greater than the first IoU and the IoU of the static object is greater than the second IoU, determine that the point cloud data to be quality inspected meets the quality inspection conditions.

[0033] Wherein, the dynamic object prediction result is the dynamic object prediction bounding box output by the 3D object detection model, and the dynamic object annotation result is the ground truth detection bounding box;

[0034] The static object prediction result is the dynamic object category data points output by the 3D object segmentation model, and the dynamic object prediction result is the dynamic object ground truth category points.

[0035] Wherein, the point cloud quality inspection method further includes:

[0036] When the similarity threshold does not meet the quality inspection conditions, compare the prediction result and the annotation result to determine the mispredicted data points;

[0037] Set the mispredicted data points to the first label and the remaining data points to the second label.

[0038] To solve the above technical problems, the present application also proposes a point cloud quality inspection device, which includes a memory and a processor coupled to the memory; wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the point cloud annotation method and / or the point cloud quality inspection method as described above.

[0039] To solve the above technical problems, the present application also proposes a computer storage medium, which is used to store program data, and when the program data is executed by a computer, it is used to implement the point cloud annotation method and / or the point cloud quality inspection method as described above.

[0040] Compared with the existing technology, the beneficial effects of the present application are as follows: the point cloud quality inspection device obtains several frames of point cloud data to be annotated; obtains pre-labeled labels for each frame of point cloud data to be annotated; overlaps the several frames of point cloud data to be annotated and their pre-labeled labels to obtain overlapped point cloud data; adds refined annotation labels to the overlapped point cloud data to generate refined point cloud data; and splits the refined point cloud data according to the number of data points in the point cloud data to obtain the annotation label results of the several frames of point cloud data to be annotated. Through the above-mentioned point cloud quality inspection method, manual refined annotation is performed in the overlapped state, and the annotation label results of each frame of point cloud data are obtained by splitting, which can effectively improve the annotation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] in:

[0043] Figure 1 This is a flow chart of an embodiment of the point cloud annotation method provided by this application;

[0044] Figure 2 This is a schematic diagram of the overall process of point cloud data annotation and quality inspection provided by this application;

[0045] Figure 3 yes Figure 1 The specific flow diagram of step S12 of the point cloud annotation method is shown;

[0046] Figure 4 This is a schematic diagram of the visualization of single-frame point cloud data provided by this application;

[0047] Figure 5 This is a schematic diagram of the visualization of point cloud data with 10 frames superimposed provided by this application;

[0048] Figure 6 This is a flow chart of an embodiment of the point cloud quality inspection method provided by this application;

[0049] Figure 7 It is a flowchart of the data inspection solution provided by this application;

[0050] Figure 8 This is a flow chart of another embodiment of the point cloud quality inspection method provided by this application;

[0051] Figure 9 It is a cross-visualization diagram of the risk data provided by this application;

[0052] Figure 10 FIG. 2 is a schematic structural diagram of an embodiment of the point cloud quality inspection method provided by the present application;

[0053] Figure 11 FIG. 3 is a schematic structural diagram of an embodiment of the computer storage medium provided by the present application. Detailed implementation manners

[0054] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here, for example, can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0056] To solve the problem of low efficiency of the current point cloud annotation technology, the present application provides a processing flow for lidar-based point cloud semantic segmentation data, including data preprocessing, data annotation methods, and data quality inspection and screening methods. The purpose is to overcome the existing data processing flow, accelerate the annotation speed of point cloud semantic segmentation data, improve the data turnover efficiency, and thus lay a solid data foundation for subsequent model training and optimization.

[0057] Specifically, please refer to Figure 1 and Figure 2 , Figure 1 FIG. 4 is a schematic flow diagram of an embodiment of the point cloud annotation method provided by the present application, Figure 2 FIG. 5 is a schematic overall flow diagram of point cloud data annotation and quality inspection provided by the present application.

[0058] The point cloud annotation method of the present application is applied to a point cloud quality inspection device. Among them, the point cloud quality inspection device of the present application can be a server, a terminal device, or a system in which the server and the terminal device cooperate with each other. Correspondingly, each part included in the point cloud quality inspection device, such as each unit, subunit, module, and submodule, can be all set in the server, all set in the terminal device, or separately set in the server and the terminal device.

[0059] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide a distributed server, or as a single software or software module, which is not specifically limited herein.

[0060] As Figure 1 shown, the specific steps are as follows:

[0061] Step S11: Obtain several frames of point cloud data to be annotated.

[0062] In the embodiment of the present application, the point cloud quality inspection device obtains the point cloud data to be annotated. The point cloud data processing of the present application targets the point cloud data in the field of autonomous driving, which is mainly collected by lidar. Sensors such as lidar can collect information such as the three-dimensional coordinates and reflectivity of points. The point cloud data in the present application can be expressed as (x, y, z, i), where the first three items in the parentheses represent the coordinates of the current data point, and the last item represents the intensity information of the current data point.

[0063] Step S12: Obtain the pre-label of each frame of point cloud data to be annotated.

[0064] In the embodiment of the present application, the point cloud quality inspection device uses a pre-trained 3D segmentation model or 3D detection model to automatically annotate each frame of point cloud data to be annotated, generating the pre-label of each frame of point cloud data to be annotated.

[0065] Furthermore, the point cloud quality inspection device can also use the 3D segmentation model and the 3D detection model at the same time to automatically annotate the point cloud data to be annotated, and then verify and fuse the two automatic annotation results with each other to improve the accuracy of the pre-label of the automatic annotation.

[0066] For details, please refer to Figure 3 , Figure 3 which Figure 1 is the specific process schematic diagram of step S12 of the point cloud annotation method shown.

[0067] As Figure 3 shown, the specific steps are as follows:

[0068] Step S121: Input each frame of point cloud data to be labeled into a 3D segmentation model to obtain the pre-labeled category data points of each frame of point cloud data to be labeled.

[0069] In the embodiment of the present application, the point cloud quality inspection device inputs the point cloud data to be labeled into the 3D segmentation model Seg-Model1 to obtain the pre-labeled category data points obtained by segmenting the point cloud data to be labeled, that is, each data point is labeled with a category label.

[0070] Step S122: Input each frame of point cloud data to be labeled into a 3D detection model to obtain the pre-labeled detection boxes of each frame of point cloud data to be labeled.

[0071] In the embodiment of the present application, the point cloud quality inspection device inputs the point cloud data to be labeled into the 3D detection model Det-Model1 to obtain the pre-labeled detection boxes obtained by detecting the point cloud data to be labeled, that is, detection boxes for each detection category are generated.

[0072] Both the 3D detection model Det-Model1 and the 3D segmentation model Seg-Model1 adopted in the present application use common model structures in the art. Since pre-labeling does not need to consider deployment, a model with high accuracy but difficult to deploy on the vehicle side is selected for pre-training to improve the accuracy of pre-training.

[0073] Step S123: Fuse the pre-labeled category data points and the pre-labeled detection boxes to generate the pre-labeled tags of each frame of point cloud data to be labeled.

[0074] In the embodiment of the present application, the point cloud quality inspection device fuses the above-generated pre-labeled category data points and the pre-labeled detection boxes to obtain the pre-labeled tags of the point cloud data to be labeled.

[0075] In a specific implementation manner, the point cloud quality inspection device uses the 3D detection model Det-Model1 to pre-label the detection boxes for dynamic obstacle targets (such as vehicles, people, cyclists, etc.), that is, the pre-labeled detection boxes in step S122 are the pre-labeled detection boxes for dynamic obstacles.

[0076] Then, the point cloud quality inspection device maps the pre-labeled category data points of the dynamic target according to the detection box pre-labeling result to refine the segmentation result of the dynamic target. Finally, the pre-labeling result is output.

[0077] Specifically, the point cloud quality inspection device maps the pre-labeled detection boxes of the dynamic obstacles, that is, sets the data points within the pre-labeled detection box range as the category labels of the dynamic obstacles, and then expands the range by 5% according to the pre-labeled detection box range. For the pre-labeled category data points of the dynamic obstacles that appear within the expanded range, the labels are retained, and for the pre-labeled category data points of the dynamic obstacles that appear outside the expanded range, the labels are deleted or set to empty.

[0078] Step S13: Stack several frames of point cloud data to be labeled and their preliminary labels to obtain stacked point cloud data.

[0079] In the embodiment of the present application, after the point cloud quality inspection device obtains the overall preliminary labeling result of each frame of point cloud data to be labeled through step S12, it uses the pose information to perform a stacking operation on several frames of point cloud data to be labeled and their preliminary labels.

[0080] Specifically, the pose information in the point cloud refers to the data describing the position and orientation of the point cloud relative to a certain reference coordinate system (usually the global coordinate system). The pose information is usually represented in the form of a transformation matrix, which is a 4x4 matrix and can be used to transform each data point in the point cloud data from one coordinate system to another. In the field of autonomous driving, the pose information is obtained through the Global Navigation Satellite System (GNSS) and the Inertial Measurement Unit (IMU) during data collection.

[0081] The point cloud quality inspection device uses the pose information to convert a batch of pcd (Point Cloud Data) data to be stacked to the same coordinate system using the transformation matrix, and at the same time stacks the preliminary label data. For details, please refer to Figure 4 and Figure 5 , Figure 4 is the visualization schematic diagram of a single-frame point cloud data provided by the present application, Figure 5 is the visualization schematic diagram of the stacked point cloud data of 10 frames provided by the present application.

[0082] Comparing Figure 4 and Figure 5 it can be seen that after stacking 10 frames, the data points of the point cloud data become denser. For the labeling personnel, it not only avoids repetitive labor, but also makes it easier to label some detailed targets.

[0083] Specifically, the stacking operation of the point cloud quality inspection device is as follows:

[0084] Frame 0 Point0 Frame 1 Point1 Frame 2 Point2 ... ... Frame n Pointn Overlapping frame Point0 Point1 Point2 ... Pointn

[0085] During the stacking process, the point cloud quality inspection device only stitches together multiple frames of point cloud data and does not modify the labels of the data points.

[0086] Step S14: Add refined annotation labels to the stacked point cloud data to generate refined point cloud data.

[0087] In the embodiment of the present application, the staff adds refined labels to the stacked point cloud data in step S13 through the point cloud quality inspection device, that is, performs manual refined annotation. At this time, the data point labels of the stacked point cloud data change according to the results of the manual refined annotation, and refined point cloud data is generated. Among them, only the labels of the refined point cloud data change compared with the stacked point cloud data, and the number of data points does not change.

[0088] Step S15: Split the refined point cloud data into the annotation label results of several frames of point cloud data to be annotated according to the number of data points of the point cloud data to be annotated.

[0089] In the embodiment of the present application, the point cloud quality inspection device sequentially splits the annotation label results of each frame according to the number of data points of each point cloud data to be annotated. Taking the example shown in the above table, the point cloud data frame 0 splits out the data points with the number of Point0.

[0090] In the present application, the point cloud quality inspection device acquires several frames of point cloud data to be annotated; acquires the pre-label of each frame of point cloud data to be annotated; stacks the several frames of point cloud data to be annotated and their pre-labels to obtain stacked point cloud data; adds refined annotation labels to the stacked point cloud data to generate refined point cloud data; splits the refined point cloud data into the annotation label results of the several frames of point cloud data to be annotated according to the number of data points of the point cloud data to be annotated. Through the above point cloud quality inspection method, manual refined annotation is performed in the stacked state, and the annotation label results of each frame of point cloud data are obtained by splitting, which can effectively improve the annotation speed.

[0091] In terms of annotation, the point cloud annotation method of the present application combines the results of the object detection model and the semantic segmentation model to initially refine the annotation results, uses the pose information to perform stacking operations on the point cloud, and speeds up the annotation efficiency.

[0092] In the existing point cloud quality inspection technology, the quality inspection link after data annotation return is a key step to ensure the quality of data annotation. Manual frame-by-frame quality inspection is relatively fine and can find detailed errors, but it is subjective and time-consuming; the sampling inspection method can save inspection time, but the sampled data may not be able to represent the overall quality of the data.

[0093] In response to this, the present application provides a point cloud quality inspection method. After annotation is completed, the data is verified through a deep learning model, and the risk data is visually inspected to speed up the data quality inspection.

[0094] Specifically, please refer to Figure 2 Continue to refer to Figure 6 , Figure 6 is a schematic flowchart of an embodiment of the point cloud quality inspection method provided by the present application.

[0095] As Figure 6 shown, the specific steps are as follows:

[0096] Step S21: Obtain the prediction result of the point cloud data to be quality inspected.

[0097] In the embodiment of the present application, the point cloud quality inspection device performs data inspection on the point cloud data to be quality inspected. Among them, the data inspection is mainly divided into static target inspection and dynamic target inspection. Among them, the point cloud data to be quality inspected is the point cloud data labeled in the above embodiment.

[0098] Specifically, please refer to Figure 7 , Figure 7 which is a schematic flowchart of the data inspection scheme provided by the present application. As Figure 7 shown, the data inspection scheme requires a 3D semantic segmentation model Seg-Model2 and a 3D object detection model Det-Model2. These two models are different in structure from the two models used in the above embodiment, but they are both commonly used models in the field of autonomous driving.

[0099] The data inspection is mainly divided into the inspection of dynamic targets and the inspection of static targets. Here, dynamic targets refer to objects moving around the host vehicle, and their behaviors and positions may change over time, such as other vehicles, pedestrians, animals, etc.; static targets refer to targets with fixed positions that do not change their positions over time. In the autonomous driving task of the present application, the static targets mainly include drivable areas, fences, curbs, pillars, etc.

[0100] Step S22: Obtain the annotation result based on the pre-annotated annotation label file.

[0101] In the embodiment of the present application, the point cloud quality inspection device extracts the annotation label file generated after the point cloud data to be quality inspected is annotated as the true annotation result.

[0102] Step S23: Obtain the similarity threshold between the prediction result and the annotation result.

[0103] In the embodiment of the present application, for dynamic targets, the point cloud quality inspection device uses the 3D object detection model Det-Model2 to generate a predicted detection box pred_box for the point cloud frame; on the other hand, using the manually returned and frame-split label file, cluster the same-class label points of the dynamic targets to generate a ground truth detection box gt_box.

[0104] The point cloud quality inspection device calculates the intersection over union 3D IoU (Intersection over Union) of all the predicted detection bounding boxes pred_box and the ground truth detection bounding boxes gt_box, that is, the dynamic target intersection over union. 3D IoU is an index used to evaluate the overlap degree between the predicted bounding box and the true bounding box in 3D object detection.

[0105] First, the point cloud quality inspection device needs to calculate the intersection volume V of the two bounding boxes intersection and the union volume V union The intersection volume V intersection It can be calculated by finding the overlap of two bounding boxes in three dimensions (length, width, height):

[0106] V intersection =max(0,I intersection )×max(0,w intersection )×max(0,h intersection )

[0107] Among them, l intersection is the lengthwise overlap of the two bounding boxes, calculated as:

[0108] l intersection =min(l gt ,l pred )-max(0,l gt_start -l pred_start )

[0109] Here l gt_start and l pred_start are the starting positions of the true bounding box and the detection bounding box in the length direction respectively. Similarly, we can get w intersection and h intersection The overlapping parts in the width direction and height direction respectively.

[0110] The formula for 3D IoU is:

[0111]

[0112] For static objects, the point cloud quality inspection device uses the 3D semantic segmentation model Seg-Model2 to generate predicted labels for the point cloud frames. Furthermore, the point cloud quality inspection device uses the label files that were manually returned and deframed to calculate the Intersection over Union (IoU) of all categories of static objects. Similar to the 3D IoU described above, the IoU here, or the IoU of static objects, is a measure of the similarity between two sets. The calculation formula is as follows:

[0113]

[0114] Step S24: When the similarity threshold meets the quality inspection condition, the point cloud data to be inspected is added to the training set.

[0115] In the embodiments of the present application, if the 3D IoU < 0.5, the point cloud quality inspection device determines that the frame data is risk data; if the 3D IoU > 0.5, the point cloud quality inspection device determines that the frame is safe data. For the IoU of each static category, if the IoU < 0.5, the point cloud quality inspection device determines that the frame is risk data; if the IoU > 0.5, the point cloud quality inspection device determines that the frame is safe data.

[0116] In summary, when the 3D IoU < 0.5 and the IoU > 0.5, the point cloud quality inspection device determines that the point cloud frame is safe data and adds it to the training set for model training.

[0117] Furthermore, if a point cloud frame is determined to be risk data in the above dynamic object inspection and static category inspection, the point cloud frame is simultaneously displayed in the visualization window.

[0118] For details, please refer to Figure 8 , Figure 8 which is a schematic flowchart of another embodiment of the point cloud quality inspection method provided by the present application.

[0119] As Figure 8 shown, the specific steps are as follows:

[0120] Step S31: When the similarity threshold does not meet the quality inspection conditions, compare the prediction result and the annotation result to determine the mispredicted data points.

[0121] Step S32: Set the mispredicted data points as the first label and the remaining data points as the second label.

[0122] In the embodiments of the present application, the present application uses category label rendering for the point cloud frames determined to be risk data; different from the conventional label rendering, the present application uses cross-label rendering. Specifically, if the annotation result label is the same as the prediction result label, the second label is 0; if they are different, the rendered result of the annotated label is displayed, that is, the first label.

[0123] As Figure 9 shown, Figure 9 is a schematic diagram of cross-visualization of risk data provided by the present application. In Figure 9 , the categories such as the ground and the car are correctly annotated, and the color with the label 0 is displayed, while other colors, that is, other label values, represent the points where the model prediction is inconsistent with the annotation result, so that the error points of the risk data can be further accurately judged.

[0124] Through cross-visualization inspection, the point cloud quality inspection device can find out the reasons for the risks in the risk data, and then write all the risk data paths into a txt file at the same time, which is convenient for the annotators to view and repair.

[0125] Finally, the annotator modifies the data and returns the final data for the algorithm engineer to perform model training.

[0126] The point cloud annotation method of the present application utilizes the high-precision advantage of the point cloud object detection model compared to the segmentation model for dynamic obstacle objects. In the preprocessing step of data labeling, the data with pre-labeled segmentation labels is pre-annotated for object detection, and the results are projected onto the original labels, optimizing the pre-labeling effect of dynamic objects.

[0127] The point cloud annotation method of the present application performs frame stacking processing on the pre-annotated data and sends the labeled data. The point cloud after frame stacking processing can be integrated from the original single frame into multiple frames, effectively alleviating the problem of repeated scene annotation within a short time interval and improving the annotation efficiency.

[0128] For the acceptance of the labeled data, the point cloud quality inspection method of the present application uses the detection model and the segmentation model to filter and screen the data. The 3D detection model is responsible for verifying the annotation quality of dynamic obstacle objects, and the 3D segmentation model is responsible for static categories. The two models are used for comprehensive evaluation to improve the inspection efficiency and quality.

[0129] For the visualization of the high-risk labeled data, the point cloud quality inspection method of the present application adopts the visualization method of cross-rendering labels. For the points where the model is inconsistent with the ground truth, the ground truth points are displayed, and the reasons for errors can be more clearly and distinctly found during manual visual inspection.

[0130] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation to the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0131] To implement the above point cloud annotation method and / or point cloud quality inspection method, the present application also proposes a point cloud quality inspection device. For details, please refer to Figure 10 , Figure 10 which is a schematic structural diagram of an embodiment of the point cloud quality inspection device provided by the present application.

[0132] The point cloud quality inspection device 400 of this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0133] The processor 41, the memory 42, and the input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the point cloud annotation method and / or point cloud quality inspection method described in the above embodiment.

[0134] In an embodiment of the present application, the processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip with signal processing capabilities. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP, Digital Signal Process), an application specific integrated circuit (ASIC, Application Specific Integrated Circuit), a field programmable gate array (FPGA, Field Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 41 may also be any conventional processor, etc.

[0135] The present application also provides a computer storage medium. Please continue to refer to Figure 11 , Figure 11 FIG. is a schematic structural diagram of an embodiment of the computer storage medium provided by the present application. A computer program 61 is stored in the computer storage medium 600. When the computer program 61 is executed by a processor, it is used to implement the point cloud annotation method and / or the point cloud quality inspection method of the above embodiment.

[0136] When the embodiments of the present application are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0137] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A point cloud annotation method, characterized in that, The point cloud annotation method includes: Obtaining several frames of point cloud data to be annotated; Obtaining the preliminary annotation labels for each frame of point cloud data to be annotated; Stacking the several frames of point cloud data to be annotated and their preliminary annotation labels to obtain stacked point cloud data; Adding refined annotation labels to the stacked point cloud data to generate refined point cloud data; Splitting the refined point cloud data according to the number of data points of the point cloud data to be annotated to obtain the annotation label results of the several frames of point cloud data to be annotated.

2. The point cloud annotation method according to claim 1, wherein The obtaining of the preliminary annotation labels for each frame of point cloud data to be annotated includes: Inputting each frame of point cloud data to be annotated into a 3D segmentation model to obtain the preliminary classification data points of each frame of point cloud data to be annotated; Inputting each frame of point cloud data to be annotated into a 3D detection model to obtain the preliminary detection boxes of each frame of point cloud data to be annotated; Fusing the preliminary classification data points and the preliminary detection boxes to generate the preliminary annotation labels for each frame of point cloud data to be annotated.

3. The point cloud annotation method according to claim 2, wherein The preliminary detection boxes are the preliminary detection boxes of dynamic obstacles; The fusing of the preliminary classification data points and the preliminary detection boxes to generate the preliminary annotation labels for each frame of point cloud data to be annotated includes: Refining the dynamic preliminary classification data points in the preliminary classification data points according to the preliminary classification of the data points in the preliminary detection boxes to obtain the dynamic target segmentation results; Fusing the dynamic target segmentation results with other preliminary classification data points to generate the preliminary annotation labels for each frame of point cloud data to be annotated.

4. The point cloud annotation method according to claim 1, wherein The stacking of the several frames of point cloud data to be annotated and their preliminary annotation labels to obtain stacked point cloud data includes: Obtaining the pose information of each frame of point cloud data to be annotated; Converting the several frames of point cloud data to be annotated to the same coordinate system according to the pose information and performing stacking to obtain the stacked point cloud data.

5. A point cloud quality inspection method, characterized in that, The point cloud quality inspection method includes: Obtaining the prediction results of the point cloud data to be quality inspected; Obtaining the annotation results based on a pre-annotated annotation label file; Obtaining the similarity threshold between the prediction results and the annotation results; When the similarity threshold meets the quality inspection conditions, adding the point cloud data to be quality inspected to the training set.

6. The point cloud quality inspection method according to claim 5, wherein The prediction results include dynamic target prediction results and static target prediction results; The annotation results include dynamic target annotation results and static target annotation results; The obtaining of the similarity threshold between the prediction results and the annotation results includes: Obtaining the dynamic target intersection over union of the dynamic target prediction results and the dynamic target annotation results; Obtaining the static target intersection over union of the static target prediction results and the static target annotation results; The point cloud quality inspection method further includes: When the dynamic target intersection over union is greater than the first intersection over union and the static target intersection over union is greater than the second intersection over union, determining that the point cloud data to be quality inspected meets the quality inspection conditions.

7. The point cloud quality inspection method according to claim 6, wherein the dynamic target prediction result is a dynamic target prediction box output by a three-dimensional target detection model, and the dynamic target annotation result is a ground truth detection box; the static target prediction result is the data points of the dynamic target category output by a three-dimensional target segmentation model, and the dynamic target prediction result is the ground truth category points of the dynamic target.

8. The point cloud quality inspection method according to claim 5, wherein the point cloud quality inspection method further includes: when the similarity threshold does not meet the quality inspection condition, comparing the prediction result and the annotation result to determine the mispredicted data points; setting the mispredicted data points as the first label and setting the remaining data points as the second label.

9. A point cloud quality inspection device, characterized in that, The point cloud quality inspection device includes a memory and a processor coupled to the memory; wherein, the memory is used for storing program data, and the processor is used for executing the program data to implement the point cloud annotation method according to any one of claims 1 to 4, and / or the point cloud quality inspection method according to any one of claims 5 to 8.

10. A computer storage medium, characterized in that, The computer storage medium is used for storing program data, and when the program data is executed by a computer, it is used to implement the point cloud annotation method according to any one of claims 1 to 4, and / or the point cloud quality inspection method according to any one of claims 5 to 8.

Images