Quality inspection method applied to 3D point cloud data annotation and electronic equipment
By using automated quality inspection methods to perform quality inspection on 3D point cloud data using pre-trained models and quality inspection rules, the problems of low efficiency and poor consistency of manual quality inspection are solved, achieving efficient and low-cost improvement of labeled data quality and ensuring the accuracy of algorithm training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUBEI TIEMUNIU INTELLIGENT ROBOT CO LTD
- Filing Date
- 2025-12-12
- Publication Date
- 2026-04-10
AI Technical Summary
Existing 3D point cloud annotation quality inspection methods rely on manual labor, which is inefficient, costly, and inconsistent, affecting the accuracy of algorithm training.
By acquiring target point cloud data and manually labeled data, a pre-trained target detection model is used for quality inspection. A quality inspection report is generated based on the predicted bounding box information and quality inspection rules, and unqualified labeled data is automatically filtered out.
This improved quality inspection efficiency, reduced costs, and enhanced the quality and consistency of labeled data, ensuring the accuracy of algorithm training.
Smart Images
Figure CN121837833A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of quality inspection applied to 3D point cloud data annotation, and more particularly to a quality inspection method and electronic device applied to 3D point cloud data annotation. Background Technology
[0002] 3D point cloud annotation plays a crucial role in fields such as autonomous driving and robot navigation, and the annotated data is the foundation for algorithm training. However, current manual annotation of 3D point clouds is prone to various errors, such as unreasonable annotation dimensions, incorrect category labeling, and omissions. Therefore, quality control of 3D point cloud annotation data is necessary.
[0003] In related technologies, the 3D point cloud annotation quality inspection method mainly relies on manual labor, which has the following problems: (1) Manual quality inspection is extremely inefficient. Faced with a large amount of point cloud annotation data, it is difficult to complete a comprehensive quality inspection in a short period of time; (2) Manual quality inspection is costly and requires a large investment of manpower; (3) Manual quality inspection is greatly affected by subjective factors. The standards of different quality inspectors may differ, resulting in poor consistency of quality inspection results, making it difficult to guarantee the quality of the annotation data, and thus affecting the accuracy of subsequent algorithm training. Summary of the Invention
[0004] This disclosure provides a quality inspection method and electronic device for 3D point cloud data annotation, so as to at least solve the above-mentioned technical problems existing in the prior art.
[0005] According to a first aspect of this application, a quality inspection method for 3D point cloud data annotation is provided, the method comprising: Obtain the target point cloud data and the corresponding manually labeled data of the target point cloud data; Analyze the manually labeled data to determine the label box information for each frame of manually labeled data; The target point cloud data is input into a pre-trained target detection model to obtain the predicted bounding box information for each frame of target point cloud data; Based on the predicted bounding box information, the labeled bounding box information is inspected using preset quality inspection rules, and a quality inspection report is generated.
[0006] In one possible implementation, acquiring the target point cloud data includes: Obtain raw point cloud data; The original point cloud data is denoised to obtain denoised data; The format of the denoised data is converted into the target format to obtain the target point cloud data.
[0007] In one possible implementation, parsing the manually labeled data and determining the label box information for each frame of manually labeled data includes: Analyze the manually labeled data to determine the geometric, semantic, and administrative attributes of each frame of manually labeled data; The geometric attributes include the center coordinates, size, rotation direction, and vertices of the bounding box; the semantic attributes include the category, label, and difficulty / occlusion / truncation level of the bounding box; and the management attributes include the tracking ID, unique ID, and remarks of the bounding box.
[0008] In one possible implementation, the step of inputting the target point cloud data into a pre-trained target detection model to obtain the predicted bounding box information for each frame of target point cloud data includes: Acquire non-ground point cloud data that is synchronized with the target point cloud data in time; Target detection is performed on any target point cloud data and the corresponding non-ground point cloud data respectively to obtain first target detection data and second target detection data; The first target detection data and the second target detection data are fused to generate fused target data; By using a detection sub-model with multi-frame temporal correlation, multi-frame target point cloud data is processed to obtain third target detection data with temporal ID; The fused target data is fused with the third target detection data to obtain the predicted bounding box information of the first frame target point cloud data; Each frame of target point cloud data is sequentially input into the target detection model to obtain the predicted bounding box information of each frame of target point cloud data.
[0009] In one possible implementation, the labeled box information is inspected using preset quality inspection rules based on the predicted box information, and a quality inspection report is generated, including: Based on the predicted bounding box information, determine whether the labeled bounding box information conforms to the standard; In response to the fact that the bounding box information meets the standard, the pairwise intersection-union ratio (CIU) between the bounding box information and the predicted box information is calculated, and the bounding box information and the predicted box information are matched based on the CIU to obtain the matching result; wherein, a bounding box information is matched only with the predicted box information that ranks first in the CIU ranking and the CIU is greater than or equal to a preset threshold, and a predicted box information is matched only with one bounding box information; Based on the matching results, a quality inspection report is generated.
[0010] In one possible implementation, determining whether the annotation box information conforms to the standard includes: Determine whether the size of the annotation box is reasonable; Determine whether the category of the labeled box is valid; Determine the validity of the number of points within the labeled box.
[0011] In one possible implementation, the matching result includes: Successfully matched bounding box and predicted bounding box information; Information on failed matching of annotation boxes; the information on failed matching of annotation boxes indicates manual annotation errors; Information on prediction boxes that failed to match; the information on prediction boxes that failed to match indicates that the target point cloud data was missed.
[0012] In one possible implementation, before acquiring the target point cloud data, the method further includes: The training of the object detection model includes: Acquire point cloud data to be trained, and annotate the point cloud data to generate annotation data. The annotation data includes a 3D bounding box corresponding to the target object in the point cloud data to be trained. The 3D bounding box includes at least the center coordinates, size parameters, and category label. Combine the point cloud data to be trained with the corresponding annotation data to form a training set, and divide the training set into a training subset, a validation subset, and a test subset. A neural network model suitable for point cloud target detection is constructed. The neural network model includes a feature extraction module and a detection head module. The feature extraction module is used to extract three-dimensional spatial features from point cloud data, and the detection head module is used to output prediction data of the target object. The prediction data includes a three-dimensional prediction box corresponding to the three-dimensional annotation box format and a confidence parameter. The training subset is input into the neural network model in a preset batch, and the neural network model outputs predicted data. Calculate the intersection-union ratio (IoU) between the 3D predicted bounding boxes in the predicted data and the 3D labeled bounding boxes in the corresponding labeled data; A loss function is constructed based on the intersection-union ratio (IU), and the loss function includes at least a regression loss term and a classification loss term. The regression loss term calculates the positional deviation between the 3D prediction box and the 3D annotation box based on the IU, and the classification loss term is used to optimize the category prediction accuracy. The parameters of the neural network model are adjusted according to the loss function using the backpropagation algorithm. The model is iteratively trained until the loss value of the neural network model on the validation subset stabilizes after a preset number of consecutive rounds. The neural network model is then determined to have converged, and the target detection model is obtained.
[0013] In one embodiment, the non-ground point cloud data is obtained by segmenting the original point cloud using a Euclidean clustering method.
[0014] According to a second aspect of this application, a quality inspection device for 3D point cloud data annotation is provided, comprising: The acquisition module is used to acquire target point cloud data and the corresponding manually labeled data of the target point cloud data; The determination module is used to parse the manually labeled data and determine the label box information for each frame of manually labeled data; The detection module is used to input the target point cloud data into a pre-trained target detection model to obtain the predicted bounding box information of each frame of target point cloud data; The generation module is used to perform quality inspection on the labeled box information based on the predicted box information using preset quality inspection rules, and generate a quality inspection report.
[0015] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.
[0016] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.
[0017] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.
[0018] Using the technical solution of this application, the quality inspection of manually annotated 3D boxes can be automatically performed, and unqualified annotation data can be screened out. The quality inspection is efficient, low-cost, and can also improve the quality of annotation data.
[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0020] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
[0021] Figure 1 This illustration shows the implementation flow of a quality inspection method applied to 3D point cloud data annotation in an embodiment of this application. Figure 1 ; Figure 2This illustration shows the implementation flow of a quality inspection method applied to 3D point cloud data annotation in an embodiment of this application. Figure 2 ; Figure 3 A block diagram of a quality inspection device applied to 3D point cloud data annotation is shown in an embodiment of this application; Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation
[0022] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0023] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0025] The following section, with reference to the accompanying drawings, describes a quality inspection method and electronic device for 3D point cloud data annotation provided in this application.
[0026] like Figure 1 As shown, this application provides a quality inspection method for 3D point cloud data annotation, the method comprising: S101, acquire the target point cloud data and the manually labeled data corresponding to the target point cloud data; The target point cloud data in this application can cover urban main roads, highways, and residential roads in autonomous driving scenarios, and include conditions such as sunny / rainy days, day / night, and congested / smooth traffic. The target point cloud data in this application can be data that has undergone format conversion. Specifically, the target point cloud data is data that the target detection model can process.
[0027] The manually labeled data is used for quality control of the technical solution in this application. It is understood that, to reduce the error rate and improve the accuracy of manually labeled data, this application may employ labeling tools suitable for the corresponding scenarios. For example, for small-scale data, the open-source tool Open3D can be used for manual labeling, while for autonomous driving data, LidarAnnotator can be used for labeling.
[0028] S102, parse the manually labeled data and determine the label box information for each frame of manually labeled data; In this application, manually labeled data is parsed to determine the label box information for each frame, thereby transforming the original label file into a structured format that can be directly used for model training or quality control.
[0029] The annotation box information includes the annotation box's geometric attributes, semantic attributes, and management attributes. The geometric attributes include the annotation box's center coordinates, size, rotation direction, and vertices. The semantic attributes include the annotation box's category, label, and difficulty / occlusion / truncation level. The management attributes include the annotation box's tracking ID, unique ID, and remarks.
[0030] S103, Input the target point cloud data into the pre-trained target detection model to obtain the prediction box information of each frame of target point cloud data; It should be noted that the object detection model provided in this application is a model with a high threshold output. The object detection model can output predicted bounding boxes of object point cloud data. The accuracy of these predicted bounding boxes is high.
[0031] S104, Based on the predicted box information, the labeled box information is inspected using preset quality inspection rules, and a quality inspection report is generated.
[0032] In this application, the predicted bounding box is compared with the labeled bounding box. If the difference between the labeled bounding box and the predicted bounding box is large, the labeled bounding box is considered to be abnormal.
[0033] It should be noted that the quality inspection rules in this application may include determining the size of the annotation box, the center point of the annotation box, and the IoU ratio between the annotation box and the predicted box. This generates a quality inspection report indicating anomalies. The quality inspection report may also include optimization suggestions. For example, if the omission rate of 6.3% in highway scenes is significantly higher than that of 1.2% in urban roads, it is recommended to optimize the annotation process for highway scenes. If the pedestrian omission rate of 7.5% is significantly higher than other categories, it is necessary to analyze the commonalities of the omissions, such as distance and sparse point cloud, and propose targeted improvements. Specifically, the improvement method could be to add a highlighting function for distant targets to the annotation tool.
[0034] The quality inspection method for 3D point cloud data annotation provided in this application, such as... Figure 2As shown, the process begins by acquiring target point cloud data and manually annotated data using sensors. The manually annotated data is then parsed using an existing parser to determine the bounding box information for each frame. Next, the target point cloud data is input into a pre-trained target detection model to obtain the predicted bounding box information for each frame. Based on the predicted bounding box information, a pre-defined quality control rule is applied to perform quality checks on the bounding box information, generating a quality control report. After processing one frame of manually annotated data, it is determined whether there is another unprocessed frame of manually annotated data. If so, the above steps are repeated for the next unprocessed frame until all frames of manually annotated data have been processed, and a quality control report is output.
[0035] This application can automatically perform quality inspection on manually annotated 3D bounding boxes, filter out unqualified annotation data, and achieve high quality inspection efficiency and low cost, while also improving the quality of annotation data.
[0036] In some embodiments, acquiring the target point cloud data includes: Obtain raw point cloud data; The original point cloud data is denoised to obtain denoised data; The format of the denoised data is converted into the target format to obtain the target point cloud data.
[0037] In this application, raw point cloud data of the target scene can be acquired through imaging using LiDAR or other sensors. The target scene can be a driving scene or an industrial inspection scene. After obtaining the raw point cloud data, it is first denoised. Denoising can remove isolated points, dense interference points in local areas, and points that are outside the effective detection range, thereby improving detection accuracy.
[0038] Because different sensors acquire raw point cloud data in different formats, it is necessary to convert the denoised data into a format that can be input into the object detection model. For example, raw point cloud data can be in various formats such as npy, pcd, and bin. Manually labeled data can be in JSON format.
[0039] In some embodiments, parsing the manually labeled data and determining the bounding box information for each frame of manually labeled data includes: Analyze the manually labeled data to determine the geometric, semantic, and administrative attributes of each frame of manually labeled data; The geometric attributes include the center coordinates, size, rotation direction, and vertices of the bounding box; the semantic attributes include the category, label, and difficulty / occlusion / truncation level of the bounding box; and the management attributes include the tracking ID, unique ID, and remarks of the bounding box.
[0040] This application aims to obtain annotation box information from manually annotated data and determine annotation quality by checking whether the annotation box information meets quality inspection specifications. Specifically, extraction rules can be formulated based on the different characteristics of geometric, semantic, and management attributes to ensure the accuracy, consistency, and usability of attribute values. Specifically, the quality inspection can be judged by checking for anomalies in the annotation box's center coordinates, size, rotation direction, and vertices, or by checking the annotation box's category, label, difficulty / occlusion / truncation level, tracking ID, unique ID, and remarks. If any of the above anomalies are found, the quality inspection is considered unqualified.
[0041] In some embodiments, the step of inputting the target point cloud data into a pre-trained target detection model to obtain the predicted bounding box information for each frame of target point cloud data includes: Acquire non-ground point cloud data that is synchronized with the target point cloud data in time; Target detection is performed on any target point cloud data and the corresponding non-ground point cloud data respectively to obtain first target detection data and second target detection data; The first target detection data and the second target detection data are fused to generate fused target data; By using a detection sub-model with multi-frame temporal correlation, multi-frame target point cloud data is processed to obtain third target detection data with temporal ID; The fused target data is fused with the third target detection data to obtain the predicted bounding box information of the first frame target point cloud data; Each frame of target point cloud data is sequentially input into the target detection model to obtain the predicted bounding box information of each frame of target point cloud data.
[0042] In this application, the non-ground point cloud data is the point cloud after removing static backgrounds such as ground and buildings. Non-ground point cloud data can reduce background interference and improve target detection accuracy. In this application, the non-ground point cloud data is obtained by segmenting the original point cloud using the Euclidean clustering method.
[0043] Time synchronization can be based on sensor timestamps to ensure the time difference between the non-ground point cloud and the target point cloud is below a time threshold. Then, target detection is performed on one frame of the target point cloud data to obtain the first target detection data. Point cloud target detection is then performed on the non-ground point cloud data, focusing on foreground targets to obtain the second target detection data, improving the detection sensitivity for small and sparse targets. The 3D IoU between the predicted bounding boxes in the first and second target detection data is calculated. If the IoU is greater than or equal to a preset threshold, the first and second target detection data are considered to represent the same target. Specifically, during fusion, a weighted average of the two values can be used for geometric attribute fusion, while for semantic attribute fusion, the category label with higher confidence can be retained, and the confidence level can be the maximum of the two values.
[0044] By utilizing a detection sub-model with multi-frame temporal correlation, the possible location of the target in the current frame can be predicted based on the target's position and velocity in the previous frame. Then, the 3D IoU between the predicted location and the detection bounding box in the current frame is calculated. Targets that successfully match inherit the previous ID, and new targets are assigned a new ID. Finally, the spatial fusion result and the temporal correlation result are fused again to output predicted bounding box information that combines high accuracy and temporal consistency.
[0045] The target detection model provided in this application utilizes non-terrestrial point clouds to reduce background interference, and the dual-model fusion complements each other's strengths and weaknesses. Multi-frame association assigns a unique temporal ID to the target, solving the ID jump problem in single-frame detection. It also demonstrates stronger adaptability to occlusion, small targets, and sparse point cloud scenes.
[0046] In some embodiments, the labeled box information is inspected using preset quality inspection rules based on the predicted box information, and a quality inspection report is generated, including: Based on the predicted bounding box information, determine whether the labeled bounding box information conforms to the standard; In response to the fact that the bounding box information meets the standard, the pairwise intersection-union ratio (CIU) between the bounding box information and the predicted box information is calculated, and the bounding box information and the predicted box information are matched based on the CIU to obtain the matching result; wherein, a bounding box information is matched only with the predicted box information that ranks first in the CIU ranking and the CIU is greater than or equal to a preset threshold, and a predicted box information is matched only with one bounding box information; Based on the matching results, a quality inspection report is generated.
[0047] In some embodiments, determining whether the annotation box information conforms to the standard includes: Determine whether the size of the annotation box is reasonable; Determine whether the category of the labeled box is valid; Determine the validity of the number of points within the labeled box.
[0048] In this application, before performing intersection-union (IU) matching, it is necessary to first filter out annotation boxes that clearly do not meet the standards based on the reasonableness benchmark of the predicted boxes, thereby reducing subsequent invalid matching calculations and eliminating basic errors in the annotations themselves, such as empty boxes or abnormal dimensions. If the predicted boxes meet the reasonableness benchmark, the pairwise IU ratio between the annotation box information and the predicted box information is further calculated to match the annotation boxes with the predicted boxes. Specifically, when the pairwise IU ratio is greater than or equal to a preset threshold, the annotation box is considered to match the predicted box. Based on the matching results, the quality status of the annotation boxes, i.e., the quality inspection report, can be displayed through statistical analysis, anomaly details, and visualization.
[0049] In some embodiments, the matching results include: Successfully matched bounding box and predicted bounding box information; Information on failed matching of annotation boxes; the information on failed matching of annotation boxes indicates manual annotation errors; Information on prediction boxes that failed to match; the information on prediction boxes that failed to match indicates that the target point cloud data was missed.
[0050] Specifically, in the intersection-union matching logic, the three categories of matching results (successful matching, failed bounding box matching, and failed predicted bounding box matching) directly correspond to the core issues of annotation quality (compliant annotation, annotation errors, and missing data). Successful matching indicates that the annotation quality is acceptable, meaning that the manually annotated target and the target detected by the model are highly consistent in spatial location and semantic category.
[0051] A match is considered to have failed if the following conditions are met.
[0052] (1) Calculate the IoU of all high-confidence prediction boxes for the labeled boxes, with the maximum value being less than the preset threshold (e.g., 0.5). (2) The highest IoU of the bounding box is greater than or equal to the preset threshold, but the corresponding predicted box has been matched by other bounding boxes with higher IoU (low priority). (3) The category of the labeled box has no overlap with the categories of all high-confidence prediction boxes (e.g., labeled as "airplane", but the prediction box categories are only "car" and "pedestrian").
[0053] In some embodiments, the process further includes: The training of the object detection model includes: Acquire point cloud data to be trained, and annotate the point cloud data to generate annotation data. The annotation data includes a 3D bounding box corresponding to the target object in the point cloud data to be trained. The 3D bounding box includes at least the center coordinates, size parameters, and category label. Combine the point cloud data to be trained with the corresponding annotation data to form a training set, and divide the training set into a training subset, a validation subset, and a test subset. A neural network model suitable for point cloud target detection is constructed. The neural network model includes a feature extraction module and a detection head module. The feature extraction module is used to extract three-dimensional spatial features from point cloud data, and the detection head module is used to output prediction data of the target object. The prediction data includes a three-dimensional prediction box corresponding to the three-dimensional annotation box format and a confidence parameter. The training subset is input into the neural network model in a preset batch, and the neural network model outputs predicted data. Calculate the intersection-union ratio (IoU) between the 3D predicted bounding boxes in the predicted data and the 3D labeled bounding boxes in the corresponding labeled data; A loss function is constructed based on the intersection-union ratio (IU), and the loss function includes at least a regression loss term and a classification loss term. The regression loss term calculates the positional deviation between the 3D prediction box and the 3D annotation box based on the IU, and the classification loss term is used to optimize the category prediction accuracy. The parameters of the neural network model are adjusted according to the loss function using the backpropagation algorithm. The model is iteratively trained until the loss value of the neural network model on the validation subset stabilizes after a preset number of consecutive rounds. The neural network model is then determined to have converged, and the target detection model is obtained.
[0054] In this application, the first step is to obtain an object detection model. This model is then trained using a neural network. The object detection model is updated after a preset time period to improve its accuracy. The high-quality labeled data used to train the object detection model in this application is obtained through multiple verifications by experienced annotators or preliminary screening using existing commercial quality control tools, ensuring its reliability as a training benchmark.
[0055] like Figure 3 As shown, this application provides a quality inspection device for 3D point cloud data annotation, comprising: The acquisition module 301 is used to acquire target point cloud data and the manually labeled data corresponding to the target point cloud data; The determination module 302 is used to parse the manually labeled data and determine the label box information of each frame of manually labeled data; The detection module 303 is used to input the target point cloud data into a pre-trained target detection model to obtain the prediction box information of each frame of target point cloud data; The generation module 304 is used to perform quality inspection on the annotation box information based on the prediction box information using preset quality inspection rules, and generate a quality inspection report.
[0056] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.
[0057] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the quality inspection method for 3D point cloud data annotation described in this application. The computer instructions are used to cause the computer to perform the quality inspection method for 3D point cloud data annotation described in this application.
[0058] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the quality inspection method of this application for 3D point cloud data annotation.
[0059] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.
[0060] like Figure 4 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.
[0061] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0062] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a quality inspection method applied to 3D point cloud data annotation. For example, in some embodiments, the quality inspection method applied to 3D point cloud data annotation can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the quality inspection method applied to 3D point cloud data annotation described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a quality control method applied to 3D point cloud data annotation.
[0063] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0064] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0065] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0066] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0067] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0068] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A quality inspection method for 3D point cloud data annotation, characterized in that, The method includes: Obtain the target point cloud data and the corresponding manually labeled data of the target point cloud data; Analyze the manually labeled data to determine the label box information for each frame of manually labeled data; The target point cloud data is input into a pre-trained target detection model to obtain the predicted bounding box information for each frame of target point cloud data; Based on the predicted bounding box information, the labeled bounding box information is inspected using preset quality inspection rules, and a quality inspection report is generated.
2. The method according to claim 1, characterized in that, The acquisition of target point cloud data includes: Obtain raw point cloud data; The original point cloud data is denoised to obtain denoised data; The format of the denoised data is converted into the target format to obtain the target point cloud data.
3. The method according to claim 1, characterized in that, The process of parsing the manually labeled data to determine the bounding box information for each frame of manually labeled data includes: Analyze the manually labeled data to determine the geometric, semantic, and administrative attributes of each frame of manually labeled data; The geometric attributes include the center coordinates, size, rotation direction, and vertices of the bounding box; the semantic attributes include the category, label, and difficulty / occlusion / truncation level of the bounding box; and the management attributes include the tracking ID, unique ID, and remarks of the bounding box.
4. The method according to claim 1, characterized in that, The step of inputting the target point cloud data into a pre-trained target detection model to obtain the predicted bounding box information for each frame of target point cloud data includes: Acquire non-ground point cloud data that is synchronized with the target point cloud data in time; Target detection is performed on any target point cloud data and the corresponding non-ground point cloud data respectively to obtain first target detection data and second target detection data; The first target detection data and the second target detection data are fused to generate fused target data; By using a detection sub-model with multi-frame temporal correlation, multi-frame target point cloud data is processed to obtain third target detection data with temporal ID; The fused target data is fused with the third target detection data to obtain the predicted bounding box information of the first frame target point cloud data; Each frame of target point cloud data is sequentially input into the target detection model to obtain the predicted bounding box information of each frame of target point cloud data.
5. The method according to claim 1, characterized in that, Based on the predicted bounding box information, the labeled bounding box information is inspected using preset quality inspection rules, and a quality inspection report is generated, including: Based on the predicted bounding box information, determine whether the labeled bounding box information conforms to the standard; In response to the fact that the bounding box information meets the standard, the pairwise intersection-union ratio (CIU) between the bounding box information and the predicted box information is calculated, and the bounding box information and the predicted box information are matched based on the CIU to obtain the matching result; wherein, a bounding box information is matched only with the predicted box information that ranks first in the CIU ranking and the CIU is greater than or equal to a preset threshold, and a predicted box information is matched only with one bounding box information; Based on the matching results, a quality inspection report is generated.
6. The method according to claim 5, characterized in that, Determining whether the annotation box information meets the standard includes: Determine whether the size of the annotation box is reasonable; Determine whether the category of the labeled box is valid; Determine the validity of the number of points within the labeled box.
7. The method according to claim 5, characterized in that, The matching results include: Successfully matched bounding box and predicted bounding box information; Information on failed matching of annotation boxes; the information on failed matching of annotation boxes indicates manual annotation errors; Information on prediction boxes that failed to match; the information on prediction boxes that failed to match indicates that the target point cloud data was missed.
8. The method according to claim 1, characterized in that, Before acquiring the target point cloud data, the following steps are also included: The training of the object detection model includes: Acquire point cloud data to be trained, and annotate the point cloud data to generate annotation data. The annotation data includes a 3D bounding box corresponding to the target object in the point cloud data to be trained. The 3D bounding box includes at least the center coordinates, size parameters, and category label. Combine the point cloud data to be trained with the corresponding annotation data to form a training set, and divide the training set into a training subset, a validation subset, and a test subset. A neural network model suitable for point cloud target detection is constructed. The neural network model includes a feature extraction module and a detection head module. The feature extraction module is used to extract three-dimensional spatial features from point cloud data, and the detection head module is used to output prediction data of the target object. The prediction data includes a three-dimensional prediction box corresponding to the three-dimensional annotation box format and a confidence parameter. The training subset is input into the neural network model in a preset batch, and the neural network model outputs predicted data. Calculate the intersection-union ratio (IoU) between the 3D predicted bounding boxes in the predicted data and the 3D labeled bounding boxes in the corresponding labeled data; A loss function is constructed based on the intersection-union ratio (IU), and the loss function includes at least a regression loss term and a classification loss term. The regression loss term calculates the positional deviation between the 3D prediction box and the 3D annotation box based on the IU, and the classification loss term is used to optimize the category prediction accuracy. The parameters of the neural network model are adjusted according to the loss function using the backpropagation algorithm. The model is iteratively trained until the loss value of the neural network model on the validation subset stabilizes after a preset number of consecutive rounds. The neural network model is then determined to have converged, and the target detection model is obtained.
9. The method according to claim 4, characterized in that, The non-ground point cloud data was obtained by segmenting the original point cloud using Euclidean clustering.
10. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.