A three-dimensional target labeling method and system for a laser radar point cloud

Abstract

This invention relates to the field of computer vision technology and discloses a method and system for 3D target annotation of LiDAR point clouds. The method includes: acquiring multi-sensor data of a target scene; determining a target set within a spatial range; constructing an initial target annotation vector set; correcting the 3D bounding box using perspective projection verification and point cloud geometric fitting; performing cross-frame consistency verification and label export; and outputting a delivered annotation dataset. Compared to existing technologies where the 3D annotation process for point clouds lacks unified frame alignment, especially under conditions of joint acquisition by multiple cameras and LiDAR, which makes it impossible to achieve stable and continuous 3D annotation results, this application achieves unified correspondence between point clouds, images, 3D bounding boxes, and label files through timestamp-synchronized frame deletion, spatial envelope filtering, and instance cross-frame consistency verification, thereby improving the annotation consistency of 3D detection and tracking training data for autonomous driving.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for three-dimensional target annotation of LiDAR point clouds. Background Technology

[0002] High-quality 3D target annotation data is crucial for training perception models in fields such as autonomous driving and robot navigation. LiDAR can provide accurate 3D point cloud information and is a key sensor for perceiving the 3D structure of the environment. However, efficiently and accurately annotating the 3D position, size, and pose of various targets (such as vehicles, pedestrians, cyclists, etc.) directly from raw LiDAR point clouds is an extremely challenging task.

[0003] Currently, most mainstream 3D target annotation methods rely on manual interactive annotation using point cloud visualization tools or auxiliary judgment using camera images. This method is not only time-consuming, labor-intensive, and costly, but the annotation results are also easily influenced by the subjective experience of the annotators, making it difficult to guarantee the consistency and accuracy of the annotations. This is especially true in scenarios where the target is far away, the point cloud is sparse, there is occlusion, or the lighting conditions are complex, where the accuracy and efficiency of manual annotation will drop significantly. In addition, some existing semi-automatic annotation tools often lack deep fusion and utilization of multi-sensor data (such as multi-view camera images and LiDAR point clouds) and cross-frame consistency verification of annotation results over time, resulting in potentially spatial deviations or temporal jitter in the generated annotation dataset, which cannot fully meet the requirements for high-precision, high-efficiency, and highly consistent 3D annotation data production. Therefore, there is an urgent need for a 3D target annotation method that can still achieve automation, high accuracy, and cross-frame consistency in complex and ever-changing real-world scenarios, in order to improve annotation efficiency, reduce labor costs, and provide more reliable training data for autonomous driving perception algorithms. Summary of the Invention

[0004] To address the aforementioned technical shortcomings, the purpose of this invention is to propose a 3D target annotation method for LiDAR point clouds. This method aims to solve the technical problem that the existing 3D annotation process for point clouds lacks unified frame alignment, especially under the condition of joint acquisition by multiple cameras and LiDAR, which makes it impossible to achieve stable and continuous 3D annotation results.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a three-dimensional target annotation method for lidar point clouds.

[0006] The aforementioned method for three-dimensional target annotation of lidar point clouds includes: Step S10: Acquire multi-sensor data of the target scene, perform frame alignment task based on the multi-sensor data using timestamp synchronization verification method, and output frame alignment dataset; Step S20: Based on the frame alignment dataset, perform the target range filtering task using the Euclidean space envelope filtering method, and output the target set within the spatial range; Step S30: Based on the target set within the spatial range, perform the annotation vector construction task using the KITTI extended annotation format, and output the initial target annotation vector set; Step S40: Based on the initial target annotation vector set, perform a 3D bounding box correction task using perspective projection verification and point cloud geometric fitting, and output the corrected annotation vector set; Step S50: Perform cross-frame consistency verification and label export based on the modified annotation vector set, and output the delivery annotation dataset.

[0007] Preferably, step S10, which involves acquiring multi-sensor data of the target scene, performing frame alignment based on the multi-sensor data using a timestamp synchronization verification method, and outputting a frame alignment dataset, specifically includes: Step S101: Read the LiDAR point cloud sequence, image sequences from multiple camera perspectives, and extrinsic parameter calibration data between the camera and LiDAR from the target scene directory; Step S102: Using the timestamp of the lidar point cloud sequence as a reference, perform synchronous matching on image sequences from multiple camera perspectives to obtain the timestamp correspondence between point cloud frames and image frames; Step S103: Delete invalid point cloud frames in the lidar point cloud sequence that do not meet the preset point cloud labeling conditions, and simultaneously delete the corresponding image frames in the image sequences of multiple camera views according to the timestamp correspondence to obtain the frame alignment dataset. The conditions for point cloud annotation include that the point cloud file is readable, the point cloud coordinates have not undergone overall abnormal shift, and there are valid point clouds in the target area for 3D bounding box fitting.

[0008] Preferably, step S20, which involves performing a target range filtering task based on the frame alignment dataset using the Euclidean space envelope filtering method and outputting the target set within the spatial range, specifically includes: Step S201: Based on the frame alignment dataset, establish point cloud spatial coordinates in the lidar coordinate system, wherein the lidar coordinate system has the X-axis pointing forward, the Y-axis pointing left, and the Z-axis pointing upward. Step S202: Preset forward distance threshold, lateral distance threshold and backward distance threshold, and construct the annotation space range according to the forward distance threshold, lateral distance threshold and backward distance threshold; Step S203: Identify objects located within the labeled space and having target contour point clouds as targets to be labeled, thus obtaining a target set within the space.

[0009] Preferably, in step S202, the forward distance threshold The lateral distance threshold is 100m. The backward distance threshold is 30m. The value is 30m; when the target object is outside the labeled space range, no 3D bounding box annotation is generated; when the target object is within the labeled space range but the point cloud is so sparse that the target spatial position cannot be determined based on the point cloud of adjacent frames and image projection, the target object is marked as an object to be confirmed.

[0010] Preferably, step S30, which involves performing a label vector construction task based on the target set within the spatial range using the KITTI extended label format and outputting an initial target label vector set, specifically includes: Step S301: For each target in the target set within the spatial range, generate a label vector containing fields such as category identifier, instance identifier, three-dimensional position, three-dimensional size, attitude angle, and number of radar points; Step S302: When it is detected that there is a missing local point cloud of a target with an instance identifier in the current point cloud frame, and there is a target point cloud with the same instance identifier in the adjacent point cloud frame, the missing area in the current point cloud frame is filled in according to the three-dimensional size of the same instance in the adjacent point cloud frames, and the initial three-dimensional bounding box is output. Step S303: Count the number of radar points falling within the initial 3D bounding box, and write the number of radar points into the annotation vector to obtain the target initial annotation vector set.

[0011] Preferably, in step S202, the annotation space range satisfies:

[0012] In the formula, Indicates the spatial range of the annotation. Indicates the first Point cloud points, Indicates the forward distance threshold. Indicates the lateral distance threshold. This represents the backward distance threshold.

[0013] Preferably, step S40, which involves performing a 3D bounding box correction task based on the initial target annotation vector set using perspective projection verification and point cloud geometric fitting, and outputting the corrected annotation vector set, specifically includes: Step S401: Adjust the 3D bounding box based on the initial target annotation vector set, so that the side of the 3D bounding box in the top view is parallel to the edge of the main point cloud of the target, and the bottom surface of the 3D bounding box is parallel to the fitting plane of the ground point cloud. Step S402: Control the minimum interval between the surface of the 3D bounding box and the outer edge of the target point cloud to satisfy: This invention also provides a 3D target annotation system for lidar point clouds, comprising: The frame alignment module is used to acquire multi-sensor data of the target scene, perform frame alignment tasks based on the multi-sensor data using a timestamp synchronization verification method, and output a frame alignment dataset. The target filtering module is used to perform a target range filtering task based on the frame alignment dataset using the Euclidean space envelope filtering method, and output the target set within the spatial range; The vector construction module is used to perform a label vector construction task based on the target set within the spatial range using the KITTI extended label format, and output the initial target label vector set; The bounding box correction module is used to perform a 3D bounding box correction task based on the target initial annotation vector set by using perspective projection verification and point cloud geometric fitting, and outputs a corrected annotation vector set; The export module is used to perform cross-frame consistency verification and label export based on the modified annotation vector set, and output the delivery annotation dataset.

[0014] The present invention also provides a three-dimensional target annotation device for lidar point clouds. The three-dimensional target annotation device for lidar point clouds includes: a memory, a processor, and a three-dimensional target annotation program for lidar point clouds stored in the memory and executable on the processor. When the three-dimensional target annotation program for lidar point clouds is executed by the processor, it implements the above-described method.

[0015] The present invention also provides a computer program product, which includes a 3D target annotation program for lidar point clouds, wherein the 3D target annotation program for lidar point clouds implements the above-described method when executed by a processor.

[0016] The beneficial effects of this invention are as follows: 1. This invention significantly improves the overall efficiency and accuracy of 3D target annotation by constructing a complete automated process, from multi-sensor data synchronization and alignment, spatial target selection, initial annotation vector generation, bounding box refinement based on multi-source information fusion, to cross-frame consistency verification. Specifically, timestamp synchronization verification ensures the spatiotemporal alignment of point cloud and image data, providing a reliable foundation for subsequent perspective projection verification; Euclidean space envelope selection effectively focuses on regions with high annotation value, reducing invalid computations; the initial vector construction in the KITTI extended format provides a structured data foundation for automated processing; the dual correction mechanism of perspective projection and point cloud geometric fit fully utilizes image semantic information and point cloud spatial geometric information, making the positioning, size, and pose of the 3D bounding box more accurate; and the final cross-frame consistency verification further ensures the smoothness and reliability of the annotation results in the temporal dimension, generating a high-quality delivery dataset that can be directly used for model training.

[0017] 2. The method proposed in this invention possesses strong robustness and practicality. By pre-setting point cloud annotation conditions, annotation spatial range thresholds, and dynamically selecting the priority correction criterion (image projection or point cloud outer edge) based on target distance, it can effectively address practical problems such as point cloud data anomalies, difficulties in annotating distant targets, and sensor extrinsic parameter calibration errors. For example, for distant targets with sparse point clouds, the method prioritizes point cloud geometric information for correction, avoiding annotation errors caused by blurred or deviated image projections; for targets at medium to close distances with high extrinsic parameter calibration accuracy, it fully utilizes high-resolution image information for fine-tuning. This adaptive strategy ensures the annotation quality of the method under different scenarios and conditions, reduces dependence on the perfection of external conditions, and enables the method to be widely applied in practical road scene data annotation tasks. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the first embodiment of a three-dimensional target annotation method for lidar point clouds according to the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1: As Figure 1 The diagram shown is a flowchart of a first embodiment of a three-dimensional target annotation method for lidar point clouds according to the present invention, which presents a first embodiment of a three-dimensional target annotation method for lidar point clouds according to the present invention.

[0022] In the first embodiment, the three-dimensional target annotation method for lidar point clouds includes: Step S10: Acquire multi-sensor data of the target scene, perform frame alignment task based on the multi-sensor data using timestamp synchronization verification method, and output frame alignment dataset; The "multi-sensor data acquisition" in this step specifically refers to the LiDAR point cloud sequence files stored in the target scene directory, the camera image sequence files from multiple different perspectives (such as forward-looking, side-looking, and rear-looking), and the "external parameter calibration data" obtained through calibration that describes the spatial position and attitude relationship between the camera and the LiDAR. The timestamp synchronization verification method uses the time of LiDAR point cloud data acquisition (timestamp) as the reference timeline, precisely matching the image frames acquired by each camera with their corresponding timestamps to ensure a one-to-one correspondence between point cloud data and image data describing the scene at the same moment. The "frame alignment dataset" output by the frame alignment task is a filtered and paired dataset, where each set of data contains a valid LiDAR point cloud frame and multiple time-synchronized image frames from multiple perspectives. This lays the data foundation for using image information to assist in point cloud annotation in subsequent steps.

[0023] This step plays a crucial role in data preprocessing and synchronous calibration in this invention. It ensures that all subsequent processing steps are based on spatiotemporally consistent multi-source data, avoiding data misalignment issues caused by different sensor acquisition frequencies or clock deviations. By performing point cloud annotation condition verification (such as file readability, coordinate normality, and target area point cloud validity) and simultaneously removing invalid frames, the quality of data input to downstream processes is guaranteed, reducing annotation errors or process interruptions caused by data anomalies at the source.

[0024] Compared to traditional techniques that might simply match point cloud sequences and image sequences by filename order or perform only rough time alignment, this invention achieves higher-precision frame-level data alignment through precise timestamp synchronization based on LiDAR time and rigorous validity verification. This solves the problem of inaccurate correspondence between images and point clouds caused by synchronization errors in traditional methods, which in turn affects the effectiveness of image-based assisted annotation. It provides a reliable prerequisite for subsequent high-precision 3D correction by fusing visual information.

[0025] For example, in data acquisition for an urban road scene, a LiDAR scanner scans at a frequency of 10Hz, while a forward-looking camera captures images at a frequency of 30Hz. Traditional methods might directly compare the point cloud of the Nth frame with the point cloud of the Nth frame. Three image frames are associated, but due to startup delays or clock drift, they may not actually describe the same instant. This invention uses timestamp matching to associate the 100th frame point cloud (timestamp t=10.000s) with the forward-looking camera image frame with the closest timestamp t=10.005s, even if their frame numbers are not simple multiples of each other. Simultaneously, if the 101st frame point cloud is found to have all coordinates as zero due to a brief equipment malfunction, it will be determined as an invalid frame and deleted in this step, along with all associated image frames, ensuring the validity of the dataset.

[0026] Step S20: Based on the frame alignment dataset, perform the target range filtering task using the Euclidean space envelope filtering method, and output the target set within the spatial range; The "Euclidean spatial envelope filtering method" in this step refers to pre-setting a three-dimensional cubic or prismatic spatial range in the LiDAR coordinate system (origin at the LiDAR center, X-axis pointing forward, Y-axis pointing left, Z-axis pointing upward) based on actual annotation requirements and application scenarios. This range is jointly defined by the forward distance threshold (Df, e.g., 100 meters), the lateral distance threshold (Ds, e.g., 30 meters), and the backward distance threshold (Db, e.g., 30 meters). The mathematical definition of the annotation spatial range Ω is clear, ensuring the logical consistency of the filtering. When performing the target range filtering task, the point cloud in the frame alignment dataset is traversed, and the point cloud clusters whose spatial coordinates fall within Ω are initially identified as "targets to be labeled," forming a "target set within the spatial range." This target set is the input for subsequent specific 3D bounding box annotation operations; it filters out irrelevant objects that are too far away or too far off the side or rear of the road.

[0027] This step in this invention achieves preliminary spatial localization and focusing of the labeled targets. By setting a reasonable physical spatial threshold, it concentrates computational resources and labeling attention on the most relevant road users and obstacles around the autonomous vehicle that are most likely to interact. This is not only for efficiency considerations, but also because beyond a certain distance, the point cloud becomes too sparse, and the accuracy of the labeling and its value for training the perception model will significantly decrease. This step provides a clear spatial boundary for the automation process, giving subsequent label vector construction and correction operations a clearly defined set of processing objects.

[0028] Compared to traditional techniques that either indiscriminately process point clouds across the entire scene (computationally intensive and containing numerous invalid annotations) or rely on manual selection of regions of interest in a visualization interface (subjective, inefficient, and inconsistent), this invention achieves objectivity, automation, and standardization in target pre-screening through parameterized and quantifiable Euclidean space envelopes. It ensures the spatial consistency of data annotations across different batches and scenes, providing a guarantee for generating standardized training datasets.

[0029] For example, in a highway scenario, there is a truck 120 meters directly in front of an autonomous vehicle and a car 25 meters to its left rear. According to the preset thresholds (Df=100m, Ds=30m, Db=30m), the truck 120 meters in front will be excluded from the spatial range of this annotation because its X coordinate (approximately 120) is greater than Df (100), and no annotation will be generated for it, because its point cloud may be very sparse and has little impact on near-end decision-making. The car 25 meters to its left rear, on the other hand, has coordinates (assuming X=-20, Y=25) that satisfy the conditions -Db(-30)≤X(-20)≤Df(100) and |Y|(25)≤Ds(30), and is therefore included in the "target set within the spatial range" and becomes an object to be annotated. For targets that are located near the boundary but have extremely sparse point clouds, the method can also mark them as "objects to be confirmed", indicating that special processing or manual review may be required later.

[0030] Step S30: Based on the target set within the spatial range, perform the annotation vector construction task using the KITTI extended annotation format, and output the initial target annotation vector set; The "KITTI Extended Annotation Format" in this step refers to a structured data representation method. Its annotation vector typically includes the target category (e.g., "Car", "Pedestrian"), instance ID (for cross-frame tracking), the coordinates (x, y, z) of the 3D bounding box center point in the LiDAR coordinate system, the dimensions of the 3D bounding box (length, width, height), the bounding box's orientation angle (usually the rotation angle around the Z-axis, i.e., yaw angle), and an extended field "number of radar points". The annotation vector construction task is to generate such a structured initial description for each target object in the "target set within the spatial range". Specifically, for cases of localized point cloud missingness (e.g., occlusion), the method utilizes the stable size information of the same instance in adjacent frames (associated through instance IDs) to complete the initial 3D bounding box of the current frame, resulting in a more reasonable initial geometry. The number of point clouds falling within this initial bounding box is counted and recorded, providing a reference for evaluating annotation quality and subsequent corrections.

[0031] This step in this invention completes the initial conversion from spatial point cloud clusters to structured labeled data. It transforms visually identifiable "targets" into machine-readable, uniformly formatted "label vectors," providing a standardized intermediate data representation for the entire automated pipeline. By using information from adjacent frames to fill in missing regions, the method's robustness to transient occlusion and other situations is enhanced, avoiding the generation of obviously erroneous initial bounding boxes due to incomplete information in a single frame. Recording the number of radar points provides a quantitative basis for subsequent steps to determine label confidence and select correction strategies.

[0032] Compared to traditional techniques where manually drawing 3D bounding boxes results in arbitrary starting box positions and sizes, or simple automated methods that generate bounding boxes based solely on current frame point cloud clustering (sensitive to occlusion), this invention ensures data output compatibility by introducing the KITTI standard format and employs a cross-frame size completion strategy to achieve better spatiotemporal consistency and rationality in the generated initial annotation vectors. This provides a higher-quality starting point for subsequent fine-tuning, reduces the workload required for correction, and improves the overall stability of the process.

[0033] For example, for a target in the "target set within the spatial range", an initial 3D bounding box is first estimated based on the distribution of its point cloud clusters, and assigned the category "Car" and instance ID "Car003". Suppose that in the current frame, the rear of the car is partially obscured by another car, resulting in missing point clouds and a shorter directly calculated bounding box length. In this case, step S302 detects that the bounding box length of the target with instance ID "Car003" in the previous frame was 4.5 meters, and uses this length value to complete the initial bounding box of the current frame, generating an initial 3D bounding box with a length of approximately 4.5 meters. Next, step S303 counts how many laser points actually fall within this completed bounding box in the current frame (e.g., 85 points are counted), and writes this number "85" into the "Radar Point Count" field of the annotation vector.

[0034] Step S40: Based on the initial target annotation vector set, perform a 3D bounding box correction task using perspective projection verification and point cloud geometric fitting, and output the corrected annotation vector set; This step is the core of the annotation quality improvement process. It employs two methods—"perspective projection verification" and "point cloud geometric fitting"—to collaboratively correct the 3D bounding box. "Perspective projection verification" involves using the aligned image data and camera-LiDAR extrinsic parameters from step S10 to project the 3D bounding box obtained in the previous step onto a 2D image plane, forming a 2D image projection frame. Annotators or automated algorithms can check the degree of fit between this projection frame and the target contour in the image. "Point cloud geometric fitting" refers to directly adjusting the pose and size of the bounding box in the 3D point cloud space, making its surface as close as possible to the outer edge of the target point cloud, and satisfying specific geometric constraints (e.g., the sides of the bounding box are parallel to the main edges of the point cloud, and the bottom surface is parallel to the fitted ground plane). The method defines a minimum interval range (e.g., 0.20 meters to 0.30 meters) between the bounding box surface and the outer edge of the point cloud as a quantitative indicator of the fit. The correction strategy dynamically adjusts its priority based on the target distance and sensor calibration accuracy to achieve the optimal correction effect.

[0035] This step represents a significant leap in annotation accuracy in this invention. It no longer relies solely on point clouds or images, but creatively combines the advantages of both: images provide rich texture and semantic boundary information, especially when the target's side profile is clear; point clouds provide precise 3D spatial geometry information. This multi-source information fusion correction mechanism can significantly correct potential positional deviations, dimensional errors, and pose inaccuracies in the initial annotation vectors. The dynamic priority strategy demonstrates the method's intelligence, ensuring that the most reliable correction criteria are used under different observation conditions.

[0036] Compared to traditional techniques that rely primarily on annotators manually fine-tuning the bounding box in a 3D point cloud view (relying on spatial imagination and struggling to precisely align image edges), or using only image projection for 2D constraints (losing depth information and failing to precisely control 3D dimensions), this invention introduces image projection as a strong constraint into the 3D correction process, supplemented by precise point cloud geometric fit requirements, achieving "what you see is what you get" accurate annotation. It solves the pain point of difficulty in precisely aligning the 3D bounding box with the 2D image contour during manual annotation, and overcomes the shortcomings of simple image guidance being susceptible to perspective distortion and calibration errors. This ensures that the final 3D bounding box fits the target both in the image view and is reasonably supported by point cloud geometry in 3D space.

[0037] For example, for a car at a medium distance (50 meters), its initial 3D bounding box may have a slight rotational deviation in the top view. Step S401 first adjusts the bounding box so that its sides are parallel to the main axis of the car body displayed in the point cloud, while ensuring that the bottom of the box is parallel to the plane fitted from the ground point cloud. Step S402 fine-tunes the size of the bounding box so that the distance from each surface to the nearest car body point cloud is controlled at about 0.25 meters. Then, step S403 projects the 3D bounding box onto the front-view camera image and finds that the upper edge of the projection frame is slightly lower than the actual outline of the car roof in the image. Since the target distance (50 meters) is less than 80 meters and the extrinsic parameter calibration accuracy is high, the height of the 3D bounding box is slightly increased based on the feedback from the image projection frame, so that its projection in the image perfectly matches the outline of the car roof. After this series of adjustments, the output corrected annotation vector set contains a more accurate 3D bounding box description in terms of spatial position, size, and orientation.

[0038] Step S50: Perform cross-frame consistency verification and label export based on the modified annotation vector set, and output the delivery annotation dataset.

[0039] This step is the final stage of the annotation process and a quality assurance step. "Cross-frame consistency verification" refers to checking and optimizing the annotation results for the same target instance (identified by its instance ID) across the entire time series (multi-frame point cloud). This includes checking for unreasonable abrupt changes or jitters in its 3D position, size, and pose between consecutive frames. For example, the change in the bounding box center position of a vehicle moving at a constant speed between adjacent frames should correspond to its speed. "Label export" involves serializing and storing the "corrected annotation vector set," processed and verified through all the aforementioned steps, according to a specified data format (such as a KITTI format text file, with each line corresponding to a target's annotation vector), generating the final "delivery annotation dataset." This dataset can be directly used to train autonomous driving perception models for 3D target detection, tracking, and other applications.

[0040] This step ensures the smoothness and reliability of the annotation results over time, generating a final product ready for direct delivery. Even the most accurate annotations within a single frame can be detrimental to perception models requiring temporal information (such as target tracking) if they fluctuate drastically over time. Cross-frame consistency verification is equivalent to filtering and smoothing the annotation results in the time domain, eliminating random noise or errors that might be introduced by single-frame processing, making the annotation data more consistent with the continuity of motion in the physical world. Label export completes the conversion from internally processed data to standard format output data, serving as the final output interface of the entire automated pipeline.

[0041] Compared to traditional techniques that typically involve independent annotation on a frame-by-frame basis, where annotators struggle to maintain continuity and consistency of the same target annotation across dozens or hundreds of frames, leading to temporal "jitter" in the generated annotation dataset, this invention solves this problem by introducing automated cross-frame consistency checks. This ensures that the final output annotation data is not only accurate within a single frame but also smooth, coherent, and conforms to motion patterns throughout the entire sequence. This significantly enhances the value of the annotation dataset for training time-series-aware models. Traditional manual annotation requires a significant amount of time for frame-by-frame checks and adjustments to achieve the same level of consistency; this invention automates this process, significantly improving both efficiency and quality.

[0042] For example, a bicycle (instance ID "Cyclist001") is labeled in 10 consecutive frames of point cloud. After correction in step S40, each frame has a corrected 3D bounding box. In the cross-frame consistency check in step S50, it is found that the X-coordinate of the bicycle's bounding box center in frame 5 suddenly jumps forward by 3 meters compared to frame 4 (while the reasonable displacement should be around 1 meter based on the point cloud velocity estimation), while the position in frame 6 returns to the reasonable trajectory. It is determined that the labeling in frame 5 may be abnormal, and the bounding box center coordinate of frame 5 may be smoothly interpolated and corrected based on the position information of adjacent frames (frames 4 and 6). Finally, the final label vector of "Cyclist001" in all frames (including the corrected frame 5), along with the label vectors of all other targets, are written into a text file organized by frame, forming a well-structured and spatiotemporally consistent delivery labeling dataset.

[0043] Example 2: Furthermore, the present invention provides a 3D target annotation system for lidar point clouds, employing a 3D target annotation method for lidar point clouds from the above embodiments, which can solve a technical problem related to 3D target annotation of lidar point clouds. The beneficial effects of the 3D target annotation system for lidar point clouds provided by the present invention are the same as those of the 3D target annotation method for lidar point clouds provided in the above embodiments, and other technical features of the 3D target annotation system for lidar point clouds are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0044] Example 3: This invention provides a 3D target annotation device for LiDAR point clouds. The device includes: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which are then executed to enable the at least one processor to perform the 3D target annotation method for LiDAR point clouds described in Example 1. The 3D target annotation device for LiDAR point clouds in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. This 3D target annotation device for LiDAR point clouds is merely an example and should not limit the functionality or scope of the invention. The device may also include a processing unit (e.g., a central processing unit, a graphics processing unit), which can perform various appropriate actions and processes based on a program stored in a read-only memory or a program loaded from a storage device into a random access memory. The random access memory also stores various programs and data required for the operation of a 3D target annotation device for LiDAR point clouds. The processing unit, read-only memory, and random access memory are interconnected via a bus. The I / O interface is also connected to the bus. Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication device allows a 3D target annotation device for LiDAR point clouds to communicate wirelessly or wiredly with other devices to exchange data. While a 3D target annotation device for LiDAR point clouds with various systems has been described, it should be understood that it is not required to implement or possess all the systems described. Alternatively, more or fewer systems may be implemented.

[0045] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for three-dimensional target annotation of LiDAR point clouds. The computer program product provided by this invention can solve a technical problem related to three-dimensional target annotation of LiDAR point clouds. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as those of the three-dimensional target annotation method for LiDAR point clouds provided in the above embodiments, and will not be repeated here.

[0046] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a read-only memory. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0047] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0048] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for three-dimensional target annotation of lidar point clouds, characterized in that, The methods include: Step S10: Acquire multi-sensor data of the target scene, perform frame alignment task based on the multi-sensor data using a timestamp synchronization verification method, and output a frame alignment dataset; wherein, the steps of acquiring multi-sensor data of the target scene, performing frame alignment task based on the multi-sensor data using a timestamp synchronization verification method, and outputting a frame alignment dataset specifically include: Read the LiDAR point cloud sequence, image sequences from multiple camera perspectives, and extrinsic parameter calibration data between the camera and LiDAR from the target scene directory; Based on the timestamp of the lidar point cloud sequence, image sequences from multiple camera perspectives are synchronously matched to obtain the timestamp correspondence between point cloud frames and image frames. Delete invalid point cloud frames in the lidar point cloud sequence that do not meet the preset point cloud labeling conditions, and simultaneously delete corresponding image frames in image sequences from multiple camera perspectives according to the timestamp correspondence to obtain a frame alignment dataset. The conditions for point cloud annotation include that the point cloud file is readable, the point cloud coordinates have not undergone overall abnormal shift, and there are effective point clouds in the target area for 3D bounding box fitting. Step S20: Based on the frame alignment dataset, perform a target range filtering task using the Euclidean space envelope filtering method, and output the target set within the spatial range; wherein, the step of performing a target range filtering task using the Euclidean space envelope filtering method based on the frame alignment dataset and outputting the target set within the spatial range specifically includes: Based on the frame alignment dataset, point cloud spatial coordinates are established in the lidar coordinate system, where the lidar coordinate system has the X-axis pointing forward, the Y-axis pointing left, and the Z-axis pointing upward. Preset forward distance thresholds, lateral distance thresholds, and backward distance thresholds, and construct the annotation space range according to the forward distance thresholds, lateral distance thresholds, and backward distance thresholds; Objects located within the labeled space and having target contour point clouds are identified as targets to be labeled, thus obtaining a target set within the space. Step S30: Based on the target set within the spatial range, perform a label vector construction task using the KITTI extended label format, and output the initial target label vector set; wherein, the step of performing a label vector construction task based on the target set within the spatial range using the KITTI extended label format and outputting the initial target label vector set specifically includes: For each target in the target set within the spatial range, a label vector is generated containing fields such as category identifier, instance identifier, three-dimensional position, three-dimensional size, attitude angle, and number of radar points; When a missing local point cloud of a target with an instance identifier is detected in the current point cloud frame, and a target point cloud with the same instance identifier exists in an adjacent point cloud frame, the missing region in the current point cloud frame is filled in according to the 3D size of the same instance in the adjacent point cloud frames, and the initial 3D bounding box is output. The number of radar points falling within the initial 3D bounding box is counted, and the number of radar points is written into the annotation vector to obtain the initial target annotation vector set; Step S40: Based on the initial target annotation vector set, perform a 3D bounding box correction task using perspective projection verification and point cloud geometric fitting, and output the corrected annotation vector set; Step S50: Perform cross-frame consistency verification and label export based on the modified annotation vector set, and output the delivery annotation dataset.

2. The three-dimensional target annotation method for lidar point clouds as described in claim 1, characterized in that, In step S20, the forward distance threshold The lateral distance threshold is 100m. The backward distance threshold is 30m. The value is 30m; when the target object is outside the labeled space range, no 3D bounding box annotation is generated; when the target object is within the labeled space range but the point cloud is so sparse that the target spatial position cannot be determined based on the point cloud of adjacent frames and image projection, the target object is marked as an object to be confirmed.

3. The three-dimensional target annotation method for lidar point clouds as described in claim 1, characterized in that, In step S20, the labeled spatial range satisfies: In the formula, Indicates the spatial range of the annotation. Indicates the first Point cloud points, Indicates the forward distance threshold. Indicates the lateral distance threshold. This represents the backward distance threshold.

4. The three-dimensional target annotation method for lidar point clouds as described in claim 3, characterized in that, Step S40, which involves performing a 3D bounding box correction task based on the initial target annotation vector set using perspective projection verification and point cloud geometric fitting, and outputting the corrected annotation vector set, specifically includes: Step S401: Adjust the 3D bounding box based on the initial target annotation vector set, so that the side of the 3D bounding box in the top view is parallel to the edge of the main point cloud of the target, and the bottom surface of the 3D bounding box is parallel to the fitting plane of the ground point cloud. Step S402: Control the minimum interval between the surface of the 3D bounding box and the outer edge of the target point cloud to satisfy: In the formula, Indicates the first The 3D bounding box of the target. Indicates the first A point cloud set of targets, This represents the minimum distance between the surface of the 3D bounding box and the outer edge of the target point cloud; Step S403: Using the extrinsic calibration data between the camera and the lidar, the 3D bounding box is projected onto the camera image to obtain an image projection frame; when the forward distance is no more than 80m and the extrinsic calibration meets the preset accuracy conditions, the 3D bounding box is corrected based on the image projection frame; when the forward distance is greater than 80m or the lateral distance is greater than 20m, the 3D bounding box is corrected based on the outer edge of the target point cloud to obtain a corrected annotation vector set.

5. A three-dimensional target annotation system for lidar point clouds, applied to the three-dimensional target annotation method for lidar point clouds according to any one of claims 1 to 4, characterized in that, The 3D target annotation system for the lidar point cloud includes: The frame alignment module is used to acquire multi-sensor data of the target scene, perform frame alignment tasks based on the multi-sensor data using a timestamp synchronization verification method, and output a frame alignment dataset. Specifically, the steps of acquiring multi-sensor data of the target scene, performing frame alignment tasks based on the multi-sensor data using a timestamp synchronization verification method, and outputting a frame alignment dataset include: Read the LiDAR point cloud sequence, image sequences from multiple camera perspectives, and extrinsic parameter calibration data between the camera and LiDAR from the target scene directory; Based on the timestamp of the lidar point cloud sequence, image sequences from multiple camera perspectives are synchronously matched to obtain the timestamp correspondence between point cloud frames and image frames. Delete invalid point cloud frames in the lidar point cloud sequence that do not meet the preset point cloud labeling conditions, and simultaneously delete corresponding image frames in image sequences from multiple camera perspectives according to the timestamp correspondence to obtain a frame alignment dataset. The conditions for point cloud annotation include that the point cloud file is readable, the point cloud coordinates have not undergone overall abnormal shift, and there are effective point clouds in the target area for 3D bounding box fitting. The target filtering module is used to perform a target range filtering task based on the frame alignment dataset using the Euclidean space envelope filtering method, and output a target set within the spatial range; wherein, the step of performing the target range filtering task based on the frame alignment dataset using the Euclidean space envelope filtering method and outputting a target set within the spatial range specifically includes: Based on the frame alignment dataset, point cloud spatial coordinates are established in the lidar coordinate system, where the lidar coordinate system has the X-axis pointing forward, the Y-axis pointing left, and the Z-axis pointing upward. Preset forward distance thresholds, lateral distance thresholds, and backward distance thresholds, and construct the annotation space range according to the forward distance thresholds, lateral distance thresholds, and backward distance thresholds; Objects located within the labeled space and having target contour point clouds are identified as targets to be labeled, thus obtaining a target set within the space. The vector construction module is used to perform a label vector construction task based on the target set within the spatial range using the KITTI extended label format, and output an initial target label vector set; wherein, the step of performing the label vector construction task based on the target set within the spatial range using the KITTI extended label format and outputting the initial target label vector set specifically includes: For each target in the target set within the spatial range, a label vector is generated containing fields such as category identifier, instance identifier, three-dimensional position, three-dimensional size, attitude angle, and number of radar points; When a missing local point cloud of a target with an instance identifier is detected in the current point cloud frame, and a target point cloud with the same instance identifier exists in an adjacent point cloud frame, the missing region in the current point cloud frame is filled in according to the 3D size of the same instance in the adjacent point cloud frames, and the initial 3D bounding box is output. The number of radar points falling within the initial 3D bounding box is counted, and the number of radar points is written into the annotation vector to obtain the initial target annotation vector set; The bounding box correction module is used to perform a 3D bounding box correction task based on the target initial annotation vector set by using perspective projection verification and point cloud geometric fitting, and outputs a corrected annotation vector set; The export module is used to perform cross-frame consistency verification and label export based on the modified annotation vector set, and output the delivery annotation dataset.

6. A three-dimensional target annotation device for lidar point clouds, characterized in that, The 3D target annotation device for the lidar point cloud includes: a memory, a processor, and a 3D target annotation program for the lidar point cloud stored in the memory and executable on the processor. When the 3D target annotation program for the lidar point cloud is executed by the processor, it implements a 3D target annotation method for lidar point clouds according to any one of claims 1 to 4.

7. A computer program product, characterized in that, The computer program product includes a 3D target annotation program for lidar point clouds, which, when executed by a processor, implements a 3D target annotation method for lidar point clouds according to any one of claims 1 to 4.

Images