Method, system, device and medium for generating label data

Abstract

This disclosure provides a method, system, device, and medium for generating labeled data. The generation method includes: acquiring original foreground target points and original background target points from a radar point cloud; generating corresponding dense foreground target points and dense background target points based on the foreground target points and the background target points; transforming the dense foreground target points and the dense background target points into their corresponding original bounding boxes and original positions, respectively, to obtain a dense point cloud; voxelizing the dense point cloud; and generating labeled data corresponding to the dense point cloud. This disclosure not only solves the problem of missing labeled data in 3D occupancy prediction tasks but also avoids the resource consumption caused by manual annotation. The labeled data generation method proposed in this solution is applicable to all outdoor road scene datasets that meet the conditions and can provide a large amount of usable data for research on 3D spatial occupancy prediction tasks in the field of autonomous driving.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a method, system, device, and medium for generating tag data. Background Technology

[0002] In the field of autonomous driving, the perception of the surrounding environment is extremely important. Understanding the occupancy status of each voxel in 3D space is helpful for vehicles to identify and avoid various obstacles in unknown environments. However, 3D occupancy network prediction is a complex task that requires a large amount of 3D scene semantic information for training. If only sparse point cloud data is used for supervision, the network will have difficulty predicting a sufficiently dense occupancy situation.

[0003] The goal of the 3D occupancy prediction task is to estimate the state of each voxel in a 3D scene using a series of sensor-inputted historical frame images (T frames). The state description includes occupancy status and semantic information. Occupancy status refers to whether the voxel is occupied or idle; semantic information refers to the object category to which the voxel belongs. Therefore, the training data requires voxel-level labels containing both occupancy and semantic information. Obtaining these labels manually would be extremely labor-intensive and inefficient. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is to overcome the shortcomings of existing technologies that require a lot of manpower and are inefficient to obtain label data directly through manual annotation, and to provide a method, system, device and medium for generating label data.

[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0006] This disclosure provides a method for generating tag data, the method comprising:

[0007] Obtain the original foreground target points and original background target points from the radar point cloud;

[0008] Based on the foreground target point and the background target point, generate corresponding dense foreground target points and dense background target points;

[0009] The dense foreground target points and the dense background target points are transformed into their corresponding original target bounding boxes and original positions, respectively, to obtain a dense point cloud;

[0010] The dense point cloud is voxelized, and the corresponding label data is generated.

[0011] Preferably, the step of generating the corresponding dense foreground target points and dense background target points includes:

[0012] The foreground target points are acquired and multi-frame densification processing is performed to generate dense foreground target points;

[0013] The background target points are acquired and multi-frame densification processing is performed to generate dense background target points.

[0014] Preferably, the step of acquiring the foreground target point and performing multi-frame densification processing includes:

[0015] Transform the points in each frame of the foreground target point to the same coordinate system and perform multi-frame aggregation, mirror symmetry and point cloud registration operations;

[0016] And / or, the step of acquiring ground points and background target points in the radar point cloud and performing multi-frame densification processing includes:

[0017] The operation involves multi-frame aggregation and least squares calibration of the points in each frame of the background target points.

[0018] Preferably, the step of generating the corresponding dense foreground target points and dense background target points includes:

[0019] The foreground target points are acquired and multi-frame densification processing is performed to generate dense foreground target points;

[0020] Ground points and background target points in the radar point cloud are acquired and multi-frame densification is performed to generate dense background target points.

[0021] Preferably, after the step of voxelizing the dense point cloud, the generation method includes:

[0022] The Poisson surface reconstruction algorithm is used to densify the voxels corresponding to dense point clouds in order to generate target voxels.

[0023] Preferably, after the step of voxelizing the dense point cloud, the generation method includes:

[0024] Obtain the line connecting each voxel of the dense point cloud to the camera optical center from each camera viewpoint;

[0025] Based on the connection, a camera visibility mask is generated.

[0026] Preferably, after the step of generating the label data corresponding to the dense point cloud, the generation method includes:

[0027] Based on the label data, a corresponding occupied or unoccupied mask is generated.

[0028] This disclosure provides a system for generating tag data, the system comprising:

[0029] The first acquisition module is used to acquire the original foreground target points and the original background target points in the radar point cloud;

[0030] The first generation module is used to generate corresponding dense foreground target points and dense background target points based on the foreground target points and the background target points;

[0031] The transformation module is used to transform the dense foreground target points and the dense background target points to their corresponding original target boxes and original positions, respectively, to obtain a dense point cloud;

[0032] The second generation module is used to voxelize the dense point cloud and generate label data corresponding to the dense point cloud.

[0033] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the aforementioned method for generating tag data.

[0034] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method for generating tag data.

[0035] The positive and progressive effects of this disclosure are as follows:

[0036] This disclosure utilizes existing radar point cloud data in existing datasets to automatically generate label data for dense occupancy network prediction. This not only solves the problem of missing label data in 3D occupancy network prediction tasks but also avoids the resource consumption caused by manual annotation. Furthermore, the label data generation method proposed in this solution is applicable to all outdoor road scene datasets that meet the requirements, providing a large amount of usable data for research on 3D spatial occupancy prediction tasks in the field of autonomous driving. Attached Figure Description

[0037] Figure 1 This is a flowchart of a common method for generating tag data in Embodiment 1 of this disclosure;

[0038] Figure 2 This is a first flowchart of step S102 in Embodiment 1 of this disclosure;

[0039] Figure 3 This is a second flowchart of step S102 in Embodiment 1 of this disclosure;

[0040] Figure 4 This is a schematic diagram of the tag data generation system in Embodiment 2 of this disclosure;

[0041] Figure 5This is a schematic diagram of the first generation module in Embodiment 2 of this disclosure;

[0042] Figure 6 This is a schematic diagram of the structure of the electronic device according to Embodiment 3 of this disclosure. Detailed Implementation

[0043] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0044] Example 1

[0045] like Figure 1 As shown, this embodiment discloses a method for generating tag data, the method comprising:

[0046] Step S101: Obtain the original foreground target points and the original background target points in the radar point cloud;

[0047] In this solution, the radar point cloud can be the radar point cloud in the outdoor road scene dataset.

[0048] Step S102: Based on the foreground target point and the background target point, generate corresponding dense foreground target points and dense background target points;

[0049] Step S103: Transform the dense foreground target points and the dense background target points into their corresponding original target boxes and original positions, respectively, to obtain a dense point cloud;

[0050] In this scheme, multiple frames of target point clouds are aggregated, and after performing denser operations such as mirror symmetry, the dense point cloud is placed back into the original target's bounding box, replacing the original sparse target point cloud.

[0051] Step S104: Voxelize the dense point cloud and generate label data corresponding to the dense point cloud.

[0052] Specifically, the dense point cloud is voxelized, and the dense point cloud is assigned a label of occupancy and a corresponding semantic label.

[0053] This solution, based on existing radar point cloud data in a current dataset, can automatically generate label data for dense occupancy network prediction. This not only solves the problem of missing label data in 3D occupancy network prediction tasks but also avoids the resource consumption caused by manual annotation. Furthermore, the label data generation method proposed in this solution is applicable to all outdoor road scene datasets that meet the requirements, providing a large amount of usable data for research on 3D spatial occupancy prediction tasks in the field of autonomous driving.

[0054] like Figure 2As shown, in one implementable manner, step S102 includes:

[0055] Step S10211: Obtain the foreground target points and perform multi-frame densification processing to generate dense foreground target points;

[0056] Specifically, the points in each frame of the foreground target are transformed to the same coordinate system and multi-frame aggregation, mirror symmetry, and point cloud registration are performed.

[0057] In this scheme, for dynamic foreground target points, in order to aggregate points belonging to the same ID (IdentityDocument) across multiple frames, the points within the 3D ground truth bounding box of each frame are first transformed to the same coordinate system. Then, all points belonging to the same ID appearing in each frame are aggregated together. To reduce jitter, RANSAC (random sample consensus) point cloud registration is also performed. Since the vehicle is symmetrical along its vertical centerline, flipping the points on the denser side of the aggregated point cloud after multiple frames to the other side using an axisymmetric operation yields even denser data. Simultaneously, velocity estimation is performed on all foreground target points. These operations result in a denser set of target points, namely, a denser foreground point set P′. fg .

[0058] This solution generates sufficiently dense 3D spatial occupancy label data through multi-frame aggregation, mirror symmetry, and point cloud registration.

[0059] Step S10212: Obtain the background target points and perform multi-frame densification processing to generate dense background target points.

[0060] Specifically, the points in each frame of the background target points are aggregated across multiple frames and calibrated using the least squares method.

[0061] In this scheme, by performing multi-frame aggregation and least squares calibration on the points of each frame in the background target points, it is ensured that the target points in the background can be kept within a reasonable range of their original positions.

[0062] like Figure 3 As shown, in one implementable embodiment, step S102 further includes:

[0063] Step S10221: Obtain the foreground target points and perform multi-frame densification processing to generate dense foreground target points;

[0064] Step S10222: Obtain ground points and background target points from the radar point cloud and perform multi-frame densification processing to generate dense background target points.

[0065] This scheme, after performing multi-frame densification processing on the foreground target points to generate dense foreground target points, extracts the ground point P from the remaining radar point cloud. g Specifically, the Cloth-Simulation-Filter (CSF) algorithm can be used to extract ground points. After extracting the foreground target points and ground points, the remaining points are the background target points. The static background points P′ obtained by aggregating multiple frames are then used to extract these points. b With the aggregated ground point set P′ g The points are pieced together to form a denser background point set P′. bg This is the dense background target point, which is also the aggregated background.

[0066] In one implementable manner, after the step of voxelizing the dense point cloud, the generation method includes:

[0067] The Poisson surface reconstruction algorithm is used to densify the voxels corresponding to dense point clouds in order to generate target voxels.

[0068] This scheme utilizes the Poisson surface reconstruction algorithm to further densify the data. Specifically, the Poisson surface reconstruction algorithm is used to further densify the voxels corresponding to the dense point cloud, resulting in denser voxels.

[0069] In one implementable manner, after the step of voxelizing the dense point cloud, the generation method further includes:

[0070] Obtain the line connecting each voxel of the dense point cloud to the camera optical center from each camera viewpoint;

[0071] Based on the connection, a camera visibility mask is generated.

[0072] This scheme calculates the connection between each voxel and the camera's optical center from each camera's viewpoint. Voxels farther away on the ray than the original voxel are recorded. Voxels recorded in every camera viewpoint are labeled "unobserved," resulting in a camera visibility / invisibility mask. Specifically, for each camera view, the center of each voxel is connected to the camera center, forming a ray. Along each ray, i.e., from near to far, voxels before the first occupied voxel are set as "observed," and the remaining voxels are set as "unobserved." Voxels not scanned by any camera ray are also labeled "unobserved." In this way, a camera visibility mask is generated.

[0073] In one implementable manner, after the step of generating the label data corresponding to the dense point cloud, the generation method includes:

[0074] Based on the label data, a corresponding occupied or unoccupied mask is generated.

[0075] This scheme acquires rays at locations without radar points, samples the rays to obtain unoccupied points, generates "unoccupied" labels, and then generates a 3D occupancy or unoccupied mask based on these labels. Cells within the target's 3D ground truth bounding box are stored, while cells outside the bounding box are left empty, forming a 3D space mask. This helps avoid over-interpretation during the training of the 3D occupancy prediction network.

[0076] Example 2

[0077] like Figure 4 As shown, this embodiment discloses a tag data generation system, which is used to implement the tag data generation method of Embodiment 1. The generation system includes:

[0078] The first acquisition module 1 is used to acquire the original foreground target points and the original background target points in the radar point cloud;

[0079] The first generation module 2 is used to generate corresponding dense foreground target points and dense background target points based on the foreground target points and the background target points;

[0080] Transformation module 3 is used to transform the dense foreground target points and the dense background target points to their corresponding original target boxes and original positions, respectively, to obtain a dense point cloud;

[0081] The second generation module 4 is used to voxelize the dense point cloud and generate label data corresponding to the dense point cloud.

[0082] like Figure 5 As shown, in one implementable embodiment, the first generation module 2 includes:

[0083] The first acquisition unit 21 is used to acquire the foreground target points and perform multi-frame densification processing to generate dense foreground target points;

[0084] The second acquisition unit 22 is used to acquire the background target points and perform multi-frame densification processing to generate dense background target points.

[0085] In one implementable manner, the first acquisition unit 21 is specifically used for:

[0086] The points in each frame of the foreground target are transformed to the same coordinate system and then subjected to multi-frame aggregation, mirror symmetry, and point cloud registration.

[0087] In one implementable manner, the second acquisition unit 22 is specifically used for:

[0088] The operation involves multi-frame aggregation and least squares calibration of the points in each frame of the background target points.

[0089] like Figure 5 As shown, in one implementable embodiment, the first generation module 2 further includes:

[0090] The third acquisition unit 23 acquires the foreground target points and performs multi-frame densification processing to generate dense foreground target points;

[0091] The fourth acquisition unit 24 acquires ground points and background target points in the radar point cloud and performs multi-frame densification processing to generate dense background target points.

[0092] like Figure 4 As shown, in one implementable embodiment, the generation system includes:

[0093] The third generation module 5 is used to perform a densification operation on the voxels corresponding to the dense point cloud using the Poisson surface reconstruction algorithm in order to generate the target voxel.

[0094] In one implementable manner, the generation system further includes:

[0095] The second acquisition module 6 is used to acquire the connection between each voxel of the dense point cloud and the camera optical center from each camera viewpoint.

[0096] The fourth generation module 7 is used to generate a camera visibility mask based on the connection lines.

[0097] In one implementable manner, the generation system further includes:

[0098] The fifth generation module 8 generates corresponding occupied or unoccupied masks based on the label data.

[0099] Example 3

[0100] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the program, it implements the tag data generation method provided in Embodiment 1. Figure 6 The electronic device 40 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0101] like Figure 6As shown, the electronic device 40 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 40 may include, but are not limited to: at least one processor 41, at least one memory 42, and a bus 43 connecting different system components (including memory 42 and processor 41).

[0102] Bus 43 includes a data bus, an address bus, and a control bus.

[0103] The memory 42 may include volatile memory, such as random access memory (RAM) 421 and / or cache memory 422, and may further include read-only memory (ROM) 423.

[0104] The memory 42 may also include a program / utility 425 having a set (at least one) of program modules 424, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0105] The processor 41 executes various functional applications and data processing by running computer programs stored in the memory 42, such as the tag data generation method provided in Embodiment 1 of this disclosure.

[0106] Electronic device 40 can also communicate with one or more external devices 44 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 45. Furthermore, the model-generated device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 46. As shown, network adapter 46 communicates with other modules of the model-generated device 40 via bus 43. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 40, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0107] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0108] Example 4

[0109] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the tag data generation method provided in Embodiment 1.

[0110] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0111] In a possible implementation, this disclosure can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to execute the tag data generation method provided in Embodiment 1.

[0112] The program code for executing this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0113] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A method for generating tag data, characterized in that, The generation method includes: The original foreground target points and original background target points in the radar point cloud are obtained, and the ground points are extracted from the radar point cloud using a cloth filtering algorithm. Based on the foreground target points and the background target points, corresponding dense foreground target points and dense background target points are generated through multi-frame densification processing. Generating the dense foreground target points includes transforming the points within the 3D ground truth box of each frame of the foreground target points to the same coordinate system, performing multi-frame aggregation, RANSAC point cloud registration, and velocity estimation on the transformed foreground target points, and, based on the symmetrical characteristic of the vehicle along its vertical centerline, flipping the points on the denser side of the aggregated point cloud to the other side through an axisymmetric operation. Generating the dense background target points includes performing multi-frame aggregation and least squares calibration on the points of each frame of the background target points, and stitching the calibrated background target points with the ground points. The dense foreground target points and the dense background target points are transformed into their corresponding original target bounding boxes and original positions, respectively, to obtain a dense point cloud; The dense point cloud is voxelized, and the corresponding label data is generated.

2. The method for generating tag data as described in claim 1, characterized in that, The steps for generating the corresponding dense foreground target points and dense background target points include: The foreground target points are acquired and multi-frame densification processing is performed to generate dense foreground target points; Ground points and background target points in the radar point cloud are acquired and multi-frame densification is performed to generate dense background target points.

3. The method for generating tag data as described in claim 1, characterized in that, After the step of voxelizing the dense point cloud, the generation method includes: The Poisson surface reconstruction algorithm is used to densify the voxels corresponding to dense point clouds in order to generate target voxels.

4. The method for generating tag data as described in claim 1, characterized in that, After the step of voxelizing the dense point cloud, the generation method includes: Obtain the line connecting each voxel of the dense point cloud to the camera optical center from each camera viewpoint; Based on the connection, a camera visibility mask is generated.

5. The method for generating tag data as described in claim 1, characterized in that, After the step of generating the label data corresponding to the dense point cloud, the generation method includes: Based on the label data, a corresponding occupied or unoccupied mask is generated.

6. A system for generating tag data, characterized in that, The generation system includes: The first acquisition module is used to acquire the original foreground target points and the original background target points in the radar point cloud, and to extract ground points from the radar point cloud using a cloth filtering algorithm. The first generation module is used to generate corresponding dense foreground target points and dense background target points based on the foreground target points and the background target points through multi-frame densification processing. Specifically, the multi-frame densification processing for generating dense foreground target points includes transforming the points within the 3D ground truth box of each frame of the foreground target points to the same coordinate system, performing multi-frame aggregation, RANSAC point cloud registration, and velocity estimation on the transformed foreground target points, and, based on the symmetrical characteristic of the vehicle along its vertical centerline, flipping the points on the denser side of the aggregated point cloud to the other side through an axisymmetric operation. The multi-frame densification processing for generating dense background target points includes performing multi-frame aggregation and least squares calibration on the points of each frame of the background target points, and stitching the calibrated background target points with the ground points. The transformation module is used to transform the dense foreground target points and the dense background target points to their corresponding original target boxes and original positions, respectively, to obtain a dense point cloud; The second generation module is used to voxelize the dense point cloud and generate label data corresponding to the dense point cloud.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the method for generating tag data as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the tag data generation method as described in any one of claims 1 to 5.

Images