A data processing method, apparatus, and storage medium

By back-projecting target points from the image into the point cloud, the problem of difficulty in identifying distant or small targets in the point cloud is solved, and an efficient and simple annotation process is achieved.

CN115330917BActive Publication Date: 2025-10-28BEIJING TUSEN ZHITU TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110437517.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-22
Publication Date
2025-10-28
Estimated Expiration
2041-04-22

AI Technical Summary

Technical Problem

In existing technologies, it is difficult for annotators to efficiently identify distant or small targets in point clouds, leading to the problem of missed annotations.

Method used

By acquiring the image annotation data, auxiliary annotation information of the target point in the point cloud frame is generated. Using the correspondence between the image and the point cloud, the target point is back-projected into the point cloud to assist in the identification and annotation of the target object.

Benefits of technology

It improves the efficiency and accuracy of labeling distant or small targets in point clouds, simplifies the operation process, and reduces the phenomenon of missing labels.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115330917B_ABST
    Figure CN115330917B_ABST
Patent Text Reader

Abstract

This invention provides a data processing method, apparatus, and storage medium. The method includes: acquiring annotation data of at least one frame of an image, the annotation data including the position data of a target point; generating auxiliary annotation information of the target point in a point cloud frame based on the annotation data of the at least one frame of the image, wherein the at least one frame of the image and the point cloud frame have a corresponding relationship; and obtaining the annotation data of the point cloud frame based on the auxiliary annotation data. The solution provided in this application, by determining the position of a target point in an image that indicates the orientation of a target object, back-projects it into the corresponding point cloud, assisting in the identification of the target object in the point cloud and facilitating the further acquisition of annotation data in the point cloud frame.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method, apparatus, and storage medium. Background Technology

[0002] In related technologies, when annotators are annotating 3D point clouds, if there are distant or small targets to be annotated in the point cloud, the point cloud data collected by LiDAR is often sparse, and the point cloud becomes sparser the farther away it is. Therefore, annotators can easily overlook such targets, resulting in missed annotations.

[0003] Even though current solutions utilize images to assist in point cloud annotation, the process remains complex or lacks accuracy, and processing efficiency still needs improvement. Summary of the Invention

[0004] Embodiments of the present invention provide a data processing scheme to solve the problem of the inability to efficiently identify distant targets in point clouds in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] According to one aspect of this disclosure, a data processing method includes:

[0007] Obtain annotation data for at least one frame of an image, wherein the annotation data includes the location data of the target point;

[0008] Based on the annotation data of the at least one frame of image, auxiliary annotation information of the target point in the point cloud frame is generated, wherein the at least one frame of image and the point cloud frame have a corresponding relationship;

[0009] The annotation data of the point cloud frame is obtained based on the auxiliary annotation data.

[0010] According to another aspect of this disclosure, a data processing apparatus includes a processor and at least one memory, wherein at least one machine-executable instruction is stored in the at least one memory, and the processor executes the at least one machine-executable instruction to perform the method described above.

[0011] According to another aspect of this disclosure, a computer-readable storage medium has a computer program stored thereon that, when executed by a processor, implements the method described above.

[0012] The data processing scheme provided in this embodiment of the invention determines the position of the target point that indicates the orientation of the target object in the image, and back-projects it into the corresponding point cloud to assist in the identification of the target object in the point cloud and facilitate the further acquisition of the annotation data in the point cloud frame. Attached Figure Description

[0013] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In all the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 This is a structural block diagram illustrating a data processing apparatus according to an exemplary embodiment;

[0015] Figure 2 This is a schematic diagram illustrating the architecture of a data processing apparatus according to an exemplary embodiment;

[0016] Figure 3 This is a flowchart illustrating a data processing method according to an exemplary embodiment;

[0017] Figures 4a-4b This is a reference schematic diagram illustrating an exemplary embodiment;

[0018] Figures 5a-5b This is a schematic diagram illustrating a real-world application scenario according to an exemplary embodiment; Detailed Implementation

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] In this disclosure, the term "multiple" means two or more, unless otherwise stated. In this disclosure, the term "and / or" describes the relationship between related objects, encompassing any one of the listed objects and all possible combinations thereof. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0021] In this disclosure, unless otherwise stated, the terms "first," "second," etc., are used to distinguish similar objects and are not intended to limit their positional, temporal, or importance relationships. It should be understood that such terms are interchangeable where appropriate so that the embodiments of the invention described herein can be implemented in ways other than those illustrated or described herein.

[0022] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, system, product, or apparatus.

[0023] In related technologies, because the point cloud data collected by lidar is relatively sparse, and the point cloud becomes sparser the farther away, smaller targets and distant targets are often not clearly displayed in the point cloud. It is difficult to accurately identify objects in the point cloud and it is easy to miss them when labeling them.

[0024] This application provides a data processing solution that effectively utilizes the advantages of high resolution and rich color and texture information in image data. It can easily mark distant, small, and difficult-to-label targets that are hard to identify directly using point clouds in the image. Furthermore, by leveraging the relationship between the image and the point cloud, the marked target points in the image are projected back into the point cloud via a camera to indicate the target's position within the point cloud, thus achieving assisted point cloud labeling. Compared to existing solutions, this approach is simpler and easier to use.

[0025] Some embodiments of this application provide a data processing scheme. Figure 1 The structure of a data processing apparatus provided in an embodiment of this application is shown. The apparatus 1 includes a processor 11 and a memory 12.

[0026] In some embodiments, the memory 12 can be a storage device of various forms, such as transient or non-transient storage media. At least one machine-executable instruction can be stored in the memory 12, and this machine-executable instruction, when executed by the processor 11, implements the data processing method provided in the embodiments of this application.

[0027] In some embodiments, the data processing device 1 may be located on a server. In other embodiments, the data processing device 1 may also be located on a cloud server. In still other embodiments, the data processing device 1 may also be located on a client.

[0028] like Figure 2As shown, the data processing provided in this embodiment may include front-end processing 13 and back-end processing 14. Front-end processing 13 displays relevant 3D point cloud frames and / or images, and receives relevant data or information input by the annotator. For example, front-end processing 13 may be implemented through a web page or through a separate application interface. Back-end processing 14 performs corresponding data processing based on the relevant data and information received by front-end processing 13. After data processing is completed, the data processing device 1 can further provide the annotation results to clients, servers, and other processing or applications on cloud servers.

[0029] In displaying 3D point cloud data, the data can be displayed according to a specified display direction. This specified display direction can be a preset direction or a direction input by the annotator. For example, in some embodiments, after the data processing device reads a frame of 3D point cloud data, it can display that frame according to a preset display direction. As another example, in some embodiments, when the annotator needs to carefully observe the scene or object represented by the 3D point cloud data, they can select and input the desired display direction, and the data processing device will display the 3D point cloud data according to the received direction to facilitate observation and identification by the annotator.

[0030] The following describes a data processing method implemented by the data processing device 1 by executing at least one machine-executable instruction.

[0031] Figure 3 The diagram illustrates a data processing method provided in an embodiment of this application, that is, a data processing device performing data processing, including:

[0032] S301 Acquires annotation data for at least one frame of image, the annotation data including the location data of the target point;

[0033] Specifically, a data processing device can display one or more frames of images containing target points. Annotators can input annotation data for target points in the image through the human-computer interaction interface provided by the data processing device in various ways. For example, they can directly input specific parameter values ​​in the data input box in the human-computer interaction interface, click preset buttons or keys on the human-computer interface (buttons or keys have corresponding preset instructions or data), or select corresponding options in the drop-down menu provided by the human-computer interface. The drop-down menu can include one or more levels of sub-menus, and each sub-menu can include one or more options. The data processing device receives the annotation data input by the annotator through the human-computer interface.

[0034] Alternatively, target detection algorithms can be used to calculate the position coordinates of the target point in the image. In some application scenarios, annotators can manually verify and calibrate the results identified by the target detection algorithm, and use the verified and calibrated data as the annotation data of the target point in the image.

[0035] S303 generates auxiliary annotation information of the target point in the point cloud frame based on the annotation data of the at least one frame image, wherein the at least one frame image and the point cloud frame have a corresponding relationship;

[0036] Specifically, image data can be acquired via a camera, while point cloud data can be acquired via LiDAR. A point cloud is a set of points representing the surface features of a target. Point cloud data acquired via LiDAR can contain 3D coordinate information and laser reflection intensity information. After 3D scene construction and 3D coordinate transformation, it is displayed on the front end as point cloud frames. Generally, in a point cloud frame, laser points belonging to the same object are relatively concentrated and can display the approximate outline of the target object. The correspondence between image data and point cloud data can be understood as follows: for a point cloud frame, the image formed by capturing the same scene at the same time corresponds to it.

[0037] S305 obtains the annotation data of the point cloud frame based on the auxiliary annotation data.

[0038] Specifically, auxiliary annotation data can help identify the location of the target object in the point cloud frame, thereby further generating annotation data of the target object in the point cloud frame through manual input or preset rules.

[0039] according to Figure 3 The method shown involves a data processing device receiving the annotation data of target points in an image and generating auxiliary annotation information of the target points in the corresponding point cloud frame. This information is used to assist in identifying target objects in the point cloud that are difficult to identify directly, thereby enabling the quick and convenient acquisition of the annotation data of the target objects in the point cloud frame.

[0040] In some embodiments, S303 can be implemented as follows: the position data of the target point in at least one image frame is back-projected onto the point cloud frame through the corresponding camera to obtain at least one ray. Here, back-projection refers to the process of mapping a point on a two-dimensional image plane to a ray in three-dimensional space. Specifically, the back-projection of point m on the image plane refers to the set of all spatial points that have image point m under the action of camera P.

[0041] In some embodiments, annotators can manually determine the region of the target object in the point cloud frame based on the generated auxiliary annotation information and input the annotation data.

[0042] For example, S305 can be implemented as follows: displaying a point cloud frame and at least one ray, the ray indicating the orientation of the target object in the point cloud frame; and receiving the annotation data of the target object in the point cloud frame.

[0043] Specifically, the data processing device can display a point cloud frame containing the target object to be labeled and rays generated by back-projection based on the position data of the target point in the image. Each ray indicates the orientation of the target object in the point cloud frame. When a target point in an image frame is back-projected into the point cloud frame, a ray is generated. Since this ray points to the orientation of the target object in the point cloud frame, and since the point cloud density in the area where the target object is located is usually greater than the point cloud density of the surrounding environment, the labeler can quickly determine the area where the target object is located by following the direction of the ray.

[0044] like Figure 4a As shown, firstly, a point within the pixel range of the target vehicle in the image is determined as target point one. Then, target point one is back-projected onto the point cloud frame at the corresponding time through the camera corresponding to the image. The ray generated from the corresponding camera will pass through the area of ​​the target vehicle in the point cloud frame. Thus, the annotator can find the location of the target vehicle in the point cloud frame along the ray and then identify the area where the target vehicle is located based on the difference in point cloud density.

[0045] When target points in at least two image frames are back-projected onto a point cloud frame, the generated two or more rays corresponding to each target point will intersect at least one point in the point cloud frame. Annotators can use the location of this intersection point, combined with the point cloud density, to determine the region of the target object within the point cloud frame. In this way, by utilizing the intersection points generated by the rays, annotators can quickly locate the position of the target object in the point cloud frame, improving the annotation efficiency for identifying distant or difficult-to-identify objects in the point cloud.

[0046] like Figure 4b As shown, firstly, two points within the pixel range of the target vehicle are identified as target point one and target point two in two images captured by two different cameras. Then, target point one and target point two are back-projected onto the corresponding point cloud frames at the corresponding times using the cameras in the two images. The resulting rays emitted from the two cameras will pass through the area of ​​the target vehicle in the point cloud frame. Simultaneously, these two rays will intersect at a point in the point cloud. This intersection point may fall within the area of ​​the target vehicle in the point cloud frame, or it may fall near that area. The annotator can quickly pinpoint the location of the target vehicle using the intersection point, and further identify the region where the target vehicle is located based on the differences in point cloud density.

[0047] In this invention, target points in the image can be manually marked and confirmed by annotators to characterize the position of the target object in the image. Alternatively, specific locations of the target vehicle, such as left / right taillights or left / right rearview mirrors, can be used as detection targets, identified by target detection algorithms as target points. The purpose of the target points is to indicate the position of the target vehicle in the image, so that the rays generated by back projection can indicate the orientation of the target vehicle in the point cloud, assisting the annotator in the next step. The method of determining these target points is not limited here. Furthermore, the use of a target vehicle as an example in this embodiment is merely for the purpose of facilitating public understanding of the invention. Any target object that can be simultaneously captured by a point cloud and a camera can be annotated using this method.

[0048] In some embodiments, after determining the region of the target object in the point cloud frame, the annotator can further input the annotation data of the target object in the point cloud frame in various ways. For example, in a human-computer interaction interface, the coordinate data of the annotation box can be determined by clicking on points on the outline of the target object, and the corresponding object attribute option can be selected from the drop-down menu provided by the human-computer interface. Based on the outline point coordinate data input by the annotator, the data processing device can further generate and display the three-dimensional annotation box of the target object in the point cloud frame. For example, a two-dimensional bounding box can be generated first along the outline points, and then a minimum rectangle containing the two-dimensional bounding box can be generated. The length and width of the minimum rectangle are used as the length and width of the three-dimensional annotation box, and then the three-dimensional annotation box is generated according to a preset height. Furthermore, the data processing device can also display the generated three-dimensional annotation box in a selected direction to facilitate further inspection and adjustment by the annotator. It should be understood that the method of generating the three-dimensional annotation box does not constitute a limitation on the scope of protection of this application.

[0049] In some embodiments, after the annotator identifies the target object in the displayed point cloud frame, they click to select a location within the area where the target object is located in the three-dimensional coordinate system space displaying the target object. The data processing device then identifies the three-dimensional coordinates of that location as the base point coordinates (x, y, z). Alternatively, if the annotator can identify the three-dimensional coordinates of the candidate location, they can directly input the location data into the numerical input box on the human-machine interface as the base point coordinates (x, y, z). Further, the data processing device can generate a three-dimensional annotation box at the location indicated by the coordinates (x, y, z). During generation, a cuboid three-dimensional annotation box can be generated with the location indicated by the three-dimensional coordinates as the center point of a cube, or a cube-shaped three-dimensional annotation box can be generated with the location indicated by the three-dimensional coordinates as a predetermined corner point, or a cube-shaped three-dimensional annotation box can be generated with the location indicated by the three-dimensional coordinates as the center point of a face. Furthermore, the data processing device can also display the generated three-dimensional annotation box in a selected direction to facilitate further inspection and adjustment by the annotator. It should be understood that the method of generating the three-dimensional annotation box does not constitute a limitation on the scope of protection of this application.

[0050] In some embodiments, the annotation data of the target object can be automatically determined after the ray is obtained by using preset rules.

[0051] Specifically, if a ray is generated in the point cloud frame through back projection, points on the ray are searched according to preset rules to determine the coordinates of the base point indicating the position of the target object in the point cloud frame. One approach is to first set a point cloud density threshold, calculate the point cloud density value corresponding to all points on the ray, place points greater than the threshold in a preliminary screening set, and then select a point from the preliminary screening set, for example, the point with the largest point cloud density value, and use the coordinates of this point as the base point coordinates (x, y, z).

[0052] If two or more rays are generated in a point cloud frame through back projection, these rays will have at least one intersection point. The data processing device can select one of these intersection points according to a preset rule and mark its coordinates as the base point coordinates (x, y, z). The selection of the intersection point can be, but is not limited to, the following methods: selecting the intersection point with the highest corresponding point cloud density by calculating the point cloud density value of each intersection point; or selecting the intersection point with the closest average distance to all cameras; or arbitrarily selecting any intersection point. When calculating the point cloud density value of points on the rays or the density value of the intersection point, the point cloud can first be projected onto a two-dimensional plane, and then the point cloud density within a preset radius area can be calculated using the corresponding position of each point in the two-dimensional plane as the center, serving as the point cloud density value for each point. It should be understood that those skilled in the art can select an appropriate point cloud density calculation method according to actual needs, and different calculation methods do not affect the scope of protection claimed in this application.

[0053] Furthermore, after determining the base point coordinates (x, y, z), the data processing device can generate a three-dimensional annotation box at the position indicated by the base point coordinates (x, y, z) according to preset rules. During generation, a cube-shaped three-dimensional annotation box can be generated with the position indicated by the base point coordinates as the center point of the cube, or with the position indicated by the base point coordinates as a predetermined corner point, such as a vertex of the cube. Alternatively, a cube-shaped three-dimensional annotation box can be generated with the position indicated by the base point coordinates as the center point of a face. Furthermore, the data processing device can also display the generated three-dimensional annotation box in a selected direction to facilitate inspection and adjustment by the annotator. It should be understood that the method of generating the three-dimensional annotation box does not constitute a limitation on the scope of protection of this application.

[0054] Furthermore, the generated 3D annotation boxes can also have predetermined dimensions. For example, in some scenarios, the target objects to be annotated are relatively uniform, and these objects have similar or identical dimensions. In this case, the dimensions of the 3D annotation boxes can be preset, and the data processing device generates 3D annotation boxes with predetermined dimensions based on the base point coordinates. Alternatively, based on the object attributes input by the annotator and the preset relationship between object attributes and dimensions, 3D annotation boxes of different sizes corresponding to objects with different attributes can be generated. It should be understood that the rules for generating 3D annotation boxes can be flexibly set according to actual needs, and this application does not impose any limitations.

[0055] Figure 5a , 5b This demonstrates typical scenarios in which the solution is applied in practice, including Figure 5a This demonstrates how to generate an auxiliary ray in a point cloud frame by selecting target points in an image. Figure 5b This demonstrates how to generate two auxiliary rays in a point cloud frame by selecting target points in two separate images. The following will combine... Figure 5a , 5b The present application provides a detailed description of its solutions. However, it should be noted that the description of the embodiments is merely for the purpose of facilitating public understanding of the solution and does not limit the scope of protection of this application.

[0056] like Figure 5a , 5b As shown, after receiving point cloud data from the LiDAR, a 3D scene and its corresponding 3D coordinate system can be constructed using WebGL technology, and the received LiDAR point cloud can be placed into the 3D scene for display. When the annotator performs the annotation task, the current point cloud frame and image frames captured by different cameras at the corresponding time can be displayed simultaneously on the operation interface. The LiDAR and cameras are all installed on the vehicle, and due to the different installation positions of the cameras, the relative position of the target vehicle in the images captured by different cameras will also be different.

[0057] exist Figure 5a The image shows images captured by two cameras at the same time and their corresponding point cloud frames. Camera 1 corresponds to image 1, and camera 2 corresponds to image 2. Cameras 1 and 2 are mounted at different positions on the vehicle. By selecting a point within the pixel area of ​​the target vehicle in image 1 as target point 1, and then generating a ray 1 emanating from camera 1 in the point cloud frame through back projection, ray 1 will pass through the area of ​​the target vehicle in the point cloud, thus providing guidance for annotators to locate the target vehicle's position within the point cloud. Furthermore, due to the characteristics of point clouds, the point cloud density within the target object area is often greater than the surrounding environment's point cloud density. Therefore, when annotators search for the target vehicle along ray 1, they can quickly determine the target vehicle's location and area by observing the point cloud density and complete the corresponding annotation operations.

[0058] exist Figure 5b The image shows images captured by two cameras at the same time and their corresponding point cloud frames. Camera 1 corresponds to image 1, and camera 2 corresponds to image 2. Cameras 1 and 2 are mounted at different positions on the vehicle. A point within the target vehicle's pixel area in image 1 is selected as target point 1, and a point within the same pixel area in image 2 is selected as target point 2. Then, through backprojection, rays corresponding to target point 1 and target point 2 are generated in the point cloud frames. Ray 1 originates from camera 1, and ray 2 originates from camera 2, both passing through the target vehicle's area in the point cloud. Ray 1 and ray 2 intersect at a point in the point cloud. This intersection point may be located within or near the target vehicle's area in the point cloud. Thus, the annotator can quickly determine the approximate location of the target vehicle in the point cloud using the intersection of the rays, and further determine the target vehicle's area in the point cloud based on the point cloud density differences between the target vehicle's location and the surrounding environment, completing the corresponding annotation operation.

[0059] With the above scheme, when annotating point clouds, annotators do not need to rely solely on visual searching for target objects. Even when the target object is difficult to identify, they can still use the image data at the corresponding time. Taking advantage of the high resolution and rich color and texture information of image data, they can simply select the pixels of the target object in the area of ​​the image and back-project them into the point cloud frame to generate one or more corresponding rays. The rays will pass through the area of ​​the target object in the point cloud or intersect to indicate the location or area of ​​the target object in the point cloud, thereby assisting annotators in quickly identifying and annotating the target object and avoiding omissions.

[0060] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0064] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A data processing method, comprising: Obtain annotation data for at least one frame of an image, wherein the annotation data includes the location data of the target point; Based on the annotation data of the at least one frame of image, auxiliary annotation information of the target point in the point cloud frame is generated, wherein the at least one frame of image and the point cloud frame have a corresponding relationship; Based on the auxiliary annotation information, the annotation data of the point cloud frame is obtained; Specifically, generating auxiliary annotation information for the target point in the point cloud frame based on the annotation data of the at least one image frame includes: The position data of the target point in the at least one frame of image is back-projected onto the point cloud frame to obtain at least one ray; The step of obtaining the annotation data of the point cloud frame based on the auxiliary annotation information includes: If a ray is generated in the point cloud frame by back projection, the point cloud density value corresponding to the point on the ray is calculated, a set of points whose point cloud density values ​​meet the preset value is selected, the point with the largest point cloud density value is selected in the set of points, and the coordinates of the point are used as the base point coordinates indicating the position of the target object in the point cloud frame. If two or more rays are generated in the point cloud frame by back projection, then the intersection point with the highest corresponding point cloud density value is selected from at least one intersection point of the at least two rays, and the coordinates of the intersection point in the point cloud frame are used as the base point coordinates. The method further includes: A 3D bounding box of the target object in the point cloud frame is generated based on the base point coordinates.

2. The data processing method according to claim 1, wherein the target point indicates the position of the target object, and the annotation data of the point cloud frame is obtained based on the auxiliary annotation information, further comprising: The point cloud frame and the at least one ray are displayed, the ray indicating the orientation of the target object in the point cloud frame; Receive the annotation data of the target object in the point cloud frame.

3. The data processing method according to claim 2, wherein receiving the annotation data of the target object in the point cloud frame specifically includes: Receive coordinate data of points along the contour of the target object; or Receive coordinate data of points along the outline of the target object and object attribute data.

4. The data processing method according to claim 2, wherein receiving the annotation data of the target object in the point cloud frame specifically includes: The coordinate data of a point of the target object within the point cloud frame are received as the base point coordinates. or The system receives the coordinate data of a point of the target object within the point cloud frame as the base point coordinates, and also receives object attribute data.

5. The data processing method according to claim 1, wherein generating the 3D bounding box of the target object in the point cloud frame based on the base point coordinates specifically includes: Using the coordinates of the base point as the coordinates of the geometric center point, a three-dimensional annotation box of a preset size is generated; or Using the base point coordinates as corner coordinates, a 3D annotation box of a preset size is generated.

6. A data processing apparatus, characterized in that, The method includes a processor and a memory, wherein the memory stores at least one machine-executable instruction, and the processor executes the at least one machine-executable instruction to perform the method as described in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Point cloud labeling method and device, computer equipment and storage medium

    CN109740487A

  • Target object marking method and device, electronic device and storage medium

    CN110197148A