Virtual frame labeling method and device based on point cloud data, vehicle and storage medium

Abstract

The application relates to a virtual frame labeling method and device based on point cloud data, a vehicle and a storage medium, which comprises the following steps: based on the attitude information and inertial measurement unit (IMU) information of to-be-labeled point cloud data, a preset global selection tracking algorithm and a preset minimum bounding box algorithm are used to generate a virtual frame corresponding to each frame of the to-be-labeled point cloud data, at least one target virtual frame meeting a preset labeling condition is screened out from the virtual frame corresponding to each frame of the to-be-labeled point cloud data, and each target virtual frame is labeled according to attribute information of the to-be-labeled point cloud data. Therefore, the problem that the target tracking ID, the category and the attribute of the labeling tool in the related art are difficult to keep consistent and depend on the labeling ability of a labeler, resulting in a long delivery cycle, low labeling quality and labeling efficiency and the like is solved, the labeling efficiency is improved, the dependence on the labeling ability of the labeler is reduced, and therefore, the scene objects in an automatic driving system are quickly and accurately labeled.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a virtual bounding box annotation method, device, vehicle, and storage medium based on point cloud data. Background Technology

[0002] Currently, most point cloud annotation methods on the market involve annotating one frame and then repeatedly annotating the next frame, or directly copying the annotation box from the previous frame to the next frame without modification, and then adjusting the box fit of the annotation box in each current frame individually. Although this ensures the consistency of tracking IDs, it consumes a lot of time and is very inefficient. Furthermore, the annotation quality of traditional annotation tools depends on the ability of the annotator and the review efforts of the annotation team, which consumes a huge amount of manpower and time to ensure annotation quality.

[0003] Some point cloud data annotation schemes have been proposed in related technologies, but they are not very universal. Most of them rely heavily on their own models or hardware, and they cannot fundamentally improve annotation efficiency, which urgently needs to be addressed. Summary of the Invention

[0004] This application provides a virtual bounding box annotation method, device, vehicle, and storage medium based on point cloud data to solve the problems in related technologies, such as the difficulty in maintaining consistency of target tracking object ID, category, and attributes, and the dependence on the annotation ability of the annotator, which leads to long delivery cycles, low annotation quality, and low annotation efficiency.

[0005] The first aspect of this application provides a virtual bounding box annotation method based on point cloud data, comprising the following steps: acquiring attitude information and inertial measurement unit (IMU) information of the point cloud data to be annotated; generating virtual bounding boxes corresponding to each frame of the point cloud data to be annotated using a preset global selection tracking algorithm and a preset minimum bounding box algorithm based on the attitude information and the IMU information; selecting at least one target virtual bounding box that meets preset annotation conditions from the virtual bounding boxes corresponding to each frame of the point cloud data to be annotated, and annotating each target virtual bounding box based on the attribute information of the point cloud data to be annotated.

[0006] Optionally, before acquiring the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled, the process includes: acquiring raw point cloud data packets collected by vehicle sensors; preprocessing the raw point cloud data packets using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising, filtering, and downsampling.

[0007] Optionally, after obtaining the initial point cloud data packet, the process includes: parsing, desensitizing, and extracting frames from the initial point cloud data packet to obtain initial point cloud data; and annotating at least one set of virtual bounding boxes at the front and back ends of the initial point cloud data according to the annotation instructions received from the annotator to obtain point cloud data to be annotated.

[0008] Optionally, the step of filtering at least one target virtual box that meets the preset annotation conditions from the virtual boxes corresponding to each frame of the point cloud data to be annotated includes: filtering virtual boxes whose number of point clouds is less than the preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, thereby obtaining at least one target virtual box that meets the preset annotation conditions.

[0009] Optionally, the attribute information of the point cloud data to be labeled includes at least one of size information, shape information, and orientation information.

[0010] Optionally, after annotating each target virtual box based on the attribute information of the point cloud data to be annotated, the process includes: storing the original point cloud data package and the annotated point cloud data package into a preset database.

[0011] A second aspect of this application provides a virtual bounding box annotation device based on point cloud data, comprising: an acquisition module for acquiring attitude information and inertial measurement unit (IMU) information of the point cloud data to be annotated; a generation module for generating virtual bounding boxes corresponding to each frame of the point cloud data to be annotated based on the attitude information and the IMU information, using a preset global selection tracking algorithm and a preset minimum bounding box algorithm; and an annotation module for selecting at least one target virtual bounding box that meets preset annotation conditions from the virtual bounding boxes corresponding to each frame of the point cloud data to be annotated, and annotating each target virtual bounding box based on the attribute information of the point cloud data to be annotated.

[0012] Optionally, before acquiring the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled, the acquisition module is further configured to: acquire the raw point cloud data packet collected by the vehicle sensors; preprocess the raw point cloud data packet using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising, filtering, and downsampling.

[0013] Optionally, after obtaining the initial point cloud data packet, the acquisition module is further configured to: parse, desensitize, and extract frames from the initial point cloud data packet to obtain initial point cloud data; and annotate at least one set of virtual bounding boxes at the front and back ends of the initial point cloud data according to the annotation instructions received from the annotator to obtain point cloud data to be annotated.

[0014] Optionally, the annotation module is further configured to: filter virtual boxes whose number of point clouds is less than a preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, so as to obtain at least one target virtual box that meets the preset annotation conditions.

[0015] Optionally, the attribute information of the point cloud data to be labeled includes at least one of size information, shape information, and orientation information.

[0016] Optionally, after annotating each target virtual box based on the attribute information of the point cloud data to be annotated, the annotation module is further configured to: store the original point cloud data package and the annotated point cloud data package in a preset database.

[0017] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the virtual bounding box annotation method based on point cloud data as described in the above embodiments.

[0018] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the virtual bounding box annotation method based on point cloud data as described in the above embodiments.

[0019] This application, based on the attitude information and IMU information of the point cloud data to be labeled, utilizes a preset global selection tracking algorithm and a preset minimum bounding box algorithm to generate virtual bounding boxes corresponding to each frame of the point cloud data to be labeled. It then selects at least one target virtual bounding box from the virtual bounding boxes corresponding to each frame of the point cloud data to be labeled, satisfying preset labeling conditions, and labels each target virtual bounding box according to the attribute information of the point cloud data to be labeled. This solves the problems in related technologies, such as the difficulty in maintaining consistency between the target tracking object ID, category, and attributes, and the reliance on the labeler's labeling ability, leading to long delivery cycles, low labeling quality, and low labeling efficiency. It improves labeling efficiency, reduces dependence on the labeler's ability, and thus enables rapid and accurate labeling of scene objects in autonomous driving systems, improving the perception and decision-making capabilities of autonomous driving systems.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart illustrating a virtual bounding box annotation method based on point cloud data according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a virtual bounding box annotation method based on point cloud data according to an embodiment of this application;

[0024] Figure 3 This is an example diagram of a virtual bounding box annotation device based on point cloud data according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a vehicle structure according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] The following description, with reference to the accompanying drawings, outlines a virtual bounding box annotation method, apparatus, vehicle, and storage medium based on point cloud data according to embodiments of this application. Addressing the difficulties in maintaining consistent target object IDs, categories, and attributes in annotation tools mentioned in the background section, and the reliance on the annotation skills of annotators, which leads to long delivery cycles and low annotation quality and efficiency, this application provides a virtual bounding box annotation method based on point cloud data. In this method, based on the attitude information and IMU information of the point cloud data to be annotated, a preset global selection tracking algorithm and a preset minimum bounding box algorithm are used to generate virtual bounding boxes corresponding to each frame of the point cloud data to be annotated. At least one target virtual bounding box that meets preset annotation conditions is selected from the virtual bounding boxes corresponding to each frame of the point cloud data to be annotated. Each target virtual bounding box is then annotated according to the attribute information of the point cloud data to be annotated. This solves the problems of maintaining consistent target object IDs, categories, and attributes in annotation tools in related technologies, and the reliance on the annotation skills of annotators, leading to long delivery cycles, low annotation quality, and low efficiency. It improves annotation efficiency, reduces reliance on the annotation skills of annotators, and thus enables rapid and accurate annotation of scene objects in autonomous driving systems, improving the perception and decision-making capabilities of autonomous driving systems.

[0028] Specifically, Figure 1 This is a flowchart illustrating a virtual bounding box annotation method based on point cloud data, provided in an embodiment of this application.

[0029] like Figure 1As shown, the virtual bounding box annotation method based on point cloud data includes the following steps:

[0030] In step S101, the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled are obtained.

[0031] The inertial measurement unit (IMU) information of the point cloud data to be labeled includes information such as velocity, acceleration, and direction of the point cloud data to be labeled.

[0032] Specifically, the vehicle's sensors, such as LiDAR, acquire point clouds and images of the surrounding environment, and collect attitude information and inertial measurement unit (IMU) information from the point clouds. The point cloud data is a collection of numerous three-dimensional points, each containing position information and other attributes.

[0033] Optionally, in some embodiments, before acquiring the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled, the method includes: acquiring raw point cloud data packets collected by vehicle sensors; preprocessing the raw point cloud data packets using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising, filtering, and downsampling.

[0034] It is understood that, in this application embodiment, the raw point cloud data packets collected from sensors such as LiDAR are stored on a server, and the raw point cloud data packets are preprocessed, such as denoising, filtering and downsampling, to obtain initial point cloud data packets, so as to improve the effect of subsequent processing.

[0035] It should be noted that the server can run on any suitable computer, whether it is a cloud server or a local server.

[0036] Furthermore, in some embodiments, after obtaining the initial point cloud data packet, the process includes: parsing, desensitizing, and extracting frames from the initial point cloud data packet to obtain initial point cloud data; and annotating at least one set of virtual bounding boxes at the front and back ends of the initial point cloud data according to the annotation instructions received from the annotator to obtain the point cloud data to be annotated.

[0037] Specifically, after obtaining the initial point cloud data packet, the server uses point cloud processing technology to parse, de-identify, and extract frames from the initial point cloud data packet to obtain the initial point cloud data, such as... Figure 2 As shown, it is deployed on the annotation platform. Annotators annotate virtual bounding boxes on at least one set of frames at the front and back ends of the initial point cloud data on the annotation platform to obtain the point cloud data to be annotated. The annotation platform is an annotation tool provided to annotators. For the annotation platform, annotators need to be given account permissions and assigned annotation tasks, and access the annotation tool link address to perform online annotation work.

[0038] In step S102, based on attitude information and IMU information, a virtual bounding box corresponding to each frame of the point cloud data to be labeled is generated using a preset global selection tracking algorithm and a preset minimum bounding box algorithm.

[0039] Among them, the preset global selection tracking algorithm and the preset minimum bounding box algorithm are global selection tracking algorithms and minimum bounding box algorithms in related technologies. The preset global selection tracking algorithm includes the pose algorithm and the Kalman filter algorithm.

[0040] In this application embodiment, different methods are used to complete the virtual bounding boxes for static and dynamic point cloud data to be labeled. For static point cloud data to be labeled, the Pose algorithm is used to complete the virtual bounding boxes for the remaining frames. For dynamic point cloud data to be labeled, the Kalman filter algorithm is used to predict and complete the virtual bounding boxes for the remaining frames, thereby enabling the estimation of the obstacle positions in the corresponding frames.

[0041] Specifically, the annotation platform receives the pose and IMU information of the point cloud data to be annotated stored on the server, and uses a preset global selection tracking algorithm and a preset minimum bounding box algorithm to calculate and generate virtual bounding boxes corresponding to each frame of the point cloud data to be annotated to surround the point cloud data or a specified region. The virtual bounding box is usually represented by a set of boundary points and has attributes such as position, size and orientation.

[0042] Therefore, by using a preset global selection tracking algorithm, after the annotator annotates virtual bounding boxes for at least one set of frames before and after the initial point cloud data packet, the virtual bounding boxes for the point cloud data to be annotated in other frames will be automatically calculated and displayed. The preset minimum bounding box algorithm can make the virtual bounding boxes closely fit the point cloud data to be annotated, thereby ensuring the consistency of obstacle tracking ID, category, and attributes. This solves the problem of tracking ID confusion and discontinuity caused by interruption in the joint annotation of tracking objects or the appearance of tracking objects after interruption.

[0043] In step S103, at least one target virtual box that meets the preset annotation conditions is selected from the virtual boxes corresponding to each frame of the point cloud data to be annotated, and each target virtual box is annotated based on the attribute information of the point cloud data to be annotated.

[0044] In some embodiments, the attribute information of the point cloud data to be labeled includes at least one of size information, shape information, and orientation information.

[0045] Optionally, in some embodiments, at least one target virtual box that meets the preset annotation conditions is selected from the virtual boxes corresponding to each frame of the point cloud data to be annotated, including: filtering virtual boxes whose number of point clouds is less than the preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, so as to obtain at least one target virtual box that meets the preset annotation conditions.

[0046] The preset number of point clouds can be a threshold set by the user, a threshold obtained through a limited number of experiments, or a threshold obtained through a limited number of computer simulations; no specific limitation is made here.

[0047] Understandably, if the number of point clouds within a virtual bounding box is less than the preset number of point clouds, or if the virtual bounding box does not fit the point cloud, or if the tracking ID and category attributes of the point cloud are inconsistent with the size or dimensions of the virtual bounding box, it means that the virtual bounding box in that part does not meet the preset annotation conditions. The annotation platform filters out the virtual bounding box in that part, thereby selecting at least one target virtual bounding box that meets the preset annotation conditions.

[0048] For example, if the number of point clouds in the virtual bounding box is less than 30, the virtual bounding box is considered not to meet the preset annotation conditions and is therefore filtered. If the tracking ID and category attributes of the point clouds in the virtual bounding box are inconsistent with the size of the virtual bounding box, the virtual bounding box is considered not to meet the preset annotation conditions and is therefore filtered.

[0049] Understandably, sensors such as LiDAR in vehicles extract features such as shape, color, and surface normals from the point cloud data to be labeled, in order to identify target objects or distinguish different categories of objects, thus facilitating the labeling of the point cloud data.

[0050] Furthermore, the annotation platform further analyzes and processes the point cloud data within at least one target virtual bounding box that meets the preset annotation conditions, extracting more detailed attribute information of the point cloud data to be annotated, such as the size, shape, and orientation information of the point cloud data to be annotated.

[0051] Furthermore, based on the attribute information of the point cloud data to be labeled obtained from the analysis, corresponding labels or annotations are added to each virtual box. The labels can be information such as the object category, position, and pose of the point cloud data to be labeled, thereby generating accurate labeling results for the point cloud data, which are provided to the autonomous driving system for subsequent path planning and decision-making. Labelers can also use the labeling platform to adjust the labeled virtual boxes into physical boxes, and adjust the position and size of the physical boxes, so that users can see obstacles around the vehicle more clearly.

[0052] Furthermore, in some embodiments, after annotating each target virtual box based on the attribute information of the point cloud data to be annotated, the process includes: storing the original point cloud data package and the annotated point cloud data package into a preset database.

[0053] Finally, the annotation platform stores the raw point cloud data packets collected by sensors such as LiDAR and the annotated point cloud data into a preset database, which can run on any suitable computer.

[0054] This application embodiment also uses a client device to display point cloud data and point cloud data information transmitted from the server, and provides a user interface for interaction with the user. The user can view the labeled point cloud data through the client device and make decisions on vehicle path planning based on the labeled point cloud data. The client device can be any device capable of running a browser, such as a desktop computer or laptop computer.

[0055] It should be noted that the technical conditions required to implement the embodiments of this application include: point cloud processing technology, WebGL technology, TypeScript technology, 3D algorithm technology, physics and 3D mathematics related technologies, JavaScript technology, HTML / CSS technology, etc.

[0056] In summary, the embodiments of this application can achieve the following beneficial effects:

[0057] (1) High degree of automation: It can complete the annotation task quickly and accurately without human intervention;

[0058] (2) Improve annotation efficiency: Compared with the traditional manual annotation method, it shortens the annotation cycle, increases annotation efficiency by 40%+, and reduces annotation costs;

[0059] (3) Improve system performance: Accurate object labeling results help autonomous driving systems make more accurate decisions and path planning.

[0060] The virtual bounding box annotation method based on point cloud data proposed in this application generates virtual bounding boxes for each frame of the point cloud data to be annotated based on the attitude information and IMU information of the point cloud data to be annotated, using a preset global selection tracking algorithm and a preset minimum bounding box algorithm. At least one target virtual bounding box that meets preset annotation conditions is selected from the virtual bounding boxes corresponding to each frame of the point cloud data to be annotated. Each target virtual bounding box is annotated according to the attribute information of the point cloud data to be annotated. This method solves the problems in related technologies, such as the difficulty in maintaining consistency between the target tracking object ID, category, and attributes, and the reliance on the annotation ability of the annotator, leading to long delivery cycles, low annotation quality, and low annotation efficiency. It improves annotation efficiency, reduces reliance on the annotation ability of the annotator, and provides a grid view to display the three views of all frame annotations for easy acceptance by the acceptance personnel. This enables fast and accurate annotation of scene objects in autonomous driving systems, improving the perception and decision-making capabilities of autonomous driving systems.

[0061] Next, referring to the accompanying drawings, a virtual box annotation device based on point cloud data proposed according to an embodiment of this application is described.

[0062] Figure 3 This is a block diagram of a virtual box annotation device based on point cloud data according to an embodiment of this application.

[0063] like Figure 3 As shown, the virtual bounding box annotation device 10 based on point cloud data includes: an acquisition module 100, a generation module 200, and an annotation module 300.

[0064] The acquisition module 100 is used to acquire the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled; the generation module 200 is used to generate virtual bounding boxes corresponding to each frame of the point cloud data to be labeled based on the attitude information and IMU information, using a preset global selection tracking algorithm and a preset minimum bounding box algorithm; the labeling module 300 is used to select at least one target virtual bounding box that meets the preset labeling conditions from the virtual bounding boxes corresponding to each frame of the point cloud data to be labeled, and to label each target virtual bounding box based on the attribute information of the point cloud data to be labeled.

[0065] Optionally, in some embodiments, before acquiring the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled, the acquisition module 100 is further configured to: acquire the raw point cloud data packet collected by the vehicle sensor; preprocess the raw point cloud data packet using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising, filtering and downsampling.

[0066] Optionally, in some embodiments, after obtaining the initial point cloud data packet, the acquisition module 100 is further configured to: parse, desensitize and extract frames from the initial point cloud data packet to obtain initial point cloud data; and annotate at least one set of virtual frames at the front and back ends of the initial point cloud data according to the annotation instructions received from the annotator to obtain the point cloud data to be annotated.

[0067] Optionally, in some embodiments, the annotation module 300 is further configured to: filter virtual boxes whose number of point clouds is less than a preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, so as to obtain at least one target virtual box that meets the preset annotation conditions.

[0068] Optionally, in some embodiments, the attribute information of the point cloud data to be labeled includes at least one of size information, shape information, and orientation information.

[0069] Optionally, in some embodiments, after annotating each target virtual box based on the attribute information of the point cloud data to be annotated, the annotation module 300 is further configured to: store the original point cloud data package and the annotated point cloud data package into a preset database.

[0070] It should be noted that the foregoing explanation of the virtual bounding box annotation method embodiment based on point cloud data also applies to the virtual bounding box annotation device based on point cloud data in this embodiment, and will not be repeated here.

[0071] The virtual bounding box annotation device based on point cloud data proposed in this application generates virtual bounding boxes for each frame of the point cloud data to be annotated based on the attitude information and IMU information of the point cloud data to be annotated, using a preset global selection tracking algorithm and a preset minimum bounding box algorithm. It then selects at least one target virtual bounding box that meets preset annotation conditions from the virtual bounding boxes corresponding to each frame of the point cloud data to be annotated, and annotates each target virtual bounding box according to the attribute information of the point cloud data to be annotated. This solves the problems in related technologies, such as the difficulty in maintaining consistency between the target tracking object ID, category, and attributes, and the reliance on the annotation ability of the annotator, leading to long delivery cycles, low annotation quality, and low annotation efficiency. It improves annotation efficiency, reduces dependence on the annotation ability of the annotator, and thus enables fast and accurate annotation of scene objects in autonomous driving systems, improving the perception and decision-making capabilities of autonomous driving systems.

[0072] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0073] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0074] When the processor 402 executes the program, it implements the virtual bounding box annotation method based on point cloud data provided in the above embodiments.

[0075] Furthermore, the vehicle also includes:

[0076] Communication interface 403 is used for communication between memory 401 and processor 402.

[0077] The memory 401 is used to store computer programs that can run on the processor 402.

[0078] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0079] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0080] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0081] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0082] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the virtual bounding box annotation method based on point cloud data as described above.

[0083] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0084] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0085] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0086] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0087] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0088] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0090] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A virtual bounding box annotation method based on point cloud data, characterized in that, Includes the following steps: Acquire raw point cloud data packets collected by vehicle sensors; preprocess the raw point cloud data packets using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising and downsampling; The initial point cloud data packet is parsed, desensitized, and frame extracted to obtain initial point cloud data; virtual bounding boxes are marked on at least one set of frames at the front and back ends of the initial point cloud data according to the annotation instructions received from the annotator to obtain the point cloud data to be annotated; Acquire attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled; Based on the pose information and the IMU information, a virtual bounding box corresponding to each frame of the point cloud data to be labeled is generated using a preset global selection tracking algorithm and a preset minimum bounding box algorithm. The preset global selection tracking algorithm includes the pose algorithm and the Kalman filter algorithm. From the virtual bounding boxes corresponding to each frame of the point cloud data to be labeled, at least one target virtual bounding box that meets the preset labeling conditions is selected, and each target virtual bounding box is labeled based on the attribute information of the point cloud data to be labeled. The step of filtering at least one target virtual box that meets the preset annotation conditions from the virtual boxes corresponding to each frame of the point cloud data to be annotated includes: filtering virtual boxes whose number of point clouds is less than the preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, thereby obtaining at least one target virtual box that meets the preset annotation conditions.

2. The method according to claim 1, characterized in that, The attribute information of the point cloud data to be labeled includes at least one of size information, shape information, and orientation information.

3. The method according to claim 1, characterized in that, After labeling each target virtual bounding box based on the attribute information of the point cloud data to be labeled, the process includes: The original point cloud data package and the labeled point cloud data are stored in a preset database.

4. A virtual bounding box annotation device based on point cloud data, characterized in that, include: The acquisition module is used to acquire the raw point cloud data packets collected by the vehicle's sensors; The original point cloud data packet is preprocessed using a preset processing strategy to obtain an initial point cloud data packet, wherein the preset processing strategy includes at least one of denoising and downsampling; the initial point cloud data packet is parsed, desensitized, and frame-extracted to obtain initial point cloud data; virtual bounding boxes are labeled on at least one set of frames at the front and back ends of the initial point cloud data according to the labeling instructions received from the labeler to obtain point cloud data to be labeled; the attitude information and inertial measurement unit (IMU) information of the point cloud data to be labeled are obtained. The generation module is used to generate virtual bounding boxes corresponding to each frame of the point cloud data to be labeled based on the pose information and the IMU information, using a preset global selection tracking algorithm and a preset minimum bounding box algorithm. The preset global selection tracking algorithm includes the pose algorithm and the Kalman filter algorithm. The annotation module is used to filter out at least one target virtual box that meets the preset annotation conditions from the virtual boxes corresponding to each frame of the point cloud data to be annotated, and to annotate each target virtual box based on the attribute information of the point cloud data to be annotated. The annotation module is further configured to: filter virtual boxes whose number of point clouds is less than the preset number of point clouds, and / or virtual boxes that do not meet the preset fitting conditions, and / or virtual boxes whose tracking ID and category attributes of the point clouds do not meet the preset consistency conditions with the attribute information of the virtual boxes, so as to obtain at least one target virtual box that meets the preset annotation conditions.

5. A vehicle, characterized in that, Including memory and processor; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the virtual bounding box annotation method based on point cloud data as described in any one of claims 1-3.

6. A computer-readable storage medium storing a computer program, characterized in that, When executed by the processor, the program implements the virtual bounding box annotation method based on point cloud data as described in any one of claims 1-3.

Images