An automated annotation method, system, electronic device, and storage medium

Abstract

This invention provides an automated annotation method comprising: identifying annotated target datasets in acquired laser point cloud data based on a 3D target detection algorithm; detecting whether each frame of the annotated target dataset is a keyframe; when the current frame is detected as a non-keyframe, extracting the position and attitude data of the LiDAR sensor from the previous keyframe and the current frame; calculating the translation vector and the angle between the view direction and the previous keyframe based on the LiDAR sensor position and attitude data; calculating each tracking target object in the current frame based on the translation vector and the angle between the view direction and the previous keyframe, and outputting the annotated dataset of the tracking targets. This invention calculates the target objects to be tracked in the current frame based on the translation vector and the angle between the view direction and the previous keyframe, thereby outputting the tracking targets, achieving automated annotation of 3D targets, reducing manual annotation workload, and improving production efficiency.

Description

Technical Field

[0001] This invention relates to the field of point cloud annotation technology for autonomous driving roads, and more specifically, to an automated annotation method, system, electronic device, and storage medium. Background Technology

[0002] There are two methods for creating labeled datasets: traditional manual annotation and automated assisted annotation. Compared to traditional manual annotation, automated assisted annotation offers advantages such as process automation, high efficiency, and low overall cost. Due to the diverse scenarios in the annotation field, automated assisted annotation requires the development of corresponding processing algorithms for different scenarios. Among these, 3D target continuous tracking annotation scenarios represent a type of dataset with significant training requirements for algorithms related to autonomous driving in the automotive industry.

[0003] Compared to ordinary 3D discrete annotation, manual annotation of 3D continuous tracking is much more labor-intensive and difficult. The identifier of an labeled target within a set of data must be unique, but the target's position and orientation can vary. During manual annotation, upon discovering a target, one must also consider whether the target existed in the previous frame and what its identifier was. Although copying can solve the identifier problem, subtle differences in the position and orientation of the same target in each frame still require significant manual adjustments, resulting in low annotation efficiency and compromised accuracy. Therefore, automating the continuous tracking and annotation of 3D targets is a pressing issue that needs to be addressed. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing an automated annotation method, system, electronic device, and storage medium to solve the problem of how to achieve automated continuous tracking and annotation of 3D targets.

[0005] A first aspect of the present invention provides an automated annotation method, comprising:

[0006] Based on 3D target detection algorithms, labeled target datasets are identified in the acquired laser point cloud data;

[0007] Detect whether each frame of the labeled target dataset is a keyframe, frame by frame.

[0008] When the current frame is detected to be a non-critical frame, extract the lidar sensor position and attitude data of the previous critical frame and the current frame;

[0009] Based on the position of the lidar sensor and the attitude data, calculate the translation vector and the angle between the view direction and the previous keyframe and the current frame;

[0010] Based on the angle between the translation vector and the view direction, calculate the tracking target object for each target in the current frame and output the labeled dataset of the tracking target.

[0011] Based on the above technical solution, the present invention can also be improved as follows.

[0012] Preferably, the step of detecting whether each frame of the labeled target dataset is a keyframe includes:

[0013] When determining that the current frame is the first frame of data based on the data sequence number of the labeled target dataset, the current frame is labeled as a key frame.

[0014] Preferably, the step of detecting whether each frame of the labeled target dataset is a keyframe frame further includes:

[0015] Based on the data sequence number of the labeled target dataset, when it is determined that the current frame is not the first frame data, the labeled target dataset of the previous frame of the current frame is obtained;

[0016] Based on the labeled target datasets of the current frame and the previous frame, calculate the difference in the total number of the two datasets, the translation of the labeled objects in the current frame relative to the previous frame, and the angle between the view direction.

[0017] Based on the difference, the position translation amount, and the angle between the viewpoint directions, the keyframe confidence value of the current frame is calculated. When the keyframe confidence value is greater than a preset confidence threshold, the current frame is determined to be a keyframe.

[0018] Preferably, after the step of determining the current frame as a key frame when the key frame confidence value is greater than a preset confidence threshold, the method includes:

[0019] Update the labeled target dataset of the current frame to the previous keyframe.

[0020] Preferably, the step of extracting the lidar sensor position and attitude data of the previous key frame and the current frame when the current frame is detected as a non-key frame includes:

[0021] When the current frame is detected as a non-key frame, radar sensor parameter data is extracted from the labeled target dataset of the previous key frame and the current frame, respectively. Based on the translation and rotation attributes of the radar sensor parameter data, the position and attitude data of the lidar sensor are extracted.

[0022] Preferably, the step of calculating the translation vector and the angle between the view direction and the previous keyframe and the current frame based on the position of the lidar sensor and the attitude data includes:

[0023] Based on the lidar sensor positions of the previous keyframe and the current frame, calculate the translation vector between the previous keyframe and the current frame;

[0024] Based on the LiDAR sensor attitude data of the previous keyframe and the current frame, the angle between the view direction of the previous keyframe and the current frame is calculated.

[0025] Preferably, the step of calculating each tracking target object in the current frame based on the angle between the translation vector and the view direction, and outputting the labeled dataset of the tracking targets, includes:

[0026] Based on the translation vector and the angle between the viewpoint direction, and taking the labeled target dataset of the previous keyframe as a reference, the labeled dataset of each tracked target in the current frame is calculated.

[0027] According to a second aspect of the present invention, an automated annotation system is provided, comprising:

[0028] The data preprocessing module is used to identify the labeled target dataset in the acquired laser point cloud data based on the 3D target detection algorithm;

[0029] The keyframe recognition module is used to detect frame by frame whether each frame of the labeled target dataset is a keyframe;

[0030] The parameter acquisition module is used to extract the position and attitude data of the lidar sensor in the previous key frame and the current frame when the current frame is detected to be a non-key frame.

[0031] The parameter calculation module is used to calculate the translation vector and the angle between the view direction and the previous keyframe and the current frame based on the position of the lidar sensor and the attitude data.

[0032] An automatic annotation module is used to calculate each tracking target object in the current frame based on the angle between the translation vector and the view direction, and output an annotation dataset of the tracking targets.

[0033] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the processor is configured to implement the steps of any of the automated annotation methods described in the first aspect when executing a computer management program stored in the memory.

[0034] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management class program is stored, wherein when executed by a processor, the computer management class program implements the steps of any of the automated annotation methods of the first aspect described above.

[0035] This invention provides an automated annotation method, system, electronic device, and storage medium. The method includes: identifying annotated target datasets in acquired laser point cloud data based on a 3D target detection algorithm; detecting whether each frame of the annotated target dataset is a keyframe; when the current frame is detected as a non-keyframe, extracting the position and attitude data of the LiDAR sensor from the previous keyframe and the current frame; calculating the translation vector and the angle between the previous keyframe and the current frame based on the LiDAR sensor position and attitude data; calculating each tracking target object in the current frame based on the translation vector and the angle between the viewpoint, and outputting the annotated dataset of the tracking targets. This invention achieves automatic annotation of 3D targets by determining the previous keyframe and the current frame in the annotated target dataset, and then calculating the target objects to be tracked in the current frame based on the translation vector and the angle between the previous keyframe and the current frame, thereby outputting the annotated dataset of the tracking targets. This significantly reduces the workload of manual annotation and improves the overall production efficiency in the annotation process. Attached Figure Description

[0036] Figure 1 A flowchart of an automated annotation method provided by the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the entire process of the automated annotation method provided by the present invention;

[0038] Figure 3 A schematic diagram of an automated annotation system provided by the present invention;

[0039] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0040] Figure 5 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0041] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0042] This invention provides an automated annotation method for continuous tracking of 3D targets based on the positioning and attitude of a lidar sensor. The annotated targets include the following two types: stationary targets and relatively stationary targets.

[0043] Furthermore, current annotations for continuous 3D target tracking are primarily used for training AI algorithms in the autonomous driving field. Since autonomous driving systems have high requirements for the accuracy and precision of AI algorithms, these algorithms also have high requirements for the accuracy and precision of the training annotation dataset. In addition, to accelerate AI algorithm training, the demand for annotation datasets is very large, and the timeframe is very short, necessitating improved annotation efficiency.

[0044] Therefore, in view of the problems existing in the existing annotation, the purpose of the present invention is to provide an automated annotation method for continuous 3D target tracking based on lidar sensor positioning and attitude. The method of the present invention can automatically identify the tracking target in the 3D point cloud scene and accurately calculate the tracking target's position and attitude information, providing an automated annotation method for continuous 3D target tracking annotation, reducing the workload of manual annotation, and thus completing the continuous 3D target tracking annotation efficiently and accurately.

[0045] Figure 1 A flowchart of an automated annotation method provided by the present invention is shown below. Figure 1 As shown, the method includes:

[0046] Step S100: Based on the 3D target detection algorithm, identify the labeled target dataset in the acquired laser point cloud data;

[0047] It should be noted that the executing entity of the method in this embodiment can be a computer terminal device with data processing, network communication, and program execution functions, such as a computer or a vehicle-mounted computer; it can also be a server device with the same or similar functions, or a cloud server with similar functions. This embodiment does not impose any restrictions on this. For ease of understanding, this embodiment and the following embodiments will be described using a vehicle-mounted computer as an example.

[0048] It is understood that the above-mentioned 3D target detection algorithm can be an algorithm with 3D target detection capability. It can be a point-based 3D target detection algorithm, a mesh-based 3D target detection algorithm, a point-Voxel-based 3D target detection algorithm, or a range-based 3D target detection algorithm. This embodiment does not limit it.

[0049] It should be understood that the aforementioned laser point cloud data can be point cloud data collected in real time by the vehicle's onboard LiDAR.

[0050] In the specific implementation, based on any 3D target detection algorithm, the labeled target dataset is extracted from the laser point cloud data in a loop.

[0051] Step S200: Detect frame by frame whether each frame of the labeled target dataset is a keyframe;

[0052] It is understandable that the aforementioned keyframe can refer to the frame that contains the key actions of the target during its motion changes. Therefore, a labeled target data may contain multiple keyframes and non-keyframes.

[0053] Furthermore, the step of detecting whether each frame of the labeled target dataset is a keyframe, as described above, includes:

[0054] Step S201: Based on the data sequence number of the labeled target dataset, when determining that the current frame is the first frame of data, label the current frame as a key frame.

[0055] In the specific implementation, after obtaining the labeled target dataset in the laser point cloud scene, it is determined whether the current frame is the first frame data according to the data sequence number. If it is the first frame data, it is directly determined as the key frame point cloud, and the identified labeled target dataset is used as the key frame tracking target dataset. If it is not the first frame data, the identified labeled target dataset in the laser point cloud scene is input into the key frame recognition module, and the output result is used to determine whether it is the key frame point cloud.

[0056] Furthermore, the step of detecting whether each frame of the labeled target dataset is a keyframe, as described above, also includes:

[0057] Step S202: Based on the data sequence number of the labeled target dataset, when it is determined that the current frame is not the first frame data, obtain the labeled target dataset of the previous frame of the current frame;

[0058] Step S203: Based on the labeled target datasets of the current frame and the previous frame, calculate the difference in the total number of the two datasets, the translation of the labeled objects in the current frame relative to the previous frame, and the angle between the view direction.

[0059] Step S204: Based on the difference, the position translation amount and the angle between the view direction, calculate the keyframe confidence value of the current frame. When the keyframe confidence value is greater than a preset confidence threshold, determine that the current frame is a keyframe.

[0060] Furthermore, after the step of determining the current frame as a key frame when the key frame confidence value is greater than a preset confidence threshold, the method includes:

[0061] Step S205: Update the labeled target dataset of the current frame to the previous keyframe.

[0062] In its specific implementation, the aforementioned keyframe recognition module is used to extract the target annotation dataset of the previous frame point cloud scene and the annotation object dataset of the current frame point cloud scene based on the input point cloud sequence number, calculate the difference between the total number of the two datasets, and simultaneously calculate the position translation amount and the angle between the view direction and the position translation amount of each annotation object in the current frame annotation dataset relative to the annotation object in the previous frame annotation dataset, using the previous frame annotation dataset as a reference. Then, based on the above difference and the values ​​of the position translation amount and the angle between the view direction, the confidence value of the keyframe is calculated. If the value is greater than 0, the current frame is determined to be a keyframe; otherwise, it is determined to be a non-keyframe.

[0063] Step S300: When the current frame is detected to be a non-key frame, extract the lidar sensor position and attitude data of the previous key frame and the current frame;

[0064] Furthermore, the steps described above for extracting the lidar sensor position and attitude data from the previous keyframe and the current frame include:

[0065] Step S301: When the current frame is detected to be a non-key frame, extract radar sensor parameter data from the labeled target datasets of the previous key frame and the current frame respectively, and extract the position and attitude data of the lidar sensor based on the translation and rotation attributes of the radar sensor parameter data.

[0066] Step S400: Based on the position of the lidar sensor and the attitude data, calculate the translation vector and the angle between the view direction and the previous keyframe and the current frame;

[0067] Furthermore, the steps described above for calculating the translation vector and the angle between the view direction and the previous keyframe and the current frame include:

[0068] Step S401: Based on the lidar sensor positions of the previous keyframe and the current frame, calculate the translation vector between the previous keyframe and the current frame;

[0069] Step S402: Based on the LiDAR sensor attitude data of the previous keyframe and the current frame, calculate the angle between the view direction of the previous keyframe and the current frame.

[0070] Step S500: Based on the angle between the translation vector and the view direction, calculate each tracking target object in the current frame and output the labeled dataset of the tracking target.

[0071] Furthermore, the steps for generating the labeled dataset for the output tracking target include:

[0072] Step S501: Based on the angle between the translation vector and the view direction, and taking the labeled target dataset of the previous keyframe as a reference, calculate the labeled dataset of each tracked target in the current frame.

[0073] Understandably, given the deficiencies in the background technology, this invention proposes an automated annotation method. The method includes: identifying the labeled target dataset in the acquired laser point cloud data based on a 3D target detection algorithm; detecting whether each frame of the labeled target dataset is a keyframe; when the current frame is detected as a non-keyframe, extracting the laser radar sensor position and attitude data of the previous keyframe and the current frame; calculating the translation vector and the angle between the previous keyframe and the current frame based on the laser radar sensor position and attitude data; calculating each tracking target object in the current frame based on the translation vector and the angle between the viewpoint direction, and outputting the labeled dataset of the tracking targets. This invention achieves automated annotation for continuous 3D target tracking and can be applied to automated annotation of continuous 3D target tracking in laser point clouds. In calculating the tracking target in subsequent frames, the automated annotation method calculates the difference in point cloud scene changes between the keyframe and subsequent frames, and then calculates the tracking target position and attitude data in the subsequent frames. This allows for precise target location, significantly improving the accuracy and precision of the labeled dataset compared to the previous simple copying of tracking target data. When continuously tracking and annotating 3D targets, automated annotation production can be achieved, requiring only a small amount of manual quality inspection, greatly reducing the workload of manual annotation and improving overall production efficiency.

[0074] In one possible embodiment, this embodiment also includes a complete automated annotation method, see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram illustrating the entire process of the automated annotation method provided by the present invention; in Figure 2 In this process, the vehicle can acquire point cloud data of the target area through LiDAR. Then, the onboard computer extracts the point cloud data and radar sensor data, and determines whether the current frame is a key frame based on the data sequence number. If it is a key frame, the tracking target of the key frame is identified and the key frame tracking target data is updated. If it is not a key frame, the radar position and attitude data are extracted, and the difference between the point cloud scene and the previous key frame is calculated. Based on the key frame tracking target dataset, the non-key frame tracking target dataset is calculated, and the tracking target annotation dataset is updated until the tracking of the last frame of point cloud data is completed, and the tracking target annotation dataset is output.

[0075] In this embodiment, automated annotation for continuous 3D target tracking is realized. It can be applied to the automated annotation of continuous 3D target tracking using laser point clouds. During the continuous 3D target tracking annotation, automated annotation production can be achieved, requiring only a small amount of manual quality inspection, which greatly reduces the workload of manual annotation and improves the overall production efficiency.

[0076] In one possible embodiment, the present invention provides an automated annotation system, which includes: a data preprocessing module, a lidar parameter calculation module, a keyframe recognition module, and an automated annotation module.

[0077] The data preprocessing module mainly reads point cloud data and sensor parameter data from the dataset in a loop, extracts key frame point cloud and radar sensor position and attitude data, and finally stores them into key frame point cloud data files and radar sensor data files respectively.

[0078] Specifically, the operation steps of the aforementioned data preprocessing module include:

[0079] S11: Extract laser point cloud data from the dataset.

[0080] S12: Identify labeled target datasets within a laser point cloud scene using a 3D target detection algorithm.

[0081] S13: Mark whether the laser point cloud is a keyframe. Determine whether it is the first frame data based on the data sequence number. If it is the first frame data, directly determine it as a keyframe point cloud, and at the same time use the identified labeled target dataset as the keyframe tracking target dataset. If it is not the first frame data, input the labeled target dataset in the laser point cloud scene identified in S12 into the keyframe recognition module, and determine whether it is a keyframe point cloud based on the output result.

[0082] S14: Extract radar sensor parameter data from the dataset.

[0083] S15: Extract radar sensor location data. Parse the radar sensor parameters extracted in S11 and extract the radar sensor location data using the `translation` attribute.

[0084] S16: Extract radar sensor position data. Parse the radar sensor parameters extracted in S11, and extract the radar sensor attitude data through the rotation attribute.

[0085] The lidar parameter calculation module mainly extracts the position and attitude data of the lidar sensor from the lidar sensor data file, and calculates the position translation and attitude view difference between the key frame point cloud scene and the subsequent frame point cloud scene.

[0086] Specifically, the operation steps of the aforementioned lidar parameter calculation module include:

[0087] S21: Extract the position coordinates and attitude data of the first frame of input LiDAR sensor.

[0088] S22: Extract the second frame of input lidar sensor position coordinates and attitude data.

[0089] S23: Calculate the translation vector between the two LiDAR sensors using the position coordinates of the two LiDAR sensors.

[0090] S24: Calculate the angle between the two LiDAR sensor attitude data frames using the two frames of LiDAR sensor attitude data.

[0091] Keyframe recognition module: It mainly extracts keyframe point cloud data from keyframe point cloud data files, identifies the tracking target through 3D target detection algorithm, and generates a tracking target annotation dataset.

[0092] Specifically, the operation steps of the aforementioned keyframe recognition module include:

[0093] S31: Extract the target annotation dataset of the previous frame point cloud scene based on the input point cloud sequence number.

[0094] S32: Extract the current frame point cloud scene annotation object dataset from the input.

[0095] S33: Calculate the difference between the totals of the two datasets obtained from S31 and S32.

[0096] S34: Using the S31 labeled dataset as a reference, calculate the translation of the position of each labeled object in the S32 labeled dataset relative to the labeled objects in the S31 labeled dataset.

[0097] S35: Using the S31 labeled dataset as a reference, calculate the angle between each labeled object in the S32 labeled dataset and the viewpoint direction of the labeled objects in the S31 labeled dataset.

[0098] S36: Based on the values ​​obtained from S33, S34, and S35, calculate the keyframe confidence value. If the value is greater than 0, it is determined to be a keyframe; otherwise, it is determined to be a non-keyframe.

[0099] The automated annotation module primarily calculates the tracking target annotation dataset on the point cloud of subsequent frames based on the tracking target annotation dataset identified in the keyframes, the position translation and attitude view difference between the keyframe point cloud scene and the subsequent frame point cloud scene.

[0100] Specifically, the operation steps of the aforementioned automated annotation module include:

[0101] S41: Extract the keyframe point cloud scene tracking target annotation dataset.

[0102] S42: Extract keyframe LiDAR sensor position coordinates and attitude data.

[0103] S43: Extract the input LiDAR sensor position coordinates and attitude data.

[0104] S44: Extract the translation vector and the angle between the viewing directions of the two LiDAR sensors based on the methods in S21 to S24.

[0105] S45: Based on the keyframe point cloud scene tracking target annotation dataset obtained in S41, and based on the translation vector and view direction angle obtained in S44, calculate each tracking target object in the current frame point cloud scene, and finally output the tracking target annotation dataset.

[0106] In this embodiment, automated annotation production can be achieved during continuous tracking and annotation of 3D targets, requiring only a small amount of manual quality inspection, greatly reducing the workload of manual annotation and improving overall production efficiency.

[0107] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an automated annotation system provided in an embodiment of the present invention, such as... Figure 3 As shown, an automated annotation system includes a data preprocessing module 100, a keyframe recognition module 200, a parameter acquisition module 300, a parameter calculation module 400, and an automatic annotation module 500, wherein:

[0108] The data preprocessing module 100 is used to identify the labeled target dataset in the acquired laser point cloud data based on a 3D target detection algorithm; the keyframe recognition module 200 is used to detect whether each frame of the labeled target dataset is a keyframe; the parameter acquisition module 300 is used to extract the position and attitude data of the laser radar sensor in the previous keyframe and the current frame when the current frame is detected as a non-keyframe; the parameter calculation module 400 is used to calculate the translation vector and the angle between the previous keyframe and the current frame based on the position and attitude data of the laser radar sensor; and the automatic labeling module 500 is used to calculate each tracking target object in the current frame based on the translation vector and the angle between the viewpoint direction, and output the labeled dataset of the tracking targets.

[0109] It is understood that the automated annotation system provided by the present invention corresponds to the automated annotation method provided in the foregoing embodiments. The relevant technical features of the automated annotation system can be referred to the relevant technical features of the automated annotation method, and will not be repeated here.

[0110] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 4As shown, this embodiment of the invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, it performs the following steps:

[0111] Based on a 3D target detection algorithm, the system identifies the labeled target dataset in the acquired laser point cloud data; it then checks each frame of the labeled target dataset to determine if it is a keyframe; when the current frame is detected as a non-keyframe, it extracts the position and attitude data of the LiDAR sensor from the previous keyframe and the current frame; based on the LiDAR sensor position and attitude data, it calculates the translation vector and the angle between the previous keyframe and the current frame; based on the translation vector and the angle between the viewpoint, it calculates each tracking target object in the current frame and outputs the labeled dataset of the tracking target.

[0112] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 5 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, it performs the following steps:

[0113] Based on a 3D target detection algorithm, the system identifies the labeled target dataset in the acquired laser point cloud data; it then checks each frame of the labeled target dataset to determine if it is a keyframe; when the current frame is detected as a non-keyframe, it extracts the position and attitude data of the LiDAR sensor from the previous keyframe and the current frame; based on the LiDAR sensor position and attitude data, it calculates the translation vector and the angle between the previous keyframe and the current frame; based on the translation vector and the angle between the viewpoint, it calculates each tracking target object in the current frame and outputs the labeled dataset of the tracking target.

[0114] This invention provides an automated annotation method, system, and storage medium. The method includes: identifying annotated target datasets in acquired laser point cloud data based on a 3D target detection algorithm; detecting whether each frame of the annotated target dataset is a keyframe; when the current frame is detected as a non-keyframe, extracting the position and attitude data of the LiDAR sensor from the previous keyframe and the current frame; calculating the translation vector and the angle between the previous keyframe and the current frame based on the LiDAR sensor position and attitude data; calculating each tracking target object in the current frame based on the translation vector and the angle between the viewpoint, and outputting the annotated dataset of the tracking targets. This invention achieves automated 3D target annotation by determining the previous keyframe and the current frame in the annotated target dataset, and then calculating the target objects in the current frame based on the translation vector and the angle between the previous keyframe and the current frame, thereby outputting the annotated dataset of the tracking targets. This significantly reduces the workload of manual annotation and improves the overall production efficiency in the annotation process.

[0115] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0121] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An automated annotation method, characterized in that, The method includes: Based on 3D target detection algorithms, labeled target datasets are identified in the acquired laser point cloud data; Detect whether each frame of the labeled target dataset is a keyframe, frame by frame. When the current frame is detected to be a non-critical frame, extract the lidar sensor position and attitude data of the previous critical frame and the current frame; Based on the position of the lidar sensor and the attitude data, calculate the translation vector and the angle between the view direction and the previous keyframe and the current frame; Based on the angle between the translation vector and the view direction, calculate each tracking target object in the current frame and output the labeled dataset of the tracking targets; The step of detecting whether each frame of the labeled target dataset is a keyframe frame includes: Based on the data sequence number of the labeled target dataset, when determining that the current frame is the first frame of data, the current frame is labeled as a keyframe; The step of detecting whether each frame of the labeled target dataset is a keyframe frame frame by frame further includes: Based on the data sequence number of the labeled target dataset, when it is determined that the current frame is not the first frame data, the labeled target dataset of the previous frame of the current frame is obtained; Based on the labeled target datasets of the current frame and the previous frame, calculate the difference in the total number of the two datasets, the translation of the labeled objects in the current frame relative to the previous frame, and the angle between the view direction. Based on the difference, the position translation amount, and the angle between the viewpoint directions, the keyframe confidence value of the current frame is calculated. When the keyframe confidence value is greater than a preset confidence threshold, the current frame is determined to be a keyframe. After the step of determining the current frame as a key frame when the key frame confidence value is greater than a preset confidence threshold, the following steps are included: Update the keyframe annotation target dataset with the annotation target dataset of the current frame.

2. The automated annotation method according to claim 1, characterized in that, The step of extracting the lidar sensor position and attitude data of the previous key frame and the current frame when the current frame is detected as a non-key frame includes: When the current frame is detected as a non-critical frame, radar sensor parameter data is extracted from the labeled target dataset of the previous critical frame and the current frame, respectively. Based on the translation and rotation attributes of the radar sensor parameter data, the position and attitude data of the lidar sensor are extracted.

3. The automated annotation method according to claim 1, characterized in that, The step of calculating the translation vector and the angle between the view direction and the previous keyframe and the current frame based on the position of the lidar sensor and the attitude data includes: Based on the lidar sensor positions of the previous keyframe and the current frame, calculate the translation vector between the previous keyframe and the current frame; Based on the LiDAR sensor attitude data of the previous keyframe and the current frame, the angle between the view direction of the previous keyframe and the current frame is calculated.

4. The automated annotation method according to claim 1, characterized in that, The step of calculating each tracking target object in the current frame based on the angle between the translation vector and the view direction, and outputting the labeled dataset of the tracking targets, includes: Based on the translation vector and the angle between the viewpoint direction, and taking the labeled target dataset of the previous keyframe as a reference, the labeled dataset of each tracked target in the current frame is calculated.

5. An automated annotation system, characterized in that, include: The data preprocessing module is used to identify the labeled target dataset in the acquired laser point cloud data based on the 3D target detection algorithm; The keyframe recognition module is used to detect frame by frame whether each frame of the labeled target dataset is a keyframe; The parameter acquisition module is used to extract the position and attitude data of the lidar sensor in the previous key frame and the current frame when the current frame is detected to be a non-key frame. The parameter calculation module is used to calculate the translation vector and the angle between the view direction and the previous keyframe and the current frame based on the position of the lidar sensor and the attitude data. An automatic annotation module is used to calculate each tracking target object in the current frame based on the angle between the translation vector and the view direction, and output an annotation dataset of the tracking targets; The step of detecting whether each frame of the labeled target dataset is a keyframe includes: Based on the data sequence number of the labeled target dataset, when determining that the current frame is the first frame of data, the current frame is labeled as a keyframe; The step of detecting whether each frame of the labeled target dataset is a keyframe also includes: Based on the data sequence number of the labeled target dataset, when it is determined that the current frame is not the first frame data, the labeled target dataset of the previous frame of the current frame is obtained; Based on the labeled target datasets of the current frame and the previous frame, calculate the difference in the total number of the two datasets, the translation of the labeled objects in the current frame relative to the previous frame, and the angle between the view direction. Based on the difference, the position translation amount, and the angle between the viewpoint directions, the keyframe confidence value of the current frame is calculated. When the keyframe confidence value is greater than a preset confidence threshold, the current frame is determined to be a keyframe. After determining that the current frame is a key frame when the key frame confidence value is greater than a preset confidence threshold, the following steps are included: Update the keyframe annotation target dataset with the annotation target dataset of the current frame.

6. An electronic device, characterized in that, It includes a memory and a processor, wherein the processor is used to implement the steps of the automated annotation method as described in any one of claims 1-4 when executing a computer management program stored in the memory.

7. A computer-readable storage medium, characterized in that, It stores a computer management program, which, when executed by a processor, implements the steps of the automated annotation method as described in any one of claims 1-4.

Images