A point cloud image automatic labeling method, device, equipment and medium

Abstract

This application relates to the field of point cloud data annotation, and discloses a method, apparatus, device, and medium for automatic annotation of point cloud images. The method includes: acquiring first annotation information and first center coordinates of a first target in the nth point cloud image of a point cloud data file, and acquiring second annotation information of a second target in the qth point cloud image; based on the first and second annotation information, calculating third annotation information and third center coordinates of a third target in the (n+j)th point cloud image using linear interpolation; determining whether a first distance between the first and third center coordinates is less than a preset distance; if the first distance is less than the preset distance, determining that the third target matches the first target, and updating the first annotation information of the first target with the third annotation information. This application can efficiently annotate targets on point cloud images to achieve target detection and tracking.

Description

Technical Field

[0001] This application relates to the field of point cloud data annotation technology, and in particular to a method, apparatus, device and medium for automatic annotation of point cloud images. Background Technology

[0002] The number of targets in point cloud images acquired by radar for obstacle avoidance ranges from a few dozen to hundreds, while the number of targets required for point cloud detection and tracking for deep learning is also thousands for each model training session.

[0003] In the process of implementing the embodiments of this application, the inventors of this application discovered that when detecting and tracking targets in point cloud images, assuming that one thousand point cloud images are needed, and each point cloud image has 50 targets, then there are more than 50,000 targets. Each target requires at least 30 seconds to label, and manual labeling of targets would take 416 hours, which is time-consuming and laborious. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, and medium for automatic annotation of point cloud images, which can efficiently annotate targets on point cloud images to achieve target detection and tracking.

[0005] To solve the above-mentioned technical problems, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide an automatic annotation method for point cloud images, including:

[0007] Obtain the first annotation information and first center coordinates of the first target in the nth point cloud image of the point cloud data file, and obtain the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q=n+m,m>0;

[0008] Based on the first and second annotation information, the third annotation information and third center coordinates of the third target in the (n+j)th point cloud image are calculated using linear interpolation, where 0 <j<m;

[0009] Determine whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance;

[0010] If the first distance is less than the preset distance, then the third target is determined to match the first target, and the first annotation information of the first target is updated with the third annotation information.

[0011] In some embodiments, after determining whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance, the method further includes:

[0012] If the first distance is greater than the preset distance, then the detection range of the third center coordinates is expanded to obtain the fourth target;

[0013] Determine whether the second distance between the fourth center coordinates of the fourth target and the first center coordinates is less than the preset distance;

[0014] If the second distance is less than the preset distance, then the fourth target is determined to match the first target, the fourth annotation information of the fourth target is calculated, and the first annotation information of the first target is updated with the fourth annotation information.

[0015] In some embodiments, expanding the detection range where the third center coordinates are located to obtain the fourth target includes:

[0016] Enlarge the length and width of the third center coordinates;

[0017] A multi-level feature extraction structure is used to detect the target by the expanded third center coordinates, thus obtaining the fourth target.

[0018] In some embodiments, obtaining the first annotation information of the first target in the nth point cloud image of the point cloud data file includes:

[0019] Read the point cloud data file to obtain the nth point cloud image;

[0020] Obtain the range of the target region that needs to be labeled in the nth point cloud image;

[0021] Within the target region, a multi-level feature extraction structure is used to perform target detection on the nth point cloud image to obtain a first target and generate the first annotation information of the first target.

[0022] In some embodiments, after generating the first annotation information of the first target, the method further includes:

[0023] If the nth point cloud image already has a first annotation file corresponding to the first target, then the first annotation information is written into the first annotation file;

[0024] If the nth point cloud image does not have a first annotation file corresponding to the first target, then a second annotation file corresponding to the first target is created, and the first annotation information is written into the second annotation file.

[0025] In some embodiments, the first annotation information includes target category, radar coordinates, target length, width and height, heading angle of radar direction and target ID.

[0026] In some embodiments, the point cloud data file is in pcd or bin format.

[0027] Secondly, embodiments of this application also provide an automatic point cloud image annotation device, the device comprising:

[0028] The acquisition module is used to acquire the first annotation information and first center coordinates of the first target in the nth point cloud image of the point cloud data file, and to acquire the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q=n+m,m>0;

[0029] The calculation module is used to calculate the third annotation information and the third center coordinates of the third target in the (n+j)th point cloud image based on the first annotation information and the second annotation information, using linear interpolation, where 0 <j<m;

[0030] The judgment module is used to determine whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance;

[0031] An update module is configured to determine that the third target matches the first target if the first distance is less than the preset distance, and update the first annotation information of the first target with the third annotation information.

[0032] Thirdly, this application also provides an automatic point cloud image annotation device, the automatic point cloud image annotation device comprising:

[0033] At least one processor, and

[0034] A memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0035] Fourthly, this application also provides a non-volatile computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, which, when executed by a point cloud image automatic annotation device, cause the point cloud image automatic annotation device to perform the method described in any of the first aspects.

[0036] Advantages of the embodiments of the present application: Different from the prior art, the method, device, equipment and medium for automatically annotating point cloud images provided in the embodiments of the present application obtain the first annotation information and the first center coordinate of the first target in the nth point cloud image in the point cloud data file when tracking a certain target in the point cloud data file, and obtain the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q = n + m, m > 0; then, based on the first annotation information and the second annotation information, the third annotation information and the third center coordinate of the third target in the (n + j)th point cloud image are calculated by linear interpolation, where 0 < j < m; next, in order to determine whether the found third target is the same as the first target, it is determined whether the first distance between the first center coordinate and the third center coordinate is less than a preset distance; if the first distance is less than the preset distance, it is determined that the third target matches the first target, and the first annotation information of the first target is updated with the third annotation information, realizing automatic annotation from the nth point cloud image to the qth point cloud image, thereby ensuring the accuracy of target tracking. Through the automatic annotation from the nth point cloud image to the qth point cloud image, the annotation efficiency is effectively improved and the annotation time is saved. Description of the Drawings

[0037] One or more embodiments are illustrated by way of example in the accompanying drawings, which do not constitute a limitation to the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the figures do not constitute a scale limitation.

[0038] Figure 1 It is a schematic flowchart of an embodiment of the method for automatically annotating point cloud images of the present application;

[0039] Figure 2 It is a schematic diagram of the target area range of the method for automatically annotating point cloud images of the present application;

[0040] Figure 3 It is a schematic diagram of the target detection of pointnet++ of the method for automatically annotating point cloud images of the present application;

[0041] Figure 4 ]>It is a schematic diagram of the target ID of the method for automatically annotating point cloud images of the present application;

[0042] Figure 5 It is a schematic diagram of the target category of the method for automatically annotating point cloud images of the present application;

[0043] Figure 6 It is a schematic diagram of the target of the method for automatically annotating point cloud images of the present application;

[0044] Figure 7This is a schematic diagram of the structure of one embodiment of the point cloud image automatic annotation device of this application;

[0045] Figure 8 This is a schematic diagram of the hardware structure of the controller in one embodiment of the point cloud image automatic annotation device of this application. Detailed Implementation

[0046] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0049] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0050] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0051] The point cloud image automatic annotation method and apparatus provided in this application can be applied to point cloud image automatic annotation devices. It is understood that the point cloud image automatic annotation device includes a controller and a LiDAR. The controller acts as the main control center, and the LiDAR is used to acquire point cloud data. The controller acquires point cloud data from the LiDAR and can efficiently annotate targets on multiple consecutive frames of point cloud images to achieve target detection and tracking.

[0052] Understandably, point cloud image automatic annotation equipment can be an in-vehicle device installed in a car, which can detect and track targets, allowing the driver to be aware of the surrounding environment.

[0053] Please see Figure 1 The above is a flowchart illustrating an embodiment of the point cloud image automatic annotation method applied in this application. The method can be executed by a controller in a point cloud image automatic annotation device, and the method includes steps S101-S104.

[0054] S101: Obtain the first annotation information and first center coordinates of the first target in the nth point cloud image of the point cloud data file, and obtain the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q = n + m, m > 0;

[0055] When obtaining the first annotation information of the first target in the nth point cloud image of the point cloud data file, the method may include:

[0056] Read the point cloud data file to obtain the nth point cloud image;

[0057] Obtain the range of the target region that needs to be labeled in the nth point cloud image;

[0058] Within the target region, a multi-level feature extraction structure is used to perform target detection on the nth point cloud image to obtain a first target and generate the first annotation information of the first target.

[0059] Specifically, the point cloud data file is read, and then all point cloud images in the point cloud data file are obtained, including mq consecutive point cloud images. When labeling the target, the nth point cloud image is labeled first, where n can be 1.

[0060] Furthermore, such as Figure 2 As shown, to obtain the target region range to be labeled in the nth point cloud image, the area where the object to be labeled is located can be initially selected in the graphical user interface (GUI) to obtain the target region range. Within the target region range, a multi-level feature extraction structure (PointNet++) is used to perform target detection on the nth point cloud image, as follows. Figure 3 As shown, the multi-level feature extraction structure (PointNet++) provides multiple pieces of information related to the first target during detection, thus obtaining the first annotation information. This first annotation information includes the target category, radar coordinates, target length, width, and height, radar heading angle, and target ID. The target category is designated as "class," meaning the category of the first target can be identified. Figure 3As shown, the category of the first target is identified as bicycle; the radar coordinates (x, y, z) refer to the coordinates of the top-left corner closest to the radar in a spatial coordinate system with the radar as the origin, including the three values ​​in the coordinates (x, y, z); the length, width, and height of the first target refer to the length, width, and height of the first target object, denoted as (length, width, height); the radar heading angle (z-axis) refers to the heading angle of the radar direction as a square; the target ID is the unique feature of the first target, and one ID corresponds to only one object target, such as... Figure 4 As shown, the target ID is set to 6; Figure 5 As shown, the target category is determined to be bicycle; Figure 6 As shown, after determining the nine values, the annotation result of the first annotation information of the first target is obtained. Then, based on the above nine values ​​[class, id, z-axis, (x, y, z), (length, width, height)], the first annotation information of the first target is generated.

[0061] It is understandable that, since target recognition is performed on point cloud data through a GUI interface and a multi-level feature extraction structure (PointNet++), the format of the point cloud data file can be either pcd or bin format, and there is no limitation on this.

[0062] Existing point cloud annotation tools typically only annotate point cloud data in PCD format. However, in this embodiment, the point cloud data file can be in PCD or BIN format, eliminating the need to convert BIN to PCD, thus simplifying the recognition process.

[0063] Similar to the method of obtaining the first annotation information of the first target, the target is extracted from the q-th point cloud image through a multi-level feature extraction structure (pointnet++) to obtain the second annotation information of the second target.

[0064] In some implementations, after generating the first annotation information for the first target, the method may further include:

[0065] If the nth point cloud image already has a first annotation file corresponding to the first target, then the first annotation information is written into the first annotation file;

[0066] If the nth point cloud image does not have a first annotation file corresponding to the first target, then a second annotation file corresponding to the first target is created, and the first annotation information is written into the second annotation file.

[0067] Specifically, after obtaining the first annotation information of the first target in the nth point cloud image, it is determined whether the first annotation file corresponding to the first target in the nth point cloud image exists. If it exists, the first annotation information can be directly written into the first annotation file; if it does not exist, a second annotation file corresponding to the first target is newly created, and then the first annotation information is written into the newly created second annotation file.

[0068] It can be understood that after obtaining the second annotation information of the second target, the second annotation information is also written into the annotation file corresponding to the second target.

[0069] After writing the annotation information into the annotation file, when subsequently locating the next point cloud image, the annotation information can be obtained from the annotation file.

[0070] S102: Based on the first annotation information and the second annotation information, use linear interpolation to calculate the third center coordinates of the third target in the (n + j)th point cloud image, where 0 < j < m.

[0071] Specifically, after obtaining the first annotation information and the second annotation information, use linear interpolation to calculate the third center coordinates of the third target in the (n + j)th point cloud image.

[0072] For example, the first annotation information of the first target A in the nth point cloud image is [(x1, y1, z1), (l1, w1, h1), (class, z - axis1, id)]. Then, the first center coordinates of the first target are (x1, y1, z1). The second annotation information of the second target (such as the same class as the first target, both are bicycles) in the qth point cloud image is [(x2, y2, z2), (l2, w2, h2), (class, z - axis2, id)]. Calculate the third center coordinates of the third target of target A in the (n + j)th point cloud image through linear interpolation, using the following formulas 1 - 3:

[0073] X C , C , ,

[0076] = x1 + j * (x2 - x1) / m + 0.5 * (l1 + l2) Formula 1

[0074] Y C = y1 + j * (y2 - y1) / m + 0.5 * (w1 + w2) Formula 2

[0075] Z C = z1 + j * (z2 - z1) / m + 0.5 * (h1 + h2) Formula 3

[0076] Where, m = q - n, 0 < j < m; Formula 1 calculates the horizontal axis X of the third center coordinates of the third target A in the (n + j)th point cloud CFormula 2 calculates the y-axis coordinate of the third center of the third target in the (n+j)th point cloud. C Formula 3 calculates the vertical axis Z of the third center coordinate of the third target in the (n+j)th point cloud. C Thus, the coordinates (X) of the third center of the (n+j)th point cloud are obtained. C Y C Z C ).

[0077] S103: Determine whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance.

[0078] S104: If the first distance is less than the preset distance, then the third target is determined to match the first target, and the first annotation information of the first target is updated with the third annotation information.

[0079] To determine whether the third target matches the first target and thus track the first target, the third center coordinates, obtained based on the relative motion between the vehicle and the first target during vehicle movement, are only an approximate range. Therefore, after obtaining the third center coordinates, it is determined whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance. If the first distance is less than the preset distance, the third target is determined to match the first target; that is, the third target found within the range of the third center coordinates is the first target. At this time, the first annotation information of the first target is updated with the third annotation information, thereby achieving the tracking of the first target from the nth point cloud image to the (n+j)th point cloud image. Based on the relative motion between the vehicle and the first target during vehicle movement, it is assumed that the relative displacement of the first target from the nth point cloud image to the (n+j)th point cloud image does not exceed 20cm; therefore, the preset distance can be set to 20cm.

[0080] S105: If the first distance is greater than the preset distance, then expand the detection range of the third center coordinates to obtain the fourth target.

[0081] Since the first target A is usually not moving in a uniform linear motion, for example, when the vehicle equipped with the onboard device is driving, there is a first target A in q point cloud images of the point cloud data file obtained by the LiDAR. The bicycle and the vehicle are traveling in the same direction. Obviously, there will be a relative speed between the bicycle and the vehicle. Moreover, the relative speed is not constant and will not move in a straight line as the road conditions change. Therefore, linear interpolation may not be able to satisfy the motion trajectory of the first target A. That is, the third center coordinates of the (n+j)th point cloud are only an approximate range. At this time, the first distance between the first center coordinates and the third center coordinates will be greater than the preset distance, indicating that the first target is no longer within the range of the third center coordinates in the (n+j)th point cloud. The third target may be another bicycle. Therefore, in order to find the first target A, the detection range of the third center coordinates is expanded to obtain the fourth target.

[0082] In some implementations, the detection range of the third center coordinates is expanded to obtain a fourth target, including:

[0083] Enlarge the length and width of the third center coordinates;

[0084] A multi-level feature extraction structure is used to detect the target by the expanded third center coordinates, thus obtaining the fourth target.

[0085] Specifically, by expanding the labeled area of ​​the third center coordinates, the length and width of the third center coordinates can be increased, for example, by doubling, while keeping the height unchanged. Then, a multi-level feature extraction structure (PointNet++) is used for target extraction to obtain the fourth target. According to the following formulas 4-9, the detection range of the third center coordinates can be obtained:

[0086] X s =X C -0.5*(l1+l2) Formula 4;

[0087] X l =X C +0.5*(l1+l2) Formula 5;

[0088] Y l =Y C +0.5*(w1+w2) Formula 6;

[0089] Y s =Y C -0.5*(w1+w2) Formula 7;

[0090] Z s =Z C -0.75*(h1+h2) Formula 8;

[0091] Z l =Z C +0.75*(h1+h2) Formula 9.

[0092] Once a range is obtained, the center coordinates of this range are calculated.

[0093] Among them, after obtaining the detection range of the third center coordinates, the expanded range of the third center coordinates (X) is obtained. S Y S Z S ).

[0094] Then, within the expanded range of the third center coordinates (X) S Y S Z S Within the scope of the target detection, a multi-level feature extraction structure, pointNet++, is used to obtain the fourth target.

[0095] After obtaining the fourth target, its fourth center coordinates are obtained. Then, following the method used for the third target, it is determined whether the second distance between the fourth center coordinates and the first center coordinates is less than a preset distance. If the second distance is less than the preset distance, the fourth target is determined to match the first target. The fourth annotation information of the fourth target is calculated, and the first annotation information of the first target is updated with the fourth annotation information. That is, if the fourth target matches the first target, it means that the fourth target is the first target that has been tracked. At this time, it is only necessary to update the first annotation information of the first target with the fourth annotation information to complete the tracking of the first target. If the fourth target does not match the first target, it means that the expanded range of the third center coordinates still does not contain the first target. Similarly, the range is expanded again, and target detection and target matching are continued until a target matching the first target is found, thus completing the tracking of the first target.

[0096] In an embodiment of the present application, the first annotation information and the first center coordinates of the first target in the nth point cloud image in the point cloud data file are obtained, and the second annotation information of the second target in the qth point cloud image is obtained; the point cloud data file includes q consecutive frames of point cloud images, and q = n + m, where m > 0; then, based on the first annotation information and the second annotation information, the third annotation information and the third center coordinates of the third target in the (n + j)th point cloud image are calculated by linear interpolation, where 0 < j < m; next, in order to determine whether the found third target is consistent with the first target, it is judged whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance; if the first distance is less than the preset distance, it is determined that the third target matches the first target, and the first annotation information of the first target is updated with the third annotation information, so as to achieve automatic annotation from the nth point cloud image to the qth point cloud image, thereby ensuring the accuracy of target tracking. By automatically annotating from the nth point cloud image to the qth point cloud image, the annotation efficiency is effectively improved and the annotation time is saved.

[0097] An embodiment of the present application further provides a point cloud image automatic annotation device. Please refer to Figure 7 , which shows the structure of a point cloud image automatic annotation device provided by an embodiment of the present application. The point cloud image automatic annotation device 700 includes:

[0098] An acquisition module 701, configured to acquire the first annotation information and the first center coordinates of the first target in the nth point cloud image in the point cloud data file, and acquire the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive frames of point cloud images, and q = n + m, where m > 0;

[0099] A calculation module 702, configured to calculate the third annotation information and the third center coordinates of the third target in the (n + j)th point cloud image by linear interpolation based on the first annotation information and the second annotation information, where 0 < j < m;

[0100] A judgment module 703, configured to judge whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance;

[0101] An update module 704, configured to determine that the third target matches the first target and update the first annotation information of the first target with the third annotation information if the first distance is less than the preset distance.

[0102] In an embodiment of the present application, the first annotation information and the first center coordinates of the first target in the n-th point cloud image in the point cloud data file are obtained, and the second annotation information of the second target in the q-th point cloud image is obtained; the point cloud data file includes q consecutive frames of point cloud images, and q = n + m, where m > 0; then, based on the first annotation information and the second annotation information, the third annotation information and the third center coordinates of the third target in the (n + j)-th point cloud image are calculated by linear interpolation, where 0 < j < m; then, in order to determine whether the found third target is consistent with the first target, it is determined whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance; if the first distance is less than the preset distance, it is determined that the third target matches the first target, and the first annotation information of the first target is updated with the third annotation information, so as to achieve automatic annotation from the n-th point cloud image to the q-th point cloud image, thereby ensuring the accuracy of target tracking. By automatically annotating from the n-th point cloud image to the q-th point cloud image, the annotation efficiency is effectively improved and the annotation time is saved.

[0103] In some embodiments, the point cloud image automatic annotation device 700 further includes an expansion module 705, which is used for:

[0104] If the first distance is greater than the preset distance, the detection range where the third center coordinates are located is expanded to obtain a fourth target;

[0105] The determination module 703 is further used to determine whether the second distance between the fourth center coordinates of the fourth target and the first center coordinates is less than the preset distance;

[0106] The update module 704 is further used to, if the second distance is less than the preset distance, determine that the fourth target matches the first target, calculate the fourth annotation information of the fourth target, and update the first annotation information of the first target with the fourth annotation information.

[0107] In some embodiments, the expansion module 705 is further used for:

[0108] Expand the length and width of the third center coordinates;

[0109] Perform target detection on the expanded third center coordinates by using a multi-level feature extraction structure to obtain a fourth target.

[0110] In some embodiments, the acquisition module 701 is further used for:

[0111] Read the point cloud data file to obtain the n-th point cloud image;

[0112] Obtain the range of the target area to be annotated in the n-th point cloud image;

[0113] Within the target region, a multi-level feature extraction structure is used to perform target detection on the nth point cloud image to obtain a first target and generate the first annotation information of the first target.

[0114] In some embodiments, the point cloud image automatic annotation device 700 further includes a writing module 706, used for:

[0115] If the nth point cloud image already has a first annotation file corresponding to the first target, then the first annotation information is written into the first annotation file;

[0116] If the nth point cloud image does not have a first annotation file corresponding to the first target, then a second annotation file corresponding to the first target is created, and the first annotation information is written into the second annotation file.

[0117] It should be noted that the above-described apparatus can execute the method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the apparatus embodiments can be found in the method provided in the embodiments of this application.

[0118] Figure 8 A schematic diagram of the hardware structure of the controller in one embodiment of a point cloud image automatic annotation device, as shown below. Figure 8 As shown, the controller includes:

[0119] One or more processors 111 and memory 112. Figure 8 The example uses a processor 111 and a memory 112.

[0120] Processor 111 and memory 112 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0121] Memory 112, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the point cloud image automatic annotation method in the embodiments of this application (e.g., attached...). Figure 7 The acquisition module 701, calculation module 702, judgment module 703, update module 704, expansion module 705, and writing module 706 are shown. The processor 111 executes various functional applications and data processing of the controller by running non-volatile software programs, instructions, and modules stored in the memory 112, thereby realizing the point cloud image automatic annotation method of the above method embodiment.

[0122] The memory 112 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the personnel access detection device. Furthermore, the memory 112 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 112 may optionally include memory remotely located relative to the processor 111, and these remote memories can be connected to the point cloud image automatic annotation device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The one or more modules are stored in the memory 112. When executed by the one or more processors 111, they perform the point cloud image automatic annotation method in any of the above method embodiments, for example, the method described above. Figure 1 Method steps S101 to S104; implementation Figure 7 The functions of modules 701-706 in the document.

[0124] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0125] This application provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 8 One of the processors 111 can enable the one or more processors to execute the point cloud image automatic annotation method in any of the above method embodiments, for example, to execute the method described above. Figure 1 Method steps S101 to S104; implementation Figure 7 The functions of modules 701-706 in the document.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic annotation of point cloud images, characterized in that, The method includes: Obtain the first annotation information and first center coordinates of the first target in the nth point cloud image of the point cloud data file, and obtain the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q=n+m, m>0; Based on the first and second annotation information, the third center coordinates of the third target in the (n+j)th point cloud image are calculated using linear interpolation, where 0 <j<m; Determine whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance; If the first distance is less than the preset distance, then the third target is determined to match the first target, and the first annotation information of the first target is updated with the third annotation information of the third target.

2. The method according to claim 1, characterized in that, After determining whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance, the method further includes: If the first distance is greater than the preset distance, then the detection range of the third center coordinates is expanded to obtain the fourth target; Determine whether the second distance between the fourth center coordinates of the fourth target and the first center coordinates is less than the preset distance; If the second distance is less than the preset distance, then the fourth target is determined to match the first target, the fourth annotation information of the fourth target is calculated, and the first annotation information of the first target is updated with the fourth annotation information.

3. The method according to claim 2, characterized in that, Expanding the detection range of the third center coordinates to obtain the fourth target includes: Enlarge the length and width of the third center coordinates; A multi-level feature extraction structure is used to detect the target by the expanded third center coordinates, thus obtaining the fourth target.

4. The method according to claim 1, characterized in that, The step of obtaining the first annotation information of the first target in the nth point cloud image of the point cloud data file includes: Read the point cloud data file to obtain the nth point cloud image; Obtain the range of the target region that needs to be labeled in the nth point cloud image; Within the target region, a multi-level feature extraction structure is used to perform target detection on the nth point cloud image to obtain a first target and generate the first annotation information of the first target.

5. The method according to claim 4, characterized in that, After generating the first annotation information of the first target, the method further includes: If the nth point cloud image already has a first annotation file corresponding to the first target, then the first annotation information is written into the first annotation file; If the nth point cloud image does not have a first annotation file corresponding to the first target, then a second annotation file corresponding to the first target is created, and the first annotation information is written into the second annotation file.

6. The method according to any one of claims 1-5, characterized in that, The first annotation information includes target category, radar coordinates, target length, width and height, heading angle of radar direction and target ID.

7. The method according to any one of claims 1-5, characterized in that, The point cloud data file is in pcd or bin format.

8. An automatic annotation device for point cloud images, characterized in that, The device includes: The acquisition module is used to acquire the first annotation information and first center coordinates of the first target in the nth point cloud image of the point cloud data file, and to acquire the second annotation information of the second target in the qth point cloud image; the point cloud data file includes q consecutive point cloud images, and q=n+m, m>0; The calculation module is used to calculate the third center coordinates of the third target in the (n+j)th point cloud image based on the first annotation information and the second annotation information using linear interpolation, where 0 <j<m; The judgment module is used to determine whether the first distance between the first center coordinates and the third center coordinates is less than a preset distance; An update module is configured to determine that the third target matches the first target if the first distance is less than the preset distance, and update the first annotation information of the first target with the third annotation information of the third target.

9. An automatic point cloud image annotation device, characterized in that, The point cloud image automatic annotation device includes: At least one processor, and A memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the point cloud image automatic annotation device, cause the point cloud image automatic annotation device to perform the method as described in any one of claims 1-7.

Images