Point cloud data labeling method and device, electronic equipment and storage medium

Abstract

The invention provides a point cloud data labeling method and device, electronic equipment and a storage medium. The invention relates to the technical field of artificial intelligence, in particular to the field of point cloud data processing, and can be applied to application scenes such as automatic driving testing, computer vision, intelligent detection and an automatic system. The specific implementation scheme is as follows: acquiring a target identifier and a target output mode; a first labeling result of a target object corresponding to the target identifier is determined, the first labeling result is presented in a target output mode, and the first labeling result is obtained by labeling the multi-frame point cloud data through a visual large model; in response to the received adjustment operation for the first labeling result, updating the first labeling result based on the adjustment operation; and obtaining a second labeling result of the target object based on the updated first labeling result. According to the scheme, the data processing efficiency and the labeling consistency can be improved, the labor cost is reduced, and meanwhile the labeling accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of point cloud data processing, and can be used in application scenarios such as autonomous driving testing, computer vision, intelligent detection and automation systems, and specifically relates to a point cloud data annotation method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence and large model algorithms, although traditional manual labeling methods are still applicable to small-scale data processing, they are inefficient and prone to labeling inconsistencies when processing complex three-dimensional (3D) point cloud data and large-scale data sets. Summary of the invention

[0003] The present invention provides a point cloud data annotation method, device, electronic device and storage medium.

[0004] According to one aspect of the present disclosure, a point cloud data annotation method is provided, the method comprising:

[0005] Get the target ID and target output mode;

[0006] Determine a first annotation result of the target object corresponding to the target identifier, and present the first annotation result in a target output manner, where the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model;

[0007] In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation;

[0008] A second annotation result of the target object is obtained based on the updated first annotation result.

[0009] According to another aspect of the present disclosure, a point cloud data annotation device is provided, the device comprising:

[0010] An acquisition module is used to obtain a target identifier and a target output mode;

[0011] A determination module, used to determine a first annotation result of a target object corresponding to the target identifier, and present the first annotation result in a target output manner, wherein the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model;

[0012] an updating module, configured to, in response to receiving an adjustment operation on the first annotation result, update the first annotation result based on the adjustment operation;

[0013] A generating module is used to obtain a second annotation result of the target object based on the updated first annotation result.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0020] The solution disclosed in the present invention can improve the efficiency of data processing and the consistency of annotation, reduce labor costs, and improve the accuracy of annotation.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a moving trajectory of a target object according to an embodiment of the present disclosure;

[0025] Figure 3 This is a schematic diagram of a point cloud data annotation editing interface according to an embodiment of the present disclosure. Figure 1 ;

[0026] Figure 4 This is a schematic diagram of a point cloud data annotation editing interface according to an embodiment of the present disclosure. Figure 2 ;

[0027] Figure 5 This is a schematic diagram of a point cloud data annotation editing interface according to an embodiment of the present disclosure. Figure 3 ;

[0028] Figure 6 This is a schematic diagram of a point cloud data annotation editing interface according to an embodiment of the present disclosure. Figure 4 ;

[0029] Figure 7 is a structural schematic diagram of a point cloud data annotation device according to an embodiment of the present disclosure;

[0030] Figure 8 is a schematic diagram of a scene of a point cloud data annotation method according to an embodiment of the present disclosure;

[0031] Fig. 9 It is a schematic diagram of the structure of an electronic device used to implement the point cloud data annotation method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0034] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0035] Before introducing the technical solutions of the embodiments of the present disclosure, the technical terms that may be used in the present disclosure are further explained:

[0036] Point cloud data: refers to a collection of surface points of an object in three-dimensional space acquired by a scanner (such as lidar, depth camera, etc.). These points are usually represented by three-dimensional coordinates (X, Y, Z), reflecting the geometry of the object or scene.

[0037] Multi-frame point cloud data: refers to a series of temporally continuous point cloud data frames obtained through multiple consecutive scans or acquisitions. Each frame of point cloud data represents the three-dimensional spatial information at a certain moment, and these frames together describe the motion trajectory, morphological changes, and other characteristics of a dynamic scene or object.

[0038] Visual large model: refers to a large-scale deep learning model used to process visual tasks after being trained with a large amount of data, especially in the field of computer vision. It usually has strong recognition, classification, detection and segmentation capabilities, and can handle complex tasks such as images, videos or point cloud data.

[0039] Annotation: In data processing, annotation refers to adding additional information or labels to data for subsequent analysis and application. In point cloud data annotation, it usually includes identifying and annotating the location, size, category and other information of the target object.

[0040] In the related art, 3D point cloud data annotation has the following problems: it relies on manual annotation of 3D point cloud data frame by frame, which is inefficient and prone to inconsistent annotation when processing large-scale data sets. Therefore, how to improve the efficiency of data processing and the consistency of annotation, reduce labor costs, and ensure the accuracy of annotation is an urgent problem to be solved in this technical field.

[0041] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, the present disclosure proposes a point cloud data annotation method, which significantly improves the efficiency of data processing and the consistency of annotation, reduces labor costs, and improves the accuracy of annotation.

[0042] The present disclosure provides a method for annotating point cloud data. Figure 1 It is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure, and the point cloud data annotation method can be applied to a point cloud data annotation device. The point cloud data annotation device is located in an electronic device. The electronic device includes but is not limited to fixed devices and / or mobile devices. For example, fixed devices include but are not limited to servers, and the server can be a cloud server or an ordinary server. For example, mobile devices include but are not limited to: mobile phones, tablet computers, and vehicle-mounted terminals. In some possible implementations, the point cloud data annotation method can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the point cloud data annotation method includes:

[0043] S101, obtaining a target identifier and a target output mode;

[0044] S102, determining a first annotation result of the target object corresponding to the target identifier, and presenting the first annotation result in a target output manner, where the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model;

[0045] S103, in response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation;

[0046] S104: Obtain a second annotation result of the target object based on the updated first annotation result.

[0047] In the disclosed embodiment, the target identifier refers to an identifier used to uniquely identify and distinguish a specific target object in the point cloud data or image. The target identifier can be selected from an element list, which includes all identifiers in the multi-frame point cloud data. After the visual large model completes the annotation of the multi-frame point cloud data, an element list can be generated.

[0048] In the embodiments of the present disclosure, the target object refers to a specific entity or object that needs to be annotated, tracked or identified in point cloud data or image data. In scenes such as autonomous driving and robot vision, the target object is usually a vehicle, pedestrian, obstacle, etc. The annotation of the target object is to determine its position, trajectory, size or other attributes in multiple frames of data.

[0049] In the disclosed embodiments, the target output mode refers to the specific mode or form used by the system to display or output the annotation results. It determines how the annotation results are presented to the user or passed to other systems to facilitate understanding, analysis and processing of the annotation data. It includes but is not limited to the following output modes: two-dimensional image display, three-dimensional model display, augmented reality (AR) display, animation display, video stream display, etc.

[0050] In the embodiment of the present disclosure, the visual large model is a pre-trained large model with multi-frame point cloud annotation capability. It should be noted that the present disclosure does not specifically limit the training method and model structure of the visual large model.

[0051] In some embodiments, obtaining the target identification and the target output method includes: automatically extracting the target identification from multiple frames of point cloud data, or manually selecting the identification of the target object through user input.

[0052] In some embodiments, obtaining the target output mode may include: automatically analyzing and selecting the most appropriate output mode according to factors such as the requirements of the current task, the type of target object, and user preferences. For example, in an autonomous driving scenario, a real-time video stream or AR display mode may be preferred for road obstacles; while for a three-dimensional reconstruction task, a high-precision three-dimensional model output may be preferred. For different output modes, their performance in practical applications (such as real-time performance, accuracy, ease of use, etc.) is evaluated, and the mode with the best performance is selected.

[0053] In some embodiments, obtaining the target output mode may include: providing an intuitive and easy-to-use user interface, allowing the user to manually select the output mode by clicking, sliding, etc. A plurality of commonly used output mode options are preset in the interface, and the user can select one of them as needed. In addition to the preset options, a custom setting function may also be provided, allowing the user to adjust the parameters and settings of the output mode according to their own needs.

[0054] In some implementations, the adjustment operation includes, but is not limited to: adjusting the size of the annotation box of the target object, adjusting the position of the annotation box, adjusting the direction of the annotation box, adjusting the properties of the target object, etc.

[0055] In some embodiments, in response to receiving an adjustment operation for a first annotation result, updating the first annotation result based on the adjustment operation includes: adjusting the first annotation result through interface interaction (such as clicking, dragging, zooming, etc.). These adjustments may include modifying the bounding box of the target object, adjusting the annotation category, adding or deleting annotation points, etc. The system updates the first annotation result in real time in response to the adjustment operation. This includes recalculating the parameters of the target object (such as position, size, direction, etc.) and updating the annotation information on the display interface. In some embodiments, a second annotation result of the target object is obtained based on the updated first annotation result, and the completeness and accuracy of the second annotation result are better than the first annotation result. This result is more accurate and meets actual needs, and can be used for subsequent data processing, analysis, and application.

[0056] For example, suppose that point cloud data annotation of a vehicle is required in an autonomous driving scenario. First, the system uses a large visual model to process multiple frames of point cloud data and automatically identifies the vehicle target object. Then, the system presents the annotation results to the user in the form of a two-dimensional image display. The user finds that the annotation bounding box of a certain vehicle is slightly different from the actual vehicle, so he adjusts the annotation result by dragging the bounding box. The system responds to the user's adjustment operation and updates the annotation result in real time. Finally, the accurate annotation result confirmed by the user is obtained and used for subsequent data processing and analysis.

[0057] In some embodiments, obtaining a second annotation result of the target object based on the updated first annotation result includes: checking the updated first annotation result, such as the completeness and accuracy of the annotation and whether it complies with a preset annotation specification or standard. Any possible errors or inconsistencies are identified by checking, which may be caused by human error, data quality issues, or limitations of the annotation tool.

[0058] In some embodiments, obtaining a second annotation result for the target object based on the updated first annotation result includes: comparing the annotation results before and after the update, analyzing the differences, such as which annotations are modified and the reasons for the modifications, which helps to understand the changes in the data and the new information that may be introduced. Verification and confirmation are performed for the modifications to the annotations. This may involve rechecking the relevant data to confirm the correctness of the modifications. If possible, cross-validation or multi-person review can also be used to ensure the consistency and accuracy of the annotation results. Based on the updated first annotation result and the inspection, analysis and verification in the above steps, a second annotation result for the target object is generated to ensure that the format, structure and content of the annotation results meet the requirements of the final use, and to obtain a more accurate and consistent second annotation result for the target object, thereby providing high-quality data support for subsequent model training and application.

[0059] The technical solution of the disclosed embodiment can quickly process a large amount of point cloud data and generate preliminary annotation results through the visual big model, and then continue to adjust and confirm the annotation results according to actual needs on this basis, which can significantly improve the accuracy of point cloud data annotation. In the later adjustment process, the annotation results are corrected and optimized according to actual needs. The processing method combining automatic annotation and adjustment can greatly improve the annotation efficiency.

[0060] In some embodiments, presenting the first annotation result in a target output manner may include:

[0061] The moving trajectory of the target object in multi-frame point cloud data is presented in the form of target output.

[0062] In the disclosed embodiment, the movement trajectory refers to displaying the trajectory of the target object in the annotated data of multiple frames to show the movement route of the object in 3D space. It connects the position information of the target object in time to help users understand the movement pattern of the object.

[0063] In some embodiments, the point cloud information of the target object in each frame of data is extracted, including parameters such as coordinate position, velocity, acceleration, etc. The position of the target object in multiple frames of point cloud data is continuously tracked using a trajectory tracking algorithm (such as Kalman filtering, particle filtering, etc.). The moving trajectory of the target object is constructed by associating and matching the target point clouds in adjacent frames. The target object is continuously tracked using a trajectory tracking algorithm to improve the accuracy and reliability of the trajectory. In some embodiments, the tracked moving trajectory is presented in a graphical manner, which can be implemented using three-dimensional visualization software or a programming library. In the visualization interface, the position, velocity vector, acceleration and other information of the target object at different times can be displayed, and different trajectories or states can be distinguished by color, line type, etc.

[0064] Figure 2 A schematic diagram of the moving trajectory of the target object is shown in FIG. Figure 2 As shown in the interface, each rectangular box represents an object, and multiple rectangles connected by the same line represent the same target object. The line represents the movement trajectory of the target object over a period of time. By checking the smoothness of the movement trajectory, it is possible to quickly identify whether there are misidentified objects in the current annotation.

[0065] In some embodiments, the movement trajectory of the target object in the multi-frame point cloud data is presented in a target output manner, including: the system continuously displays the position changes of the target object in each frame in the form of frame playback, so that the user can clearly see the movement trajectory of the target object over time.

[0066] In some embodiments, the moving trajectory of the target object in the multi-frame point cloud data is presented in a target output manner, including: the system displays the multi-frame point cloud data simultaneously in a multi-frame view, so that the user can compare the position and posture changes of the target object at different time points. Here, the multi-frame view refers to the simultaneous display of the annotation results of multiple frames of point cloud data or image data, usually displaying multiple frames of data in the form of parallel views, which is convenient for comparing and analyzing the annotation conditions at multiple times. This method is suitable for comparing the changes of objects or the continuity of annotations in a short time.

[0067] For example, suppose a car is driving on the road in an autonomous driving scenario. The laser radar installed on the vehicle continuously scans the surrounding environment to obtain multi-frame point cloud data. The trajectory tracking algorithm is used to continuously track the position of the car in the multi-frame point cloud data, and the tracked movement trajectory is presented in the form of a three-dimensional graph. In the visualization interface, you can see the trajectory line of the car along the road, as well as information such as the velocity vector and acceleration at different times.

[0068] In this way, by outputting the moving trajectory of the target object in multi-frame point cloud data, it is helpful to analyze the motion laws and characteristics of the target object and realize the tracking and visualization of the moving trajectory of the target object.

[0069] In some embodiments, the target output method presents the first annotation result, and may further include:

[0070] When the moving trajectory jumps, outputting first prompt information, the first prompt information is used to prompt that there is a problem frame in the multi-frame point cloud data, and the problem frame is a frame where the jump occurs;

[0071] In response to receiving a positioning operation for a problem frame, the problem frame is displayed and the position of the target object in the problem frame is indicated.

[0072] In the disclosed embodiment, a jump refers to a situation where the motion trajectory of the target object suddenly changes discontinuously or abnormally. The jump may be caused by labeling errors, sensor failure, or object occlusion.

[0073] In the disclosed embodiment, a problem frame refers to a frame in which anomalies or jumps occur during the annotation process, for example, a frame in which the size, position or direction of the annotation box is inconsistent with adjacent frames, or a frame in which the motion trajectory of an object is suddenly interrupted.

[0074] In some embodiments, outputting the first prompt information includes: by selecting a target identifier in the list, the system highlights the effect of the corresponding target according to the track function. When the system detects a track jump, it automatically locates the jump position and prompts the user to make corresponding adjustments.

[0075] For example, suppose that in the environmental perception system of an autonomous vehicle, the on-board laser radar continuously collects multi-frame point cloud data of the vehicle in front. The system generates annotation results through the visual large model and tracks the motion trajectory of the vehicle in front. When the vehicle suddenly accelerates to overtake, due to the loss of sensor data, the system detects that the vehicle trajectory in a certain frame has jumped, and the annotation box is inconsistent with the previous and next frames. At this time, the system outputs the first prompt information, prompting the user that there may be a problem with the frame. The user clicks on the target logo in the prompt, and the system highlights the vehicle's motion trajectory and automatically locates the frame where the jump occurs. The system displays the vehicle position in the problem frame, and the user can adjust the position, size or direction of the annotation box to correct the trajectory error and ensure the continuity and accuracy of the annotation results.

[0076] In this way, when the vehicle trajectory jumps, the system automatically identifies and prompts the existence of problematic frames in the multi-frame point cloud data. By highlighting the trajectory and accurately locating the jump position, timely adjustments can be made to ensure the continuity and accuracy of the annotation, reduce the workload of manual inspection, and improve the annotation efficiency and reliability of data processing.

[0077] In some embodiments, after the system prompts the problem frame, the point cloud data annotation method may further include:

[0078] In response to detecting an adjustment operation on the current annotation box of the target object in the problem frame, a position or angle of the current annotation box is adjusted based on the adjustment operation.

[0079] In some embodiments, adjusting the position or angle of the current annotation box based on the adjustment operation includes: receiving the adjustment operation input by the user through the system interface, such as dragging the annotation box to change its position in the point cloud data, or rotating the annotation box to more accurately align it with the actual contour of the target object. The system responds to the user's adjustment operation in real time and automatically updates the parameters of the annotation box to ensure that the annotation box more accurately fits the position and direction of the target object.

[0080] For example, suppose the autonomous driving system detects that the annotation box of a moving vehicle in a certain frame is inconsistent with the previous and next frames. The system prompts that the frame is a problem frame. The user clicks on the vehicle's annotation box through the interface and finds that the annotation box deviates from the actual position of the vehicle. The user drags the annotation box through the system interface to move it to the correct position, or rotates the annotation box to make it more closely fit the actual contour of the vehicle. The system updates the position and angle of the annotation box in real time based on the user's adjustment operation to ensure that the annotation box is accurately aligned with the position of the vehicle in the point cloud data, thereby correcting the annotation errors of the frame and maintaining the continuity and accuracy of the annotation.

[0081] In this way, the system responds to adjustment operations in real time, accurately updates the position and angle of the annotation box, ensures that the annotation results fit the target object, improves the accuracy and consistency of the annotation, reduces annotation deviation, and improves data processing efficiency.

[0082] In some embodiments, after the system prompts the problem frame, the point cloud data annotation method may further include:

[0083] In response to detecting a deletion operation on a current annotation box of a target object in a problem frame, the current annotation box of the target object is deleted, and a new annotation box is generated for the target object.

[0084] In some embodiments, deleting the current annotation box of the target object and regenerating the annotation box for the target object includes: receiving a deletion operation input by the user through the system interface, and the system automatically removes the annotation box of the target object in the frame; then, the system locates the blank frame after the trajectory and checks whether the target object exists. If the target object exists and has no real frame, the user can click the virtual frame to generate a real frame. If the target object already has a real frame, but the target identifier is inconsistent, the user can modify the target identifier to ensure that the target identifier in the blank frame is consistent with the target identifier in other frames.

[0085] For example, suppose that when a vehicle is driving on a certain road section, the system detects that there is an error in the annotation box of a certain frame. For example, the position of the system annotation box does not match the actual position of the vehicle. The user deletes the vehicle annotation box in this frame through the interface. The system immediately removes the annotation box of the frame and starts to check the next frame. The system locates the subsequent frame and finds that the vehicle still exists in the trajectory, but because the annotation box is deleted, only a virtual box remains in the current frame. At this time, the user clicks on the virtual box, and the system generates a real box that matches the actual position of the vehicle, making the annotation box reappear in the frame. If in a blank frame, the system detects that there is already a real box for the vehicle, but the identification is inconsistent, the system will prompt the user. The user can manually change the target identification of the frame to make it consistent with the target identification of the same vehicle in other frames, ensuring that the annotation box and identification are consistent throughout the trajectory to avoid subsequent recognition or tracking problems.

[0086] In this way, the system can easily delete and regenerate the annotation box to ensure that the annotation box matches the position of the target object. The system automatically checks subsequent frames and generates missing annotation boxes or corrects the target identification to ensure the continuity and consistency of annotation, reduce the complexity of manual operations, and improve annotation efficiency and accuracy.

[0087] In some embodiments, in response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation further includes:

[0088] When the moving trajectory of the target object is correct, a key frame is determined from the multi-frame point cloud data, where the key frame is a frame with a complete size of the target object;

[0089] Based on the size of the target object's annotation box in the keyframe, adjust the size of the target object's annotation box in other frames.

[0090] In some embodiments, determining a key frame from multiple frames of point cloud data includes: the system obtains the point cloud information of the target object of each frame from the multiple frames of point cloud data, analyzes its size (such as length, width, height), and determines the coverage of the target object in three-dimensional space by calculating the bounding box of the point cloud. The system calculates the size integrity based on the density and coverage of the point cloud, and frames with higher density and no occlusion are considered to have more complete size. At the same time, the system compares the size changes of the previous and next frames, and frames with smaller and stable size changes are considered to have higher size integrity. The system also detects the visibility of the target object in each frame, and unobstructed frames will receive higher weights. Based on these analyses, the system selects the frames with the most complete size, highest visibility, and most stable trajectory as key frames for subsequent annotation box adjustment and synchronization.

[0091] Figure 3 Shows the point cloud data annotation editing interface Figure 1 ,like Figure 3As shown in the figure, the type of the target object is a sedan, the target ID = 2, and the size of the annotation box (also called the reference box) is 4.71 in length, 2.01 in frame, and 1.75 in height; the batch operation types are divided into attribute labels and point cloud box length, width, and height, and the batch operation range is divided into current frame, all frames, current frame forward, current frame backward, and range frame. After selecting the point cloud box length, width, and height, you can select the range of the batch operation, for example, operate only on the current frame.

[0092] In some embodiments, adjusting the size of the annotation box of the target object in other frames includes: first, extracting the annotation box information of the target object from the key frame, including size parameters such as length, width, and height, and using these as reference standards for adjusting other frames. Then, by comparing the motion trajectory, posture, and position of the target object in other frames and the key frame, the similarity between the frames is analyzed to ensure accurate alignment. Then, using the annotation box size of the key frame as a benchmark, the annotation box in each frame is adjusted proportionally. If the target object is scaled or the angle changes in other frames, it is scaled or rotated according to the reference of the key frame. At the same time, adjust the position of the annotation box, compare the position information of the target object in the key frame and the current frame, and correct the position deviation of the annotation box in real time to ensure that the annotation box is aligned with the actual position of the target object. Finally, the annotation boxes in all frames are synchronously updated frame by frame to keep them consistent with the key frame in size and position, ensuring the accuracy and consistency of multi-frame annotation.

[0093] For example, suppose that the autonomous driving system is processing multi-frame point cloud data of a vehicle in front of it. The system first detects that the vehicle's movement trajectory in multiple frames is correct and begins to determine key frames from these frames. The system analyzes the point cloud information of the vehicle in each frame and calculates its size (such as length, width, height) and coverage. By comparing the density and occlusion of each frame, the system finds that the point cloud information of the vehicle in a certain frame is the most complete and not occluded, and selects this frame as the key frame. Next, the system extracts the size of the vehicle's annotation box from the key frame and uses it as a reference to adjust the annotation boxes in other frames. By analyzing the vehicle's trajectory and posture in other frames, the system finds that the vehicle's annotation box is offset in some frames due to turning. The system proportionally adjusts the annotation boxes of these frames based on the size of the key frame, and corrects the position and angle of the annotation box to ensure that it matches the actual position and shape of the vehicle. Finally, the system synchronously updates all annotation boxes frame by frame to ensure that the vehicle's annotation box is consistent with the key frame in size and position throughout the multi-frame point cloud data, ensuring the accuracy and consistency of the annotation.

[0094] In this way, by selecting the key frame with the most complete size and the highest visibility as the benchmark, the system can automatically adjust the annotation boxes of other frames to ensure the consistency of the size and position of the annotations in multiple frames. This not only improves the accuracy of the annotations and reduces the annotation errors caused by the rotation or movement of the target object, but also ensures the consistency of the annotation results in the entire point cloud data, improving the efficiency and accuracy of data processing.

[0095] In some embodiments, in response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation may further include:

[0096] While keeping the size of the annotation box unchanged, use the move function in the annotation tool to adjust the position and direction of the annotation box so that the edge of the annotation box fits the point cloud data of the target object.

[0097] In the disclosed embodiments, the annotation tool refers to an interactive software tool or function for creating, adjusting and editing annotation boxes in point cloud data or image data. It helps users or systems to automatically or manually mark the location, size, shape and orientation of target objects.

[0098] In some embodiments, the position and direction of the annotation box are adjusted using the moving function in the annotation tool, including: the system provides an interactive tool that allows the user to drag the annotation box to adjust the position so that its center is aligned with the target object, and adjust the direction by rotating the annotation box so that the edge is more closely aligned with the target object. The system provides real-time feedback on the adjustment results and displays the matching degree between the annotation box and the target object. When the annotation box is adjusted to a suitable position and angle, the system updates the position information to ensure that the annotation box accurately fits the point cloud contour of the target object.

[0099] For example, suppose the autonomous driving system detects that the position of the annotation box of a vehicle in a certain frame is inaccurate, but the size is correct. The user selects the annotation box through the interface, keeps the size unchanged, and uses the annotation tool to drag the annotation box so that its center is aligned with the actual position of the vehicle. The user further rotates the annotation box so that its edges are close to the outline of the vehicle. The system feedbacks the adjustment results in real time. When the annotation box accurately matches the vehicle, the system updates the position information to ensure that the annotation box is completely aligned with the point cloud data of the vehicle.

[0100] In this way, by keeping the size of the annotation box unchanged and using the move and rotate functions to accurately adjust the position and direction, the annotation box can more closely fit the actual contour of the target object and correct the annotation deviation. This not only improves the accuracy of annotation, but also reduces recognition errors caused by improper position of the annotation box, and improves the efficiency of system processing and annotation consistency.

[0101] In some embodiments, in response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation further includes:

[0102] When the moving trajectory of the target object is correct, the attributes of the target object are judged frame by frame. When the attributes of the target object in the current frame and its adjacent frames are not completely the same, a second prompt information is output. The second prompt information is used to prompt the verification of the attributes of the target object. The second prompt information includes the frame number and attribute identifier to be verified.

[0103] In the embodiment of the present disclosure, the second prompt information refers to a prompt provided to the user when the system detects that certain attributes of the target object are inconsistent, abnormal or erroneous between different frames or at different times during the labeling process, aiming to remind the user to check or correct.

[0104] In the disclosed embodiments, attribute identification refers to a label or mark used to identify and describe a specific attribute or feature of a target object, including but not limited to vehicle type (sedan, two-wheeled vehicle, three-wheeled vehicle, pedestrian, truck, bus, road pile / warning column, ignorable category, etc.), identification number, degree of occlusion (no occlusion, partial occlusion, severe occlusion, etc.), and size of reference frame, to help the system or user better understand and distinguish different objects.

[0105] In some embodiments, when the moving trajectory of the target object is correct, the system will automatically determine the attributes of the target object (such as type, size, color, etc.) frame by frame. If the system finds that the attributes in the current frame are inconsistent with those in the adjacent frame, for example, the target object is marked as a "car" in the current frame and as a "truck" in the adjacent frame, the system will automatically output a second prompt message. The second prompt message includes a frame number and an attribute identifier, which is used to indicate the frame in which the difference occurs and the specific attributes of the target object. The second prompt message will inform the user in which frames the attribute difference occurs, such as the prompt "The attribute type of target ID=001 in frame 10 and frame 11 is inconsistent", and display the specific difference content, such as the vehicle type. Through the prompt, the user can quickly verify and manually correct the annotated attribute information to make it consistent in all frames, ensuring the accuracy and consistency of the annotation results.

[0106] In this way, by detecting the differences in the attributes of the target object frame by frame, the system can automatically discover the inconsistency of the attributes during the annotation process, and remind the user to verify and correct it through the second prompt information. This not only improves the accuracy of the annotation, but also ensures the consistency of the attributes in the multi-frame annotation, reducing the misidentification or annotation errors caused by attribute differences.

[0107] In some embodiments, in response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation may further include:

[0108] In response to receiving a synchronization operation for a target attribute of a target object in a current frame, target data corresponding to the target attribute in the current frame is synchronized with target attributes of target objects in other frames.

[0109] In some embodiments, synchronizing the target data corresponding to the target attribute in the current frame to the target attribute in the target object in other frames includes: the system first detects the synchronization operation performed by the user in the current frame, and obtains the attribute data of the target object in the current frame, such as the type, size, color or other attribute identification of the target object. Then, the system locates the target object in the multi-frame point cloud data and compares the target attributes in these frames frame by frame. During the synchronization process, the system copies and applies the adjusted or confirmed target attributes (such as the target type is "car" or size information) in the current frame to the corresponding target object in other frames, overwriting the original attribute information. The system automatically identifies the frames that need to be synchronized according to the trajectory of the target object to ensure the accuracy of attribute synchronization. At the same time, the system checks the abnormal conditions in the synchronization process in real time, such as the target object is blocked or part of the point cloud data is lost in some frames, and the system will issue a prompt to remind the user to manually verify or correct it. After the synchronization is completed, the system updates the attribute data of the target object frame by frame to ensure that the attributes of the target object in all frames are consistent, avoiding recognition or tracking errors caused by inconsistent attributes.

[0110] Figure 4 Shows the point cloud data annotation editing interface Figure 2 ,like Figure 4 As shown, the type of the target object is a sedan, the target identifier ID = 2, and the dimensions of the annotation box (also called the reference box) are 4.71 in length, 2.01 in frame, and 1.75 in height; the batch operation types are divided into attribute labels and point cloud frame length, width, and height, and the batch operation range is divided into current frame, all frames, current frame forward, current frame backward, and range frame. After selecting the attribute label, elements of the same type and elements of different types are displayed, and both types of elements support modification. Elements of the same type include ID and occlusion degree. For example, after modifying the occlusion degree attribute of the current frame, you can continue to select the batch operation range for this attribute.

[0111] For example, suppose that the autonomous driving system is processing vehicle annotations in multi-frame point cloud data in a certain scenario. The system annotates a moving vehicle and finds that the target attribute of the vehicle is incorrectly marked as "truck" in frame 20, but it is actually a "car". So the user manually changes the target attribute of the vehicle from "truck" to "car" in frame 20 and performs a synchronization operation, requiring this attribute update to be applied to the same target object in all frames. The system first detects the synchronization operation input by the user and extracts the target attribute data in frame 20, including the "car" type, the length, width, height and other related information of the vehicle. Then, the system locates the target vehicle frame by frame in the multi-frame point cloud data along the vehicle's motion trajectory, and checks the attribute identification of the vehicle in these frames frame by frame. The system finds that the target vehicle between frames 10 and 30 is still marked as "truck" in some frames, and the size information is also different in some frames. During the synchronization process, the system copies the modified "car" attribute and size data in frame 20 to these frames with inconsistent attributes, overwriting the original incorrect attribute identification. At the same time, the system intelligently identifies abnormal situations in certain frames. For example, in the 25th frame, the vehicle is partially blocked by other objects, resulting in incomplete point cloud data. The system issues a prompt at this time to remind the user to manually check or adjust these special frames. After the synchronization operation is completed, the system will update the attribute data of the target vehicle frame by frame to ensure that from the 10th frame to the 30th frame, the attribute identification of the vehicle is consistent as "sedan" and the size data is synchronized and unified.

[0112] In this way, through the automated attribute synchronization operation, the system can efficiently apply the correct attributes of the target object in the current frame to other frames, avoiding the tedious operation of manual frame-by-frame modification by the user. This synchronization process ensures the consistency of the attributes of the target object in multiple frames of data, especially in key attributes such as the type and size of the vehicle, avoiding recognition or tracking errors caused by inconsistent attributes.

[0113] In some embodiments, the point cloud data annotation method may further include:

[0114] Verifying the second annotation result of the target object frame by frame;

[0115] When both the previous frame and the next frame of the current frame have target identifiers and the current frame does not have a target identifier, a third prompt information is output, which is used to prompt a supplementary label, and includes a missing frame number and a missing identifier.

[0116] In the disclosed embodiment, supplementary annotation refers to checking the missing frame number and missing label identifier provided by the system according to the third prompt information, locating the specific missing frame, and supplementing the lost annotation data in the frame to ensure that the target object remains completely labeled in all frames.

[0117] In the disclosed embodiment, the missing frame number refers to the number of the frame where the system detects that the annotation result is missing. That is, in multi-frame point cloud data, when the system detects that the target object has labels in the previous and next frames, but lacks annotations in the current frame, the system will output a prompt message containing the current missing frame number to help users quickly locate the frame.

[0118] In the disclosed embodiments, the missing label indicator refers to a unique identifier of a target object, which is usually used to mark and track the existence of the object in different frames. It helps users to clearly know which target object's label information is missing in the frame. The identifier can be a number, letter or other mark, used to identify a specific target object, such as a vehicle, pedestrian, etc.

[0119] In some embodiments, the second labeling result of the target object is checked frame by frame, including: the system checks the data of each frame frame by frame to confirm whether the labeling of the target object in each frame exists continuously. The system first determines whether the target object exists in these frames by detecting the target identification in the previous frame and the next frame. If the system detects that the current frame lacks the target identification (such as the labeling box is missing or not labeled), and the identification (such as ID number) of the target object in the previous frame and the next frame exists, the system confirms that the current frame is a "missing label frame". After the system detects the missing label, it automatically generates a third prompt information, the content of which includes: missing label frame number, missing label identification. The third prompt information may be presented in a pop-up window, text prompt or other form to remind the user to make up the label in time. When the user receives the third prompt information, the system allows the user to click the third prompt information to jump to the corresponding missing label frame. The user can manually supplement the missing label box through the user interface. The system can also provide suggestions and automatically generate a virtual box for the user to confirm and generate an actual label box to help the user quickly complete the labeling.

[0120] Figure 5 Shows the point cloud data annotation editing interface Figure 3 ,like Figure 5 As shown in the figure, the interface supports single-frame view and multi-frame view inspection modes, supports selected elements and non-selected elements, can adjust the key frames to be marked, and can view all frames. Through intelligent detection technology, it can detect frames under certain conditions such as the frame with the most points, the frame with the largest size, the frame with the largest compact frame, and the frame closest to the collection vehicle. At the same time, it also displays the labeling results, annotation results and associated relationships of each frame.

[0121] For example, suppose the system detects that the target object ID "ID001" has a label box in the 15th and 17th frames, but is missing in the 16th frame. The system immediately generates a third prompt message, prompting the user that "target ID001 is missing in the 16th frame" and provides the frame number and target ID001 of the 16th frame. The user clicks the prompt to jump to the 16th frame, generates a label box manually or through automatic suggestions, and supplements the missing label data of the 16th frame to ensure the coherence and consistency of multi-frame labeling.

[0122] In this way, by automatically detecting the missing frames and generating prompt information, the system can quickly remind users to make additional annotations, avoiding the tediousness of manual frame-by-frame inspection. Since the prompt information clearly identifies the missing frame number and target identification, it helps users accurately locate the problem and make additional annotations in a timely manner, ensuring the integrity and consistency of annotations in multiple frames of data and improving annotation efficiency.

[0123] Figure 6 Shows the point cloud data annotation editing interface Figure 4 ,like Figure 6 As shown, in this editing interface, by selecting all states and all views, the top view, rear view and side view corresponding to the current frame number (such as the 27th frame), the next frame (such as the 28th frame) and the next frame (such as the 29th frame) are displayed; based on the point cloud data under the three perspectives corresponding to each frame, the top view, rear view and side view corresponding to each frame can be outlined. By comparing the images of the 27th, 28th and 29th frames, it can be determined that the 27th frame has the most points and can be used as a key frame. In practical applications, the frame with the "most points", the frame with the "largest size", the frame with the "largest compact frame" and the frame with the "largest distance from the collection vehicle" can be intelligently detected. It can be understood that the above editing interface can be selected or set according to needs.

[0124] Users can use the annotation system to perform the following operations: observe the pre-annotation effect, and choose whether to clear and re-annotate according to the effect; use the trajectory function to check and adjust the effect of ID concatenation; determine the key frame in the multi-frame view and adjust the size and position of the annotation box; judge and modify the attributes to ensure the accuracy of the annotation. Through the automated annotation process, the efficiency of 3D data processing and the consistency of annotation are significantly improved, the labor cost is reduced, and the accuracy of annotation is guaranteed.

[0125] It should be understood that Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 The schematic diagram shown is only exemplary and not restrictive, and it is scalable, and those skilled in the art can Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0126] The present disclosure provides a point cloud data annotation device. Figure 7 As shown, the point cloud data annotation device may include: an acquisition module 701, used to acquire a target identifier and a target output mode; a determination module 702, used to determine a first annotation result of a target object corresponding to the target identifier; an output module 703, used to present the first annotation result in a target output mode, the first annotation result being obtained by annotating multiple frames of point cloud data with a large visual model; an update module 704, used to update the first annotation result based on the adjustment operation in response to receiving an adjustment operation for the first annotation result; and a generation module 705, used to obtain a second annotation result of the target object based on the updated first annotation result.

[0127] In some embodiments, the output module 703 includes: a first output submodule, configured to present the moving trajectory of the target object in the multi-frame point cloud data in a target output manner.

[0128] In some embodiments, the output module 703 also includes: a second output submodule, used to output a first prompt message when a jump occurs in the moving trajectory, the first prompt message is used to prompt that there is a problem frame in the multi-frame point cloud data, and the problem frame is a frame where a jump occurs; in response to receiving a positioning operation for the problem frame, the problem frame is displayed and the position of the target object in the problem frame is prompted.

[0129] In some embodiments, the updating module 704 further includes: a first adjustment submodule, configured to adjust the position or angle of the current annotation box based on the adjustment operation in response to detecting an adjustment operation on the current annotation box of the target object in the problem frame.

[0130] In some embodiments, the updating module 704 further includes: a second adjusting submodule, configured to delete the current annotation box in response to detecting a deletion operation of the current annotation box for the target object in the problem frame, and to regenerate an annotation box for the target object.

[0131] In some embodiments, the update module 704 includes: a determination submodule, used to determine the key frame from the multi-frame point cloud data when the movement trajectory of the target object is correct; a third adjustment submodule, used to adjust the annotation box size of the target object in other frames based on the annotation box size of the target object in the key frame.

[0132] In some embodiments, the updating module 704 further includes: a fourth adjustment submodule, which is used to adjust the position and direction of the annotation box by using the moving function in the annotation tool while keeping the size of the annotation box unchanged, so that the edge of the annotation box fits the point cloud data of the target object.

[0133] In some embodiments, the update module 704 includes: a judgment submodule, which is used to judge the properties of the target object frame by frame when the moving trajectory of the target object is correct. The output module 703 also includes: a third output submodule, which is used to output a second prompt information when the properties of the target object in the current frame and its adjacent frame are not completely the same, and the second prompt information is used to prompt the verification of the properties of the target object, and the second prompt information includes the frame number and property identifier to be verified.

[0134] In some embodiments, the update module 704 further includes: a synchronization submodule for synchronizing target data corresponding to the target attribute in the current frame with the target attribute of the target object in other frames in response to receiving a synchronization operation for the target attribute of the target object in the current frame.

[0135] In some embodiments, the device further comprises: a testing module ( Figure 7 The output module 703 further includes: a fourth output submodule, which is used to output a third prompt message when the target identifier exists in both the previous frame and the next frame of the current frame and the current frame does not have a target identifier, and the third prompt message is used to prompt for supplementary annotation, and the third prompt message includes a missing frame number and a missing identifier.

[0136] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0137] The point cloud data annotation device of the embodiment of the present disclosure can improve the efficiency of data processing and the consistency of annotation, reduce labor costs, and improve the accuracy of annotation.

[0138] The present disclosure provides a scene diagram of a point cloud data annotation method, such as Figure 8 shown.

[0139] As mentioned above, the point cloud data annotation method provided by the embodiment of the present disclosure is applied to electronic devices. The electronic devices are intended to represent various forms of digital computers, such as

[0140] For example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0141] Specifically, the electronic device can perform the following operations:

[0142] Get the target ID and target output mode;

[0143] Determine a first annotation result of the target object corresponding to the target identifier, and present the first annotation result in a target output manner, where the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model;

[0144] In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation;

[0145] A second annotation result of the target object is obtained based on the updated first annotation result.

[0146] It should be understood that Figure 8 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Figure 8 Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0147] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0148] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0149] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0150] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0151] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0152] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the point cloud data annotation method. For example, in some embodiments, the point cloud data annotation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the point cloud data annotation method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the point cloud data annotation method in any other appropriate manner (for example, by means of firmware).

[0153] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0155] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0157] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

[0158] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0159] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A point cloud data annotation method, comprising: Get the target ID and target output mode; Determine a first annotation result of the target object corresponding to the target identifier, and present the first annotation result in the target output mode, wherein the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model; In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation; A second labeling result of the target object is obtained based on the updated first labeling result.

2. The method according to claim 1, wherein: The presenting the first annotation result in the target output mode includes: The moving trajectory of the target object in the multi-frame point cloud data is presented in the target output manner.

3. The method according to claim 2, wherein: The presenting the first annotation result in the target output mode further includes: When the moving trajectory jumps, outputting first prompt information, wherein the first prompt information is used to prompt that there is a problem frame in the multi-frame point cloud data, and the problem frame is a frame that jumps; In response to receiving a positioning operation for the problem frame, the problem frame is displayed and the position of the target object in the problem frame is indicated.

4. The method according to claim 3, further comprising: In response to detecting an adjustment operation on a current annotation box of the target object in the problem frame, a position or angle of the current annotation box is adjusted based on the adjustment operation.

5. The method according to claim 3, further comprising: In response to detecting a deletion operation on a current annotation box of the target object in the problem frame, the current annotation box is deleted, and a new annotation box is generated for the target object.

6. The method according to claim 2, wherein: In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation includes: When the moving trajectory of the target object is correct, determining a key frame from the multi-frame point cloud data; Based on the size of the annotation box of the target object in the key frame, the size of the annotation box of the target object in other frames is adjusted.

7. The method according to claim 6, wherein: In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation further includes: When the size of the annotation box remains unchanged, the position and direction of the annotation box are adjusted using the moving function in the annotation tool so that the edge of the annotation box fits the point cloud data of the target object.

8. The method according to claim 2, wherein: In response to receiving an adjustment operation on the first annotation result, updating the first annotation result based on the adjustment operation includes: When the moving trajectory of the target object is correct, the attributes of the target object are judged frame by frame. When the attributes of the target object in the current frame and its adjacent frames are not completely the same, a second prompt information is output, and the second prompt information is used to prompt the verification of the attributes of the target object. The second prompt information includes the frame number and attribute identifier to be verified.

9. The method according to claim 8, further comprising: In response to receiving a synchronization operation for a target attribute of the target object in the current frame, the target data corresponding to the target attribute in the current frame is synchronized with the target attribute of the target object in other frames.

10. The method according to any one of claims 1 to 9, further comprising: Verifying the second labeling result of the target object frame by frame; When the target identifier exists in both the previous frame and the next frame of the current frame and the current frame does not have the target identifier, a third prompt information is output, wherein the third prompt information is used to prompt for supplementary labeling, and the third prompt information includes a missing frame number and a missing identifier.

11. A point cloud data annotation device, comprising: An acquisition module is used to obtain a target identifier and a target output mode; A determination module, used to determine a first marking result of the target object corresponding to the target identifier; An output module, configured to present the first annotation result in the target output mode, wherein the first annotation result is obtained by annotating multiple frames of point cloud data with a large visual model; an updating module, configured to, in response to receiving an adjustment operation on the first annotation result, update the first annotation result based on the adjustment operation; A generating module is used to obtain a second marking result of the target object based on the updated first marking result.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are for causing a computer to perform a method according to any one of claims 1-10.

14. A computer program product comprising a computer program stored on a storage medium, the computer program implementing the method according to any one of claims 1 to 10 when executed by a processor.

Images