Point cloud data labeling method, device and equipment, readable storage medium and product
By obtaining the correlation between point cloud frame sequence and target keyframe and performing prediction and correction operations, the problems of low efficiency and insufficient accuracy of three-dimensional point cloud data are solved, and more efficient and accurate labeling effects are achieved to meet the field of autonomous driving data quality requirements.
Patent Information
- Application Number
- CN202411238348.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, the labeling efficiency of three-dimensional point cloud data is low and the accuracy cannot meet the actual application needs. Especially in the field of autonomous driving, the quality requirements for the three-dimensional point cloud labeling data are high, but the manual labeling method is low and the accuracy is insufficient. .
By obtaining the point cloud frame sequence, the target keyframe associated with each point cloud frame is determined, and the prediction operation is performed based on the annotation data of the target keyframe is obtained, and the position correction is performed through the preset data correction algorithm to improve the annotation Data accuracy.
It improves the accuracy and efficiency of three-dimensional point cloud data annotation, and can more accurately label target objects to meet high-quality data needs in fields such as autonomous driving.
Smart Images

Figure CN119091253A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to intelligent transportation in data processing, and in particular to a point cloud data annotation method, device, equipment, readable storage medium and product. Background Art
[0002] With the continuous development of artificial intelligence technology, the demand for data of various models is increasing. Especially in the field of autonomous driving, the demand for 3D point cloud annotation data is increasing, and the quality requirements for 3D point cloud annotation data are getting higher and higher.
[0003] Currently, the annotation of 3D point cloud data mainly relies on manual annotation methods. However, manual annotation often has low annotation efficiency, and the accuracy of the obtained 3D point cloud annotation data cannot meet the needs of practical applications. Summary of the invention
[0004] The present disclosure provides a point cloud data labeling method, device, equipment, readable storage medium and product for accurately labeling three-dimensional power supply data.
[0005] According to a first aspect of the present disclosure, a point cloud data annotation method is provided, comprising:
[0006] Acquire a point cloud frame sequence, wherein the point cloud frame sequence includes a plurality of unlabeled point cloud frames and at least one key frame for labeling a target object, wherein the key frame is associated with labeling data;
[0007] For the point cloud frame, determining at least one target key frame spatially associated with the point cloud frame;
[0008] Performing a prediction operation on a target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain predicted annotation data associated with the target object;
[0009] Performing a posture correction operation on the predicted labeled data according to a preset data correction algorithm to obtain target labeled data;
[0010] A data annotation operation is performed on the point cloud frame based on the target annotation data to obtain a labeled target point cloud frame.
[0011] According to a second aspect of the present disclosure, a point cloud data annotation device is provided, comprising:
[0012] An acquisition module, used for acquiring a point cloud frame sequence, wherein the point cloud frame sequence includes a plurality of unlabeled point cloud frames and at least one key frame for labeling a target object, wherein the key frame is associated with labeling data;
[0013] A determination module, configured to determine, for the point cloud frame, at least one target key frame spatially associated with the point cloud frame;
[0014] A prediction module, configured to perform a prediction operation on a target object in the point cloud frame based on the annotation data associated with the at least one target key frame, to obtain predicted annotation data associated with the target object;
[0015] A correction module, used to perform a posture correction operation on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data;
[0016] The labeling module is used to perform a data labeling operation on the point cloud frame based on the target labeling data to obtain a labeled target point cloud frame.
[0017] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0018] at least one processor; and
[0019] a memory communicatively connected to the at least one processor; wherein,
[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.
[0021] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.
[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.
[0023] The technology disclosed herein improves the accuracy of three-dimensional point cloud data annotation.
[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0026] Figure 1 A diagram of the system architecture on which the present disclosure is based;
[0027] Figure 2 A schematic diagram of a process of a point cloud data annotation method provided in an embodiment of the present disclosure;
[0028] Figure 3 A schematic diagram of a point cloud frame sequence provided in an embodiment of the present disclosure;
[0029] Figure 4 A schematic diagram of a process of a point cloud data annotation method provided by another embodiment of the present disclosure;
[0030] Figure 5 A schematic diagram of a process of a point cloud data annotation method provided by another embodiment of the present disclosure;
[0031] Figure 6 A schematic diagram of correction of a prediction annotation frame provided in an embodiment of the present disclosure;
[0032] Figure 7 A schematic diagram of a process of a point cloud data annotation method provided by another embodiment of the present disclosure;
[0033] Figure 8 A schematic diagram of the structure of a point cloud data annotation device provided in an embodiment of the present disclosure;
[0034] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0035] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0036] Glossary:
[0037] 3D Point Cloud: A data structure used to represent objects or scenes in three-dimensional space. In its most basic form, it is a collection of multiple three-dimensional coordinate points (X, Y, Z). These points are obtained by discrete sampling of the surface of the real object or scene, so the point cloud can be regarded as a discrete representation of the scene surface in a given coordinate system.
[0038] Ground fitting algorithm: used to separate ground point clouds from point cloud data to improve the calculation speed and point cloud segmentation quality. This algorithm is particularly important in the fields of autonomous driving, robot navigation, etc., because they need to accurately identify and exclude ground point clouds in order to better identify and track non-ground objects.
[0039] The present invention provides a point cloud data annotation method, device, equipment, readable storage medium and product, which are applied to intelligent transportation in the field of data processing to achieve the technical effect of accurately annotating targets in three-dimensional point cloud data.
[0040] In the related art, the annotation of 3D point cloud data is generally achieved through manual annotation or simple interpolation. However, the manual annotation method for 3D point cloud annotation often requires a large amount of annotations, and the manual annotation efficiency is low and the accuracy is poor. The simple interpolation method for point cloud annotation cannot cope with more complex actual scenes, so the accuracy cannot meet expectations.
[0041] In the process of solving the above technical problems, the inventors found through research that in order to improve the accuracy of 3D point cloud annotation, the spatial correlation between the target objects in the 3D point cloud sequence frames should be considered first. Therefore, after obtaining the point cloud sequence frames, for each point cloud frame, at least one pre-annotated valid key frame with which there is a spatial correlation can be determined. Interpolation calculation is performed on at least one valid key frame, and the predicted annotation data of the target object in the point cloud frame is predicted based on the interpolation result.
[0042] Furthermore, in order to improve the accuracy of the predicted annotation data, the predicted annotation data can be subjected to posture correction based on a preset data correction algorithm to obtain target annotation data. The point cloud frame is subjected to data annotation based on the target annotation data to obtain an annotated target point cloud frame.
[0043] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0044] In order to enable readers to more deeply understand the implementation principle of the present disclosure, the following Figure 1-Figure 8 The embodiments of the present disclosure are further refined.
[0045] Figure 1 This is the system architecture diagram on which the present disclosure is based, such as Figure 1As shown, the system architecture on which the present disclosure is based includes at least a server 11 and a data server 12. Among them, a point cloud data annotation device is pre-installed in the server 11. The point cloud data annotation device can be written in languages such as C / C++, Java, Shell or Python; the data server 12 can be a cloud server or a server cluster, which stores a large amount of point cloud data. The point cloud data stored in the data server 12 can be uploaded by the data collection vehicle after the point cloud data is collected, and the user can pre-annotate at least part of the point cloud frames according to actual needs, and form a point cloud frame sequence based on multiple annotated key frames and unannotated point cloud frames that are continuous in time sequence.
[0046] Based on the above system architecture, the server 11 can obtain a point cloud frame sequence from the data server 12. For each unlabeled point cloud frame in the point cloud frame sequence, at least one valid key frame associated with the point cloud frame can be determined, and a position prediction operation is performed on the target object in the point cloud frame based on the at least one valid key frame to obtain predicted annotation data associated with the target object. A position correction operation is performed on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data. An annotation operation is performed on the point cloud frame based on the target annotation data.
[0047] Figure 2 A schematic diagram of the process of the point cloud data annotation method provided by the embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the method includes:
[0048] Step 201: Acquire a point cloud frame sequence, wherein the point cloud frame sequence includes a plurality of unlabeled point cloud frames and at least one key frame for labeling a target object, wherein the key frame is associated with labeling data.
[0049] The execution subject of this embodiment is a point cloud data annotation device, which can be coupled to a server.
[0050] In this embodiment, in order to implement the annotation operation on point cloud data, a point cloud frame sequence can be first obtained, wherein the point cloud frame sequence includes multiple unlabeled point cloud frames and at least one key frame for labeling the target object, and the key frame is associated with the annotation data.
[0051] Optionally, the point cloud frame sequence may be collected by a data collection vehicle. The user may annotate some of the point cloud frames in the point cloud frame sequence according to actual needs. The user may manually annotate the position, rotation direction, size, etc. of the target object in the point cloud frame sequence. The target object may be a dynamic obstacle in the point cloud frame, such as a motor vehicle, a bicycle, a pedestrian, etc.
[0052] As an implementable method, the user can manually add a labeling box in the point cloud frame to implement the labeling operation. Alternatively, the user can also select or adjust the automatically identified labeling box to implement the labeling operation, which is not limited in the present disclosure.
[0053] For example, multiple point cloud frames in a point cloud frame sequence are continuous in time sequence. The user can perform annotation operations in chronological order. For example, for a point cloud frame in a point cloud frame sequence, the user can first determine whether to annotate the point cloud frame. If annotation is not required, the user can jump to the next point cloud frame. If annotation is required, an annotation box can be added to the current point cloud frame, or the automatically identified annotation box can be adjusted in the three views to complete the annotation operation and obtain a key frame. Continue to annotate the next point cloud frame until the annotation operation of the point cloud frame sequence is completed.
[0054] Optionally, during the annotation process, the user can adjust or perform annotation operations on the position, rotation direction, and size of the annotation box.
[0055] Step 202: for the point cloud frame, determine at least one target key frame spatially associated with the point cloud frame.
[0056] In this embodiment, for a point cloud frame sequence that is continuous in time, the target object has spatial continuity between each point cloud frame in the point cloud frame sequence. For example, the first point cloud frame, the second point cloud frame, and the third point cloud frame are continuous in time, and the timestamp of the first point cloud frame is earlier than that of the second point cloud frame and the third point cloud frame. The target vehicle is driving on the road. In the first point cloud frame, the target vehicle may be located on the left side of the point cloud frame, in the second point cloud frame, the target vehicle may drive to the middle of the point cloud frame, and in the third point cloud frame, the target vehicle may drive to the right side of the point cloud frame.
[0057] Therefore, since the point cloud frame sequence includes multiple annotated key frames, the annotated data associated with the key frame can accurately represent the position and pose information of the target object. Therefore, the position and pose of the target object can be predicted based on the spatial continuity between the point cloud frame and the target object in the key frame. Therefore, for each point cloud frame, at least one target key frame that is spatially associated with the point cloud frame is determined.
[0058] Step 203: Perform a prediction operation on the target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain predicted annotation data associated with the target object.
[0059] In this embodiment, after determining at least one target key frame associated with the point cloud frame, a prediction operation can be performed on the target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain the predicted annotation data associated with the target object.
[0060] The predicted annotation data may be a predicted annotation frame associated with the target object, and the predicted annotation frame may represent parameters such as the position, rotation direction, and size of the target object.
[0061] For example, the motion parameters such as the moving speed and moving direction of the target object can be determined according to the first target key frame before the point cloud frame and the second target key frame after the point cloud frame, and the position and posture of the target object in the current point cloud frame can be predicted based on the above motion parameters.
[0062] Step 204: Perform a posture correction operation on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data.
[0063] In this embodiment, in order to further improve the accuracy of data annotation, for each target object associated predicted annotation data, a posture correction operation can be performed on the predicted annotation data based on a preset data correction algorithm to obtain target annotation data, so that the corrected target annotation data is more consistent with the actual posture of the target object.
[0064] As an implementable method, the user can determine whether each point cloud frame needs data correction during the annotation process. For example, the user can set data correction parameters for more complex point cloud frames during the annotation process. Therefore, after the point cloud frame sequence is acquired, the point cloud frame associated with the data correction parameters can be corrected.
[0065] Step 205: perform a data annotation operation on the point cloud frame based on the target annotation data to obtain a labeled target point cloud frame.
[0066] In this embodiment, after the corrected target annotation data is obtained, a data annotation operation can be performed on the point cloud frame based on the target annotation data to obtain an annotated target point cloud frame.
[0067] The point cloud data annotation method provided by the embodiment of the present disclosure predicts the target object according to at least one target key frame associated with the point cloud frame, so that the pose prediction operation can be performed based on the spatial correlation between the target object and the point cloud frame sequence, thereby improving the accuracy of the pose prediction of the target object. In addition, by performing a pose correction operation on the predicted annotation data through a preset data correction algorithm, the accuracy of the pose prediction of the target object can be further improved, and then the point cloud frame annotation operation can be accurately performed based on the corrected target annotation data, thereby improving the accuracy and efficiency of the annotation.
[0068] Optionally, based on any of the above embodiments, the point cloud frame sequence includes multiple key frames, and the number of the target key frames is two. Step 202 includes:
[0069] For all key frames in the point cloud frame sequence, the key frames are arranged according to the time sequence of the key frames to obtain a key frame sequence.
[0070] The first key frame and the last key frame in the key frame sequence are determined as the at least one target key frame. Alternatively, the time sequence relationship between the point cloud frame and the key frames in the key frame sequence is determined, and two key frames adjacent to the point cloud frame in time sequence are determined as the at least one target key frame.
[0071] In this embodiment, the point cloud frame sequence includes multiple key frames, so the key frames can be arranged in time sequence to obtain a key frame sequence.
[0072] As an implementable manner, the number of target key frames is two. For each point cloud frame, the first key frame and the last key frame in the key frame sequence can be directly determined as at least one target key frame.
[0073] Alternatively, in order to improve the accuracy of data prediction, a key frame that is temporally adjacent to the point cloud frame may be determined as at least one target key frame. The temporal relationship between the point cloud frame and each key frame in the key frame sequence is determined, and two key frames that are temporally adjacent to the point cloud frame are determined as at least one target key frame.
[0074] Optionally, the user may determine the number of target key frames according to actual needs. For example, multiple key frames that are adjacent to the point cloud frame in time sequence may be determined as at least one target key frame. The present disclosure does not limit this.
[0075] Figure 3 A schematic diagram of a point cloud frame sequence provided in an embodiment of the present disclosure, such as Figure 3 As shown, the point cloud frame sequence 31 includes a plurality of point cloud frames that are continuous in time sequence. Among them, some point cloud frames are pre-annotated key frames 32. For the target point cloud frame 33 to be annotated currently, the first key frame and the last key frame in the key frame sequence consisting of all the key frames 32 can be determined as the at least one target key frame 34.
[0076] The point cloud data annotation method provided by the embodiment of the present disclosure determines the key frames located at both ends of the point cloud frame as the target key frames based on the temporal relationship between the point cloud frame and the key frames, thereby being able to perform pose prediction operations based on the spatial correlation between the target object in the point cloud frame sequence, thereby improving the accuracy of the target object pose prediction.
[0077] Figure 4 A schematic diagram of a point cloud data annotation method according to another embodiment of the present disclosure is shown in FIG. Figure 4As shown, the number of the target key frames is at least two. Step 203 includes:
[0078] Step 401: Perform interpolation calculation on at least two target key frames based on the annotation data associated with the target key frames to obtain interpolation data.
[0079] Step 402: Predict the position and posture of the target object in the point cloud frame according to the interpolation data to obtain the predicted annotation data.
[0080] In this embodiment, after determining at least two target key frames associated with the point cloud frame, an interpolation operation may be performed on the at least two target key frames to implement a prediction operation on the target object in the point cloud frame.
[0081] Optionally, at least two target key frames may be interpolated based on the annotation data associated with the target key frames to obtain interpolation data. The interpolation data may accurately represent the motion parameters of the target object, and then the position and posture of the target object in the point cloud frame may be predicted based on the interpolation data to obtain predicted annotation data.
[0082] The point cloud data annotation method provided by the embodiment of the present disclosure obtains interpolation data by performing interpolation calculations on at least two target key frames based on the annotation data associated with the target key frames, so that the position and posture of the target object in the point cloud frame can be accurately predicted based on the interpolation data. Combined with the spatial continuity of the target object, the accuracy of the target object position and posture prediction can be improved.
[0083] Further, based on any of the above embodiments, step 401 includes:
[0084] The pose information of the at least two target key frames is converted to the same preset coordinate system to obtain a pose conversion result associated with the at least two target key frames.
[0085] Based on the pose conversion result and the interpolation factor associated with the point cloud frame, position interpolation settlement and rotation angle interpolation calculation are performed to obtain the interpolation data, wherein the interpolation data includes position interpolation data and rotation angle interpolation data, and the interpolation factor is determined based on the relative position between the point cloud frame and the at least two target key frames.
[0086] In this embodiment, since the target object is a dynamic obstacle, its position and posture will change as the target object moves, so it is necessary to determine the position and rotation angle of the target object.
[0087] For the convenience of calculation, the pose information of at least two target key frames can be converted to the same preset coordinate system to obtain the pose conversion results associated with at least two target key frames. Based on the pose conversion results and the interpolation factors associated with the point cloud frame, position interpolation settlement and rotation angle interpolation calculation are performed to obtain interpolation data, wherein the interpolation data includes position interpolation data and rotation angle interpolation data, and the interpolation factor is determined based on the relative position between the point cloud frame and at least two target key frames. Since the interpolation factor is determined based on the relative position of the current point cloud frame between at least one target key frame, it effectively reflects the position of the current point cloud frame on the timeline, ensures the continuity in time, and improves the accuracy of the prediction.
[0088] Optionally, during the data annotation process, the size of the target object also needs to be annotated, and although the dynamic movement of the target object will cause a change in posture, it will not cause a change in size. Therefore, the size annotation data of the target object in the target key frame can be determined as the size annotation data of the target object in the point cloud frame.
[0089] The point cloud data annotation method provided by the embodiment of the present disclosure performs position interpolation calculations and rotation angle interpolation calculations based on the annotation data associated with at least two target key frames, thereby being able to accurately predict the position and posture of the target object based on the spatial continuity of the target object in the point cloud frame, thereby improving the accuracy of the target object position and posture prediction.
[0090] Further, based on any of the above embodiments, step 402 includes:
[0091] Based on the position interpolation data and the rotation angle interpolation data, a prediction operation is performed on the moving position and the rotation angle of the target object in the point cloud frame to obtain posture prediction information.
[0092] The predicted annotation data is determined based on the pose prediction information and size information associated with the target object, wherein the size information is determined based on the annotation data, or the size information is determined by performing target object recognition on the point cloud frame.
[0093] In this embodiment, after performing an interpolation operation on at least two target key frames to obtain interpolation data, a prediction operation can be performed on the moving position and rotation angle of the target object in the point cloud frame based on the position interpolation data and the rotation angle interpolation data to obtain posture prediction information. Prediction annotation data is determined based on the posture prediction information and the size information associated with the target object, wherein the size information is determined based on the annotation data associated with the target key frame, or the size information is determined by identifying the target object for the point cloud frame.
[0094] The point cloud data annotation method provided by the embodiment of the present disclosure can accurately predict the pose of the target object by predicting the pose of the target object based on the time difference information between the point cloud frame and at least two valid key frames, as well as the position interpolation data and the rotation angle interpolation data, thereby improving the accuracy of the pose prediction of the target object.
[0095] Optionally, based on any of the above embodiments, step 204 includes:
[0096] It is determined whether the point cloud frame is associated with a data correction parameter, wherein the data correction parameter is pre-set by a user for the point cloud frame.
[0097] If the point cloud frame is associated with data correction parameters, a posture correction operation is performed on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data.
[0098] If the point cloud frame is not associated with a data correction parameter, a data labeling operation is performed on the point cloud frame based on the predicted labeling data.
[0099] In this embodiment, when the user is annotating key frames, for point cloud frames with more or more complex sampling points or closer to the sampling vehicle, the result of predicting the target object therein through at least one target key frame associated therewith may not be accurate enough, so data correction parameters can be set for it. For example, each point cloud frame is associated with a correction switch control. In response to the user's triggering operation on the correction switch control associated with the point cloud frame, data correction parameters can be set for the point cloud frame. And the data correction parameters can be added to the point cloud frame sequence.
[0100] Furthermore, after acquiring the point cloud frame sequence, for each point cloud frame, it can be determined whether the point cloud frame is associated with a data correction parameter. If the point cloud frame is associated with a data correction parameter, after performing a prediction operation on the target object in the point cloud frame based on at least one target key frame to obtain the predicted annotation data, a posture correction operation can be performed on the predicted annotation data according to a preset correction algorithm to obtain the target annotation data.
[0101] On the contrary, if the point cloud frame is not associated with data correction parameters, after a prediction operation is performed on the target object in the point cloud frame based on at least one target key frame to obtain predicted annotation data, a data annotation operation can be performed directly based on the predicted annotation data.
[0102] The point cloud data annotation method provided by the embodiment of the present disclosure sets data correction parameters for some point cloud frames during the annotation process, so that data correction operations can be performed on the point cloud frames based on the data correction parameters to further improve the accuracy of data annotation. In addition, the data annotation results can be made more in line with the actual needs of users.
[0103] Figure 5 A flowchart of a point cloud data annotation method provided by another embodiment of the present disclosure is provided. Based on any of the above embodiments, the predicted annotation data includes a predicted annotation box associated with the target object. Step 204 includes:
[0104] Step 501: for the predicted annotation box associated with the target object, determine the rotation angle data and longitudinal displacement data of the target object after it touches the ground based on a ground fitting algorithm.
[0105] Step 502: Adjust the position and posture of the predicted annotation box based on the rotation angle data and the longitudinal displacement data to obtain the target annotation data.
[0106] In this embodiment, the predicted annotation data includes a predicted annotation box associated with the target object. The predicted annotation box obtained based on the target key frame prediction may have an angle deviation. In order to improve the accuracy of data annotation, the predicted annotation box can also be corrected by a preset correction algorithm.
[0107] Optionally, for the predicted annotation frame associated with the target object, the rotation angle data and longitudinal displacement data of the target object after it touches the ground can be determined based on the ground fitting algorithm. Therefore, the predicted annotation frame can be adjusted in position according to the rotation angle data and the longitudinal displacement data to obtain the target annotation data. Then, the annotation operation of the point cloud frame can be accurately implemented based on the target annotation data.
[0108] Figure 6 A schematic diagram of the correction of the prediction annotation frame provided by the embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the predicted annotation frame 61 obtained based on the target key frame prediction may have an angle deviation, and in the predicted annotation frame 61, the point cloud 62 of the target vehicle is not touching the ground. The rotation angle data and longitudinal displacement data of the target object after touching the ground can be determined based on the ground fitting algorithm, and the position and posture of the predicted annotation frame can be adjusted based on the rotation angle data and the longitudinal displacement data to obtain the target annotation data 63. Therefore, in the target annotation data 63, the point cloud 62 of the target vehicle is in a state of touching the ground, which can more accurately represent the real position and posture of the target vehicle.
[0109] The point cloud data annotation method provided by the embodiment of the present disclosure determines the rotation angle data and longitudinal displacement data of the target object after it touches the ground based on the ground fitting algorithm for the predicted annotation box associated with the target object, so that the posture of the predicted annotation box can be corrected based on the rotation angle data and the longitudinal displacement data, so that the predicted annotation box associated with the target object can touch the ground, thereby improving the accuracy of data annotation.
[0110] Further, based on any of the above embodiments, step 501 includes:
[0111] For each sampling point in the predicted annotation box, the standard deviation of the vertical axis coordinates between the sampling point and a preset number of neighboring points associated with the sampling point is determined to obtain a density weight matrix.
[0112] A weighted covariance matrix is determined according to the position information of the sampling points in the predicted annotation box and the density weight matrix.
[0113] The normal vector, the centroid and the standard deviation corresponding to the predicted annotation box are calculated based on the weighted covariance matrix.
[0114] The plane equation is generated by fitting according to the normal vector, the centroid and the standard deviation.
[0115] The cross product of the original rotation vector associated with the predicted annotation box and the normal vector is calculated, and the cross product is normalized to obtain a rotation axis.
[0116] The rotation angle data corresponding to the predicted annotation box is determined based on the fitting plane equation and the rotation axis.
[0117] An angle adjustment operation is performed on the predicted annotation frame based on the rotation angle data to obtain an adjusted predicted annotation frame.
[0118] The center point coordinates of the adjusted predicted annotation frame are determined, and based on the vertical axis coordinates of the center point coordinates, the offset of the vertical axis coordinates relative to the ground plane is determined, and the offset is determined as the longitudinal displacement data.
[0119] In this embodiment, during the prediction process, the prediction annotation box associated with the target object can be determined separately. Since the point cloud frame is a collection of multiple three-dimensional coordinate points, the three-dimensional coordinate points can be obtained by discrete sampling of the target object or scene, so the prediction annotation box also includes multiple sampling points. Therefore, for each sampling point in the prediction annotation box, the standard deviation of the vertical axis coordinates between the sampling point and the preset number of neighboring points associated with the sampling point can be determined to obtain a density weight matrix. For example, each sampling point and N neighboring points adjacent to the sampling point can be determined. Among them, the number of neighboring points can be adjusted according to actual needs. The weighted covariance matrix is determined based on the position information of each sampling point in the prediction annotation box and the density weight matrix. Among them, the position information can be the three-dimensional coordinate information of the sampling point.
[0120] Furthermore, the normal vector, centroid and standard deviation corresponding to the predicted annotation box can be calculated based on the weighted covariance matrix. The normal vector, centroid and standard deviation corresponding to the predicted annotation box can be determined based on the weighted covariance matrix by principal component analysis (PCA). The plane equation is generated by fitting the normal vector, centroid and standard deviation.
[0121] Calculate the cross product of the original rotation vector and the normal vector associated with the predicted annotation box, normalize the cross product, and obtain the rotation axis. Determine the rotation angle data corresponding to the predicted annotation box based on the fitted plane equation and the rotation axis. Perform an angle adjustment operation on the predicted annotation box based on the rotation angle data to obtain an adjusted predicted annotation box. After the angle adjustment is performed based on the rotation angle data, the center point of the adjusted predicted annotation box changes. Therefore, the center point coordinates of the adjusted predicted annotation box can be determined, and based on the vertical axis coordinates of the center point coordinates, the offset of the vertical axis coordinates relative to the ground plane can be determined, and the offset is determined as the longitudinal displacement data. Therefore, after adjustment based on the longitudinal displacement data, an accurate annotation box after the target object is attached to the ground can be obtained.
[0122] Optionally, based on any of the foregoing embodiments, calculating the normal vector, the centroid, and the standard deviation corresponding to the predicted annotation box based on the weighted covariance matrix includes:
[0123] The normal vector, the centroid and the standard deviation corresponding to the predicted annotation box are determined based on the weighted covariance matrix by using the principal component analysis technique.
[0124] In this embodiment, after obtaining the weighted covariance matrix, the normal vector, centroid and standard deviation corresponding to the predicted annotation box can be determined based on the weighted covariance matrix by principal component analysis (PCA). The principal component analysis technology can use the idea of dimensionality reduction to transform multiple indicators into a few comprehensive indicators, so the normal vector, centroid and standard deviation corresponding to the predicted annotation box can be accurately calculated based on the weighted covariance matrix.
[0125] The point cloud data annotation method provided by the embodiment of the present disclosure can accurately determine the position and posture of the predicted annotation box after it is in contact with the ground by determining the rotation angle data and the longitudinal displacement data of the target object after it is in contact with the ground. Therefore, after adjusting the predicted annotation box based on the rotation angle data and the longitudinal displacement data, more accurate annotation data can be obtained, thereby improving the accuracy of data annotation.
[0126] Figure 7 A flowchart of a point cloud data annotation method provided by another embodiment of the present disclosure is provided. Based on any of the above embodiments, Figure 7 As shown, the point cloud frame sequence includes a single key frame. Step 202 includes:
[0127] Step 701: Determine the key frame as the target key frame.
[0128] Step 203 includes:
[0129] Step 702: perform a prediction operation on the target object in the point cloud frame based on the annotation data associated with the target key frame and the pose information of the acquisition vehicle that acquires the point cloud frame sequence to obtain the predicted annotation data associated with the target object.
[0130] In this embodiment, the point cloud frame sequence may include only one pre-annotated key frame. Therefore, the key frame may be determined as the current target key frame. Therefore, the target object in the point cloud frame may be predicted based on the annotation data associated with the target key frame and the position information of the acquisition vehicle that acquires the point cloud frame sequence, and the predicted annotation data associated with the target object may be obtained. Subsequently, the point cloud frame may be annotated based on the predicted annotation data to obtain the target point cloud frame.
[0131] For example, the moving speed and rotation angle of the target object can be predicted based on the pose information of the acquisition vehicle in the target key frame and the pose information of the acquisition vehicle in the current point cloud frame. Then, the pose of the target object in the point cloud frame can be predicted based on the time interval between the point cloud frame and the target key frame, as well as the moving speed and rotation angle, and prediction annotation data can be generated based on the prediction results. The prediction annotation data can be a three-dimensional prediction box of the target object.
[0132] The point cloud data annotation method provided by the embodiment of the present disclosure performs annotation operations based on the annotation information of the key frame and the position and posture information of the collection vehicle when only one key frame is included. Thus, the point cloud data annotation method can be accurately implemented in the case of only one key frame, so that the point cloud data annotation method can be applicable to more application scenarios.
[0133] Further, based on any of the above embodiments, after step 205, the method further includes:
[0134] Generate a labeling report based on the labeling process data associated with the target point cloud frame, wherein the labeling process data includes the labeling data of the target key frame associated with the point cloud frame, the target point cloud frame, the process data of the target point cloud frame during the labeling process, and the process data during the correction process.
[0135] The annotation report is stored in a preset storage path.
[0136] In this embodiment, for each point cloud frame, the pose of the target object is predicted based on at least one target key frame associated with the point cloud frame, and the labeling operation is completed to obtain the target point cloud frame. Then, a labeling report can be generated based on the labeling process data associated with the target point cloud frame, wherein the labeling process data includes the labeling data of the target key frame associated with the point cloud frame, the target point cloud frame, the process data of the target point cloud frame during the labeling process, and the process data of the correction process.
[0137] Furthermore, the annotation report may be stored in a preset storage path. For example, the annotation report may be submitted to a server, so that subsequent users can view the annotation report on the server for subsequent data analysis and algorithm optimization.
[0138] As an implementable method, after completing the storage of the annotation report, for the prediction scenario with exactly the same key frame parameters, the annotation report can be directly obtained, and the prediction results, annotation results, etc. can be returned based on the annotation report. There is no need to perform prediction operations and annotation operations again, thereby improving response efficiency.
[0139] The point cloud data annotation method provided by the embodiment of the present disclosure generates and stores an annotation report after completing the data annotation, so that data analysis and backtracking can be performed based on the annotation report later, and in the prediction scenario with exactly the same key frame parameters, the prediction results can be returned through the above-mentioned caching mechanism, thereby improving the system response efficiency.
[0140] Figure 8 A schematic diagram of the structure of a point cloud data annotation device provided in an embodiment of the present disclosure, such as Figure 8 As shown, the device includes: an acquisition module 81, a determination module 82, a prediction module 83, a correction module 84 and a labeling module 85. Among them, the acquisition module 81 is used to acquire a point cloud frame sequence, wherein the point cloud frame sequence includes multiple unlabeled point cloud frames and at least one key frame for labeling a target object, and the key frame is associated with labeling data. The determination module 82 is used to determine at least one target key frame that is spatially associated with the point cloud frame. The prediction module 83 is used to perform a prediction operation on the target object in the point cloud frame based on the labeling data associated with the at least one target key frame, and obtain the predicted labeling data associated with the target object. The correction module 84 is used to perform a posture correction operation on the predicted labeling data according to a preset data correction algorithm to obtain the target labeling data. The labeling module 85 is used to perform a data labeling operation on the point cloud frame based on the target labeling data to obtain the labeled target point cloud frame.
[0141] Further, on the basis of any of the above embodiments, wherein the point cloud frame sequence includes multiple key frames, and the number of the target key frames is two. The determination module includes: a generation unit, used to arrange the key frames in the point cloud frame sequence according to the time sequence of the key frames to obtain a key frame sequence. A determination unit, used to determine the first key frame and the last key frame in the key frame sequence as the at least one target key frame. Alternatively, determine the timing relationship between the point cloud frame and the key frames in the key frame sequence, and determine two key frames that are adjacent to the point cloud frame in timing as the at least one target key frame.
[0142] Further, based on any of the above embodiments, the number of the target key frames is at least two. The prediction module includes: a calculation unit, configured to perform interpolation calculation on at least two target key frames based on the annotation data associated with the target key frames to obtain interpolation data. A prediction unit, configured to perform a prediction operation on the pose of the target object in the point cloud frame according to the interpolation data to obtain the predicted annotation data.
[0143] Further, based on any of the above embodiments, the calculation unit includes: a conversion subunit, used to convert the pose information of the at least two target key frames to the same preset coordinate system to obtain the pose conversion results associated with the at least two target key frames. An interpolation subunit, used to perform position interpolation settlement and rotation angle interpolation calculation based on the pose conversion results and the interpolation factors associated with the point cloud frame to obtain the interpolation data, wherein the interpolation data includes position interpolation data and rotation angle interpolation data, and the interpolation factor is determined based on the relative position between the point cloud frame and the at least two target key frames.
[0144] Further, on the basis of any of the above embodiments, the prediction unit includes: a prediction subunit, used to predict the moving position and rotation angle of the target object in the point cloud frame based on the position interpolation data and the rotation angle interpolation data to obtain posture prediction information. A processing subunit, used to determine the predicted annotation data based on the posture prediction information and the size information associated with the target object, wherein the size information is determined based on the annotation data, or the size information is determined by identifying the target object for the point cloud frame.
[0145] Further, on the basis of any of the above embodiments, the correction module includes: a determination unit, used to determine whether the point cloud frame is associated with data correction parameters, wherein the data correction parameters are pre-set by the user for the point cloud frame. A correction unit, used to perform a posture correction operation on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data if the point cloud frame is associated with data correction parameters. A labeling unit, used to perform a data labeling operation on the point cloud frame based on the predicted annotation data if the point cloud frame is not associated with data correction parameters.
[0146] Further, based on any of the above embodiments, the predicted annotation data includes a predicted annotation box associated with the target object. The correction unit includes: a calculation subunit, for determining the rotation angle data and longitudinal displacement data of the target object after it touches the ground based on a ground fitting algorithm for the predicted annotation box associated with the target object. An adjustment subunit, for adjusting the position and posture of the predicted annotation box based on the rotation angle data and the longitudinal displacement data to obtain the target annotation data.
[0147] Further, on the basis of any of the above embodiments, the calculation subunit is used to: for each sampling point in the predicted annotation frame, determine the standard deviation of the vertical axis coordinates between the sampling point and a preset number of neighboring points associated with the sampling point, and obtain a density weight matrix. Determine a weighted covariance matrix according to the position information of the sampling points in the predicted annotation frame and the density weight matrix. Calculate the normal vector, centroid and standard deviation corresponding to the predicted annotation frame based on the weighted covariance matrix. Generate a plane equation by fitting according to the normal vector, centroid and standard deviation. Calculate the cross product of the original rotation vector associated with the predicted annotation frame and the normal vector, normalize the cross product and obtain the rotation axis. Determine the rotation angle data corresponding to the predicted annotation frame based on the fitted plane equation and the rotation axis. Perform an angle adjustment operation on the predicted annotation frame based on the rotation angle data to obtain an adjusted predicted annotation frame. Determine the center point coordinates of the adjusted predicted annotation frame, determine the offset of the vertical axis coordinates relative to the ground plane based on the vertical axis coordinates of the center point coordinates, and determine the offset as the longitudinal displacement data.
[0148] Further, based on any of the above embodiments, the calculation subunit is used to: determine the normal vector, center of mass and standard deviation corresponding to the predicted annotation box based on the weighted covariance matrix through principal component analysis technology.
[0149] Further, based on any of the above embodiments, the point cloud frame sequence includes a single key frame. The determination module includes: a determination unit, used to determine the key frame as the target key frame. The prediction module includes: a prediction unit, used to perform a prediction operation on the target object in the point cloud frame based on the annotation data associated with the target key frame and the posture information of the collection vehicle that collects the point cloud frame sequence, and obtain the predicted annotation data associated with the target object.
[0150] Further, based on any of the above embodiments, the device further includes: a generation module for generating a labeling report based on the labeling process data associated with the target point cloud frame, wherein the labeling process data includes the labeling data of the target key frame associated with the point cloud frame, the target point cloud frame, the process data of the target point cloud frame during the labeling process, and the process data during the correction process. A storage module for storing the labeling report to a preset storage path.
[0151] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0152] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.
[0153] Fig. 9 A schematic diagram of an electronic device provided for an embodiment of the present disclosure. The electronic device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0154] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0155] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0156] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the point cloud data annotation method. For example, in some embodiments, the point cloud data annotation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the point cloud data annotation method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the point cloud data annotation method in any other appropriate manner (e.g., by means of firmware).
[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0158] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0159] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0161] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0162] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0164] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A point cloud data annotation method, comprising: Acquire a point cloud frame sequence, wherein the point cloud frame sequence includes a plurality of unlabeled point cloud frames and at least one key frame for labeling a target object, wherein the key frame is associated with labeling data; For the point cloud frame, determining at least one target key frame spatially associated with the point cloud frame; Performing a prediction operation on a target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain predicted annotation data associated with the target object; Performing a posture correction operation on the predicted labeled data according to a preset data correction algorithm to obtain target labeled data; A data annotation operation is performed on the point cloud frame based on the target annotation data to obtain a labeled target point cloud frame.
2. The method according to claim 1, wherein: The point cloud frame sequence includes a plurality of key frames, and the number of the target key frames is two; and the determining of at least one target key frame spatially associated with the point cloud frame includes: For all key frames in the point cloud frame sequence, the key frames are arranged according to the time sequence of the key frames to obtain a key frame sequence; The first key frame and the last key frame in the key frame sequence are determined as the at least one target key frame; or, the timing relationship between the point cloud frame and the key frames in the key frame sequence is determined, and two key frames adjacent to the point cloud frame in timing are determined as the at least one target key frame.
3. The method according to claim 1, wherein: The number of the target key frames is at least two; the predicting operation on the target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain the predicted annotation data associated with the target object includes: Performing interpolation calculation on at least two target key frames based on the annotation data associated with the target key frames to obtain interpolation data; A prediction operation is performed on the position and posture of the target object in the point cloud frame according to the interpolation data to obtain the predicted annotation data.
4. The method according to claim 3, wherein the step of performing interpolation calculation on at least two target key frames based on the annotation data associated with the target key frames to obtain interpolation data comprises: Converting the pose information of the at least two target key frames to the same preset coordinate system to obtain pose conversion results associated with the at least two target key frames; Based on the pose conversion result and the interpolation factor associated with the point cloud frame, position interpolation settlement and rotation angle interpolation calculation are performed to obtain the interpolation data, wherein the interpolation data includes position interpolation data and rotation angle interpolation data, and the interpolation factor is determined based on the relative position between the point cloud frame and the at least two target key frames.
5. The method according to claim 3, wherein: The step of performing a prediction operation on the position and posture of the target object in the point cloud frame according to the interpolation data to obtain the predicted annotation data includes: Based on the position interpolation data and the rotation angle interpolation data, a prediction operation is performed on the moving position and the rotation angle of the target object in the point cloud frame to obtain posture prediction information; The predicted annotation data is determined based on the pose prediction information and size information associated with the target object, wherein the size information is determined based on the annotation data, or the size information is determined by performing target object recognition on the point cloud frame.
6. The method according to claim 1, wherein: The step of performing a posture correction operation on the predicted labeled data according to a preset data correction algorithm to obtain target labeled data includes: Determining whether the point cloud frame is associated with a data correction parameter, wherein the data correction parameter is pre-set by a user for the point cloud frame; If the point cloud frame is associated with a data correction parameter, a posture correction operation is performed on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data; If the point cloud frame is not associated with a data correction parameter, a data labeling operation is performed on the point cloud frame based on the predicted labeling data.
7. The method according to claim 1, wherein: The predicted annotation data includes a predicted annotation box associated with the target object; The step of performing a posture correction operation on the predicted labeled data according to a preset data correction algorithm to obtain target labeled data includes: For the predicted annotation frame associated with the target object, determine the rotation angle data and longitudinal displacement data of the target object after it touches the ground based on a ground fitting algorithm; The position and posture of the predicted annotation frame is adjusted based on the rotation angle data and the longitudinal displacement data to obtain the target annotation data.
8. The method according to claim 7, wherein: The method of determining the rotation angle data and the longitudinal displacement data of the target object after it touches the ground based on the ground fitting algorithm includes: For each sampling point in the predicted annotation box, determine the standard deviation of the vertical axis coordinates between the sampling point and a preset number of neighboring points associated with the sampling point to obtain a density weight matrix; Determining a weighted covariance matrix according to the position information of the sampling points in the predicted annotation box and the density weight matrix; Calculating the normal vector, the centroid and the standard deviation corresponding to the predicted annotation box based on the weighted covariance matrix; Generate a plane equation according to the normal vector, centroid and standard deviation fitting; Calculating the cross product of the original rotation vector associated with the predicted annotation box and the normal vector, and normalizing the cross product to obtain a rotation axis; Determine the rotation angle data corresponding to the predicted annotation frame based on the fitting plane equation and the rotation axis; Performing an angle adjustment operation on the predicted annotation frame based on the rotation angle data to obtain an adjusted predicted annotation frame; The center point coordinates of the adjusted predicted annotation frame are determined, and based on the vertical axis coordinates of the center point coordinates, the offset of the vertical axis coordinates relative to the ground plane is determined, and the offset is determined as the longitudinal displacement data.
9. The method according to claim 8, wherein: The calculating the normal vector, the centroid and the standard deviation corresponding to the predicted annotation box based on the weighted covariance matrix includes: The normal vector, the centroid and the standard deviation corresponding to the predicted annotation box are determined based on the weighted covariance matrix by using the principal component analysis technique.
10. The method according to claim 1, wherein the point cloud frame sequence includes a single key frame; The determining at least one target key frame spatially associated with the point cloud frame comprises: Determining the key frame as the target key frame; The step of performing a prediction operation on the target object in the point cloud frame based on the annotation data associated with the at least one target key frame to obtain the predicted annotation data associated with the target object includes: Based on the annotation data associated with the target key frame and the posture information of the acquisition vehicle that acquires the point cloud frame sequence, a prediction operation is performed on the target object in the point cloud frame to obtain the predicted annotation data associated with the target object.
11. The method according to any one of claims 1 to 10, wherein after performing a data annotation operation on the point cloud frame based on the target annotation data to obtain the annotated target point cloud frame, the method further comprises: Generate a labeling report according to the labeling process data associated with the target point cloud frame, wherein the labeling process data includes the labeling data of the target key frame associated with the point cloud frame, the target point cloud frame, the process data of the target point cloud frame during the labeling process, and the process data during the correction process; The annotation report is stored in a preset storage path.
12. A point cloud data annotation device, comprising: An acquisition module, used for acquiring a point cloud frame sequence, wherein the point cloud frame sequence includes a plurality of unlabeled point cloud frames and at least one key frame for labeling a target object, wherein the key frame is associated with labeling data; A determination module, configured to determine, for the point cloud frame, at least one target key frame spatially associated with the point cloud frame; A prediction module, configured to perform a prediction operation on a target object in the point cloud frame based on the annotation data associated with the at least one target key frame, to obtain predicted annotation data associated with the target object; A correction module, used to perform a posture correction operation on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data; The labeling module is used to perform a data labeling operation on the point cloud frame based on the target labeling data to obtain a labeled target point cloud frame.
13. The device according to claim 12, wherein: The point cloud frame sequence includes a plurality of key frames, and the number of the target key frames is two; the determination module includes: A generating unit, configured to arrange all key frames in the point cloud frame sequence according to the time sequence of the key frames to obtain a key frame sequence; A determination unit is used to determine the first key frame and the last key frame in the key frame sequence as the at least one target key frame; or, determine the timing relationship between the point cloud frame and the key frames in the key frame sequence, and determine two key frames that are adjacent to the point cloud frame in timing as the at least one target key frame.
14. The device according to claim 12, wherein: The number of the target key frames is at least two; the prediction module comprises: A calculation unit, configured to perform interpolation calculation on at least two target key frames based on the annotation data associated with the target key frames to obtain interpolation data; A prediction unit is used to perform a prediction operation on the position and posture of the target object in the point cloud frame according to the interpolation data to obtain the predicted annotation data.
15. The device according to claim 14, wherein the computing unit comprises: A conversion subunit, used to convert the pose information of the at least two target key frames into the same preset coordinate system to obtain a pose conversion result associated with the at least two target key frames; An interpolation subunit is used to perform position interpolation settlement and rotation angle interpolation calculation based on the posture conversion result and the interpolation factor associated with the point cloud frame to obtain the interpolation data, wherein the interpolation data includes position interpolation data and rotation angle interpolation data, and the interpolation factor is determined based on the relative position between the point cloud frame and the at least two target key frames.
16. The device according to claim 14, wherein: The prediction unit comprises: A prediction subunit, configured to perform a prediction operation on the moving position and the rotation angle of the target object in the point cloud frame based on the position interpolation data and the rotation angle interpolation data to obtain posture prediction information; A processing subunit is used to determine the predicted annotation data based on the posture prediction information and size information associated with the target object, wherein the size information is determined based on the annotation data, or the size information is determined by performing target object recognition on the point cloud frame.
17. The device according to claim 12, wherein: The correction module comprises: A determination unit, configured to determine whether the point cloud frame is associated with a data correction parameter, wherein the data correction parameter is pre-set by a user for the point cloud frame; A correction unit, configured to perform a posture correction operation on the predicted annotation data according to a preset data correction algorithm to obtain target annotation data if the point cloud frame is associated with a data correction parameter; A labeling unit is used to perform a data labeling operation on the point cloud frame based on the predicted labeling data if the point cloud frame is not associated with a data correction parameter.
18. The device according to claim 12, wherein: The predicted annotation data includes a predicted annotation box associated with the target object; The correction unit comprises: A calculation subunit, for determining the rotation angle data and longitudinal displacement data of the target object after it touches the ground based on a ground fitting algorithm for the predicted annotation box associated with the target object; The adjustment subunit is used to adjust the position and posture of the predicted annotation box based on the rotation angle data and the longitudinal displacement data to obtain the target annotation data.
19. The device according to claim 18, wherein: The computing subunit is used for: For each sampling point in the predicted annotation box, determine the standard deviation of the vertical axis coordinates between the sampling point and a preset number of neighboring points associated with the sampling point to obtain a density weight matrix; Determining a weighted covariance matrix according to the position information of the sampling points in the predicted annotation box and the density weight matrix; Calculating the normal vector, the centroid and the standard deviation corresponding to the predicted annotation box based on the weighted covariance matrix; Generate a plane equation according to the normal vector, centroid and standard deviation fitting; Calculating the cross product of the original rotation vector associated with the predicted annotation box and the normal vector, and normalizing the cross product to obtain a rotation axis; Determine the rotation angle data corresponding to the predicted annotation frame based on the fitting plane equation and the rotation axis; Performing an angle adjustment operation on the predicted annotation frame based on the rotation angle data to obtain an adjusted predicted annotation frame; The center point coordinates of the adjusted predicted annotation frame are determined, and based on the vertical axis coordinates of the center point coordinates, the offset of the vertical axis coordinates relative to the ground plane is determined, and the offset is determined as the longitudinal displacement data.
20. The device according to claim 19, wherein The computing subunit is used for: The normal vector, the centroid and the standard deviation corresponding to the predicted annotation box are determined based on the weighted covariance matrix by using the principal component analysis technique.
21. The apparatus according to claim 12, wherein the point cloud frame sequence comprises a single key frame; The determination module comprises: A determining unit, configured to determine the key frame as the target key frame; The prediction module comprises: The prediction unit is used to perform a prediction operation on the target object in the point cloud frame based on the annotation data associated with the target key frame and the posture information of the collection vehicle that collects the point cloud frame sequence, so as to obtain the predicted annotation data associated with the target object.
22. The device according to any one of claims 12 to 21, further comprising: A generating module, configured to generate a labeling report according to the labeling process data associated with the target point cloud frame, wherein the labeling process data includes the labeling data of the target key frame associated with the point cloud frame, the target point cloud frame, the process data of the target point cloud frame during the labeling process, and the process data during the correction process; The storage module is used to store the annotation report in a preset storage path.
23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.
24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.
25. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.
Citation Information
Cited By
Cross-laser projection method and device of 3D target, electronic equipment and storage medium
CN119805490A
Dynamic and static target distinguishing and marking method and system for point cloud data
CN120031970A
A method and system for distinguishing and labeling moving and static targets in point cloud data
CN120031970B
Three-dimensional data intelligent labeling method and system based on key frame
CN121527763A
