Point cloud labeling method and device, electronic equipment and storage medium

Abstract

The invention provides a point cloud labeling method and device, electronic equipment and a storage medium, and relates to the technical field of automatic driving and unmanned vehicles. The method comprises the steps of obtaining to-be-labeled point cloud data, wherein the to-be-labeled point cloud data comprises a labeling target; based on the to-be-labeled point cloud data, a pre-selection box corresponding to the labeled target is determined, and the pre-selection box is used for showing the boundary of the labeled target; predicting a motion track of the pre-selected box, and generating tracking information corresponding to the labeled target based on the motion track; and on the basis of the pre-selected box corresponding to the labeling target and the tracking information, generating the point cloud labeling data corresponding to the labeling target so as to realize automatic labeling of the point cloud data and improve the accuracy and robustness of the point cloud labeling data.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of autonomous driving and unmanned vehicles, and in particular to a point cloud annotation method and device, an electronic device, and a storage medium. Background Art

[0002] Deep learning-based perception methods are widely used in the field of unmanned driving to achieve accurate perception of the vehicle's surrounding environment. In the process of training, optimizing and testing such perception methods, a large amount of point cloud annotation data is required. However, point cloud annotation data is usually obtained through manual annotation, which is inefficient. Therefore, an automatic point cloud annotation method is urgently needed. Summary of the invention

[0003] In view of this, the present disclosure provides a point cloud annotation method and device, an electronic device and a storage medium.

[0004] In a first aspect, a point cloud annotation method is provided, comprising: acquiring point cloud data to be annotated, the point cloud data to be annotated including an annotation target; determining a pre-selected box corresponding to the annotation target based on the point cloud data to be annotated, the pre-selected box being used to show a boundary of the annotation target; predicting a motion trajectory of the pre-selected box, and generating tracking information corresponding to the annotation target based on the motion trajectory; and generating point cloud annotation data corresponding to the annotation target based on the pre-selected box corresponding to the annotation target and the tracking information.

[0005] In combination with the first aspect, in certain implementation methods of the first aspect, point cloud annotation data corresponding to the annotation target is generated based on a pre-selected box corresponding to the annotation target and tracking information, including: based on the pre-selected box, cropping the point cloud data to be annotated to obtain target point cloud data; based on the target point cloud data, optimizing the attributes of the pre-selected box using an attribute optimization model to obtain a target pre-selected box; based on the tracking information and the target pre-selected box, obtaining point cloud annotation data.

[0006] In combination with the first aspect, in certain implementations of the first aspect, based on a pre-selected box, the point cloud data to be labeled is cropped to obtain target point cloud data, including: dilating the pre-selected box; and cropping at least part of the point cloud data to be labeled within the pre-selected box after the dilation to obtain the target point cloud data.

[0007] In combination with the first aspect, in some implementations of the first aspect, the attribute optimization model includes a geometric attribute optimization model and / or a position attribute optimization model, and based on the target point cloud data, the attribute optimization model is used to optimize the attributes of the pre-selected box to obtain the target pre-selected box, including: using the geometric attribute optimization model to perform point-to-surface feature encoding on the target point cloud data, and optimizing the geometric attributes of the pre-selected box based on the point-to-surface feature encoding result of the target point cloud data to obtain the target pre-selected box; and / or using the position attribute optimization model to perform point-to-corner feature encoding on the target point cloud data, and optimizing the position attributes of the pre-selected box based on the point-to-corner feature encoding result of the target point cloud data to obtain the target pre-selected box. Preferably, the geometric attribute optimization model and / or the position attribute optimization model are obtained by training based on the self-attention mechanism.

[0008] In combination with the first aspect, in certain implementations of the first aspect, after optimizing the attributes of the pre-selection box based on the target point cloud data using the attribute optimization model to obtain the target pre-selection box, it also includes: determining the confidence information of the target pre-selection box based on the confidence evaluation model, wherein the confidence information includes the category score and the intersection-over-union (IOU) score of the target pre-selection box; obtaining point cloud annotation data based on the tracking information and the target pre-selection box, including: determining the point cloud annotation data based on the tracking information, as well as the geometric attributes, position attributes and confidence information of the target pre-selection box.

[0009] In combination with the first aspect, in some implementations of the first aspect, there are multiple labeled targets, multiple pre-selected boxes, and there is a corresponding relationship between the labeled targets and the pre-selected boxes; predicting the motion trajectory of the pre-selected boxes, and generating tracking information corresponding to the labeled targets based on the motion trajectory, including: using a trajectory tracking model to determine the motion trajectory of each of the multiple pre-selected boxes; based on the motion trajectory of each of the multiple pre-selected boxes, filtering invalid pre-selected boxes in the multiple pre-selected boxes to obtain multiple valid pre-selected boxes; based on the motion trajectory of each of the multiple valid pre-selected boxes, respectively determining the tracking information corresponding to each labeled target. Preferably, the trajectory tracking model includes a Kalman filter model.

[0010] In combination with the first aspect, in certain implementations of the first aspect, based on the motion trajectories of each of the multiple valid pre-selection boxes, the tracking information corresponding to each labeled target is determined respectively, including: for the motion trajectories of each of the multiple valid pre-selection boxes, the motion trajectories that do not belong to the same labeled target are split; the motion trajectories belonging to the same labeled target are merged to obtain the tracking information corresponding to each labeled target respectively.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the point cloud data to be labeled is collected by a radar carried by a vehicle, and the radar carried by the vehicle includes a main radar and a blind radar, and there is an overlapping area between the field of view of the main radar and the field of view of the blind radar. Obtaining the point cloud data to be labeled includes: obtaining the point cloud data collected by the main radar and the point cloud data collected by the blind radar; based on the overlapping area, densely combining at least part of the point cloud data collected by the main radar and at least part of the point cloud data collected by the blind radar to obtain the point cloud data to be labeled. Preferably, the main radar includes a high-beam laser radar, and the blind radar includes a low-beam laser radar. Preferably, there are multiple blind radars, and the multiple blind radars are respectively located at the left front, right front and rear of the vehicle.

[0012] In a second aspect, a point cloud annotation device is provided, including: an acquisition module, configured to acquire point cloud data to be annotated, the point cloud data to be annotated including an annotation target; a determination module, configured to determine a pre-selection box corresponding to the annotation target based on the point cloud data to be annotated, the pre-selection box is used to show the boundary of the annotation target; a first generation module, configured to predict the motion trajectory of the pre-selection box, and generate tracking information corresponding to the annotation target based on the motion trajectory; a second generation module, configured to generate point cloud annotation data corresponding to the annotation target based on the pre-selection box corresponding to the annotation target and the tracking information.

[0013] In a third aspect, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the point cloud annotation method provided in the first aspect by executing the executable instructions.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the point cloud annotation method provided in the first aspect is implemented.

[0015] In the method provided by the present disclosure, a pre-selected box corresponding to the annotation target is determined in the point cloud data to be annotated, and the position information and geometric information of the annotation target are determined based on the pre-selected box; the tracking information of the annotation target is obtained by predicting the motion trajectory of the pre-selected box. In this way, the point cloud annotation data corresponding to the annotation target is obtained, and automatic annotation of the point cloud data is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a schematic diagram of a scenario applicable to an embodiment of the present disclosure.

[0017] Figure 2 The figure is a flow chart of a point cloud annotation method provided in one embodiment of the present disclosure.

[0018] Figure 3The figure is a flow chart of steps for generating point cloud annotation data corresponding to an annotation target based on a pre-selected box and tracking information corresponding to the annotation target provided by an embodiment of the present disclosure.

[0019] Figure 4 The figure is a flow chart of the steps of cropping the point cloud data to be annotated based on the pre-selected box to obtain the target point cloud data provided by an embodiment of the present disclosure.

[0020] Figure 5 The figure shows a flow chart of the steps of optimizing the attributes of a pre-selection box based on target point cloud data using an attribute optimization model to obtain a target pre-selection box, provided by an embodiment of the present disclosure.

[0021] Figure 6 The figure is a flowchart of the steps of predicting the motion trajectory of a pre-selected box and generating tracking information corresponding to a marked target based on the motion trajectory provided by an embodiment of the present disclosure.

[0022] Figure 7 The figure is a flowchart of steps for respectively determining tracking information corresponding to each marked target based on the motion trajectories of multiple valid pre-selected boxes according to an embodiment of the present disclosure.

[0023] Figure 8 Shown is a schematic diagram of a vehicle and a radar carried by the vehicle provided in an embodiment of the present disclosure.

[0024] Fig. 9 Shown is a schematic structural diagram of a point cloud annotation device provided in one embodiment of the present disclosure.

[0025] Fig.10 Shown is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0027] Application Overview

[0028] In the field of autonomous driving, point cloud annotation data is crucial for training and improving the perception capabilities of autonomous driving systems. For example, point cloud annotation data can help autonomous driving systems accurately perceive the environment around the vehicle, including obstacles, pedestrians, road signs, etc., thereby achieving precise positioning and navigation.

[0029] The annotation of point cloud data usually relies on manpower. The current mainstream annotation method is that annotators use relevant tools to complete tasks such as classification, framing, annotation and marking on the data according to the annotation requirements, which has low annotation efficiency. In addition, annotators cannot accurately mark the movement state and trajectory of the annotation target over time.

[0030] One of the related technologies is to combine point cloud data with two-dimensional images to improve the annotation accuracy, but this method is relatively complex and costly. In addition, this method has high requirements for the image quality of two-dimensional images and is not suitable for bad weather such as rain and snow. In addition, dust and particulate matter in the environment will affect the clarity and recognition of two-dimensional images, making this method unsuitable for scenes with harsh environments such as mining areas.

[0031] In view of the above technical problems, the present disclosure provides a point cloud annotation method, which obtains point cloud data to be annotated, wherein the point cloud data to be annotated includes an annotation target; based on the point cloud data to be annotated, a pre-selected box corresponding to the annotation target is determined, and the pre-selected box is used to show the boundary of the annotation target; the motion trajectory of the pre-selected box is predicted, and tracking information corresponding to the annotation target is generated based on the motion trajectory; based on the pre-selected box corresponding to the annotation target and the tracking information, point cloud annotation data corresponding to the annotation target is generated, so as to realize automatic annotation of the point cloud data to be annotated, and improve the accuracy and robustness of the point cloud annotation data.

[0032] Example scenarios

[0033] Figure 1 FIG. 1 is a schematic diagram of a scenario applicable to an embodiment of the present disclosure. Figure 1 As shown, the scenario includes a vehicle 10 and a server 20. The vehicle 10 is connected to the server 20 in communication, for example, via a wired or wireless network connection.

[0034] The vehicle 10 is equipped with a radar device, which is used to collect three-dimensional point cloud data around the vehicle 10 , including other surrounding vehicles, pedestrians, obstacles, etc. Then, the vehicle 10 sends the three-dimensional point cloud data to the server 20 .

[0035] The server 20 annotates the three-dimensional point cloud data sent by the vehicle 10 based on the point cloud annotation method provided in the embodiment of the present disclosure to obtain point cloud annotation data with high accuracy and robustness. The server 20 can be integrated in the vehicle 10 or can be an independent server. For example, the server can be an intercommunication server or a background server between multiple heterogeneous systems, an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0036] Exemplary Methods

[0037] Combine the following Figures 2 to 7 The point cloud annotation method provided by the embodiment of the present disclosure is illustrated by way of example.

[0038] Figure 2 FIG. 1 is a flow chart of a point cloud annotation method provided by an embodiment of the present disclosure. Figure 2 As shown, the point cloud annotation method provided in the embodiment of the present disclosure includes the following steps.

[0039] S210, obtaining point cloud data to be annotated.

[0040] Point cloud data is a data structure used to represent objects or scenes in three-dimensional space. It usually contains multiple three-dimensional coordinate points, which are obtained by discrete sampling of the surface of the actual object or scene. Therefore, point cloud data can be regarded as a discrete representation of the scene surface in a given coordinate system. The point cloud data to be annotated is the original point cloud data without annotation.

[0041] Point cloud data can be obtained through laser scanning, photogrammetry, drone aerial photography, etc. Specifically, in the field of autonomous driving, the point cloud data to be annotated can be directly collected by the radar device on the vehicle. By emitting laser beams and recording the reflected signals, the radar can efficiently obtain the three-dimensional coordinate information of other vehicles, pedestrians, obstacles and other objects around the vehicle, and generate point cloud data accordingly.

[0042] The point cloud data to be annotated includes annotation targets, which refer to objects that need to be identified and marked during the point cloud annotation process. Depending on the annotation task, the annotation target can be a vehicle, pedestrian, traffic sign, building, etc. Different types of annotation targets correspond to different category labels.

[0043] In the point cloud data to be annotated, the number of the annotated objects may be one or more, and the types of the annotated objects may be the same or partially different, and the present disclosure does not impose any specific limitation on this.

[0044] S220: Determine a pre-selected box corresponding to the annotation target based on the point cloud data to be annotated.

[0045] In this step, the target detection model can be used to identify the labeled target in the point cloud data to be labeled, and generate a pre-selected box corresponding to the labeled target. The pre-selected box is used to show the boundary of the labeled target in the point cloud data to be labeled. In addition, the target detection model can also output the category information, confidence information, etc. corresponding to the labeled target.

[0046] Optionally, the target detection model can be any target detection method. For different target detection methods, a unified post-detection processing algorithm can be set accordingly to ensure the consistency of the output results of the target detection model, so as to ensure that the output results can be adapted to the subsequent processing process.

[0047] Optionally, in order to ensure a high recall rate, relevant parameters of the target detection model may be adjusted so that the target detection model outputs more pre-selected boxes during the detection process to reduce missed detections.

[0048] S230, predicting a motion trajectory of the pre-selected box, and generating tracking information corresponding to the marked target based on the motion trajectory.

[0049] The trajectory tracking model is used to predict the motion trajectory of the pre-selected box corresponding to the labeled target, and the tracking information corresponding to the labeled target is generated based on the predicted motion trajectory. The tracking information represents the motion state and position of the labeled target over time.

[0050] Optionally, the trajectory tracking model may be any tracking algorithm. In this step, the pre-selected box corresponding to the marked target is used as input information of the trajectory tracking model, so that the trajectory tracking model generates a motion trajectory corresponding to the marked target.

[0051] S240, generating point cloud annotation data corresponding to the annotation target based on the pre-selected box corresponding to the annotation target and the tracking information.

[0052] Point cloud annotation data includes the location information, geometric information and tracking information corresponding to the annotation target.

[0053] Based on the above steps, the pre-selected box corresponding to the labeled target can show the position and approximate size of the labeled target. Therefore, the position information and geometric information corresponding to the labeled target can be generated based on the pre-selected box. Combined with the tracking information determined based on the motion trajectory of the pre-selected box, the point cloud annotation data corresponding to the labeled target can be obtained. The position information corresponding to the target indicates the position of the labeled target in three-dimensional space, which can be represented by the coordinates of the center point of the pre-selected box. The geometric information corresponding to the target indicates the approximate size of the labeled target, which can be represented by the size and direction of the pre-selected box.

[0054] In addition, point cloud annotation data can also include category information, confidence information, etc. corresponding to the target.

[0055] In the disclosed embodiment, a preselected box corresponding to the annotation target is determined in the point cloud data to be annotated, and the position information and geometric information of the annotation target are determined based on the preselected box; the tracking information of the annotation target is obtained by predicting the motion trajectory of the preselected box. In this way, the point cloud annotation data corresponding to the annotation target is obtained, and automatic annotation of the point cloud data is realized.

[0056] The following will continue to introduce the specific implementation method of generating point cloud annotation data corresponding to the annotation target.

[0057] Figure 3 FIG. 1 is a flow chart of steps for generating point cloud annotation data corresponding to an annotation target based on a pre-selected box and tracking information corresponding to the annotation target provided by an embodiment of the present disclosure. Figure 3 As shown, the steps of generating point cloud annotation data corresponding to the annotation target based on the pre-selected box and tracking information corresponding to the annotation target provided by the embodiment of the present disclosure include the following steps.

[0058] S241, based on the pre-selected box, crop the point cloud data to be marked to obtain target point cloud data.

[0059] In order to improve the accuracy of the point cloud annotation data corresponding to the annotation target, the pre-selection box can be optimized. First, the point cloud data to be annotated is cropped based on the pre-selection box. The part of the point cloud data to be annotated in the pre-selection box is used as the target point cloud data.

[0060] Since the preselected box can define the approximate range of the labeled target, most of the point cloud data corresponding to the labeled target should be within the preselected box, that is, most of the point cloud data corresponding to the labeled target should be included in the target point cloud data. Accordingly, the target point cloud data includes the point cloud data corresponding to the labeled target and some environmental information around the labeled target.

[0061] Through the target point cloud data, the information irrelevant to the target can be effectively deleted from the point cloud data to be annotated, ensuring that only the points directly related to the target are analyzed. At the same time, the computational cost can also be reduced.

[0062] S242, based on the target point cloud data, using the attribute optimization model to optimize the attributes of the pre-selected box to obtain the target pre-selected box.

[0063] The target point cloud data is used as input information, and the attribute optimization model is used to learn the interaction between multiple point cloud data belonging to the annotated target in the target point cloud data, as well as the interaction between the annotated target and the surrounding environment, so as to optimize the attributes of the pre-selected box. Among them, the attribute optimization model can optimize the position attributes and / or geometric attributes of the pre-selected box.

[0064] The target pre-selection box is determined based on the optimized position attributes and / or geometric attributes. Compared with the original pre-selection box, the target pre-selection box can more accurately reflect the position information and / or geometric information of the marked target.

[0065] S243, obtaining point cloud annotation data based on the tracking information and the target pre-selection box.

[0066] The position information and geometric information corresponding to the labeled target are determined based on the target pre-selection box, and the point cloud annotation data corresponding to the labeled target is obtained by combining the tracking information determined based on the motion trajectory of the pre-selection box.

[0067] In this embodiment, by cropping the point cloud data to be annotated, the attribute optimization model is helped to better understand the local environment and context information of the annotated target, so that the position attributes and / or geometric attributes of the target pre-selection box are more accurate, thereby improving the accuracy of the point cloud annotation data.

[0068] Figure 4 FIG. 1 is a flow chart of the steps of clipping the point cloud data to be annotated based on the pre-selected box to obtain the target point cloud data according to an embodiment of the present disclosure. Figure 4 As shown, the embodiment of the present disclosure provides a step of cropping the point cloud data to be annotated based on a pre-selected box to obtain target point cloud data, which includes the following steps.

[0069] S2411, dilate the pre-selected box.

[0070] In this step, the size of the pre-selected box can be changed based on the expansion coefficient, and the size of the pre-selected box can be enlarged so that the pre-selected box after the expansion process can better wrap the edge of the labeled object. At the same time, the environmental information around the labeled object in the pre-selected box can also be enriched. In the subsequent processing process, the attribute optimization model can understand the posture and shape of the labeled object based on the environmental information, as well as the relative position of the labeled object and other objects in the surrounding environment.

[0071] The expansion coefficient may be preset or determined based on the sparsity of the point cloud data, and the present disclosure does not impose any specific limitation on this.

[0072] S2412, cropping at least a portion of the point cloud data to be labeled within the pre-selected box after the expansion process to obtain target point cloud data.

[0073] Next, at least a portion of the point cloud data to be labeled in the pre-selected frame after the expansion process is cropped, and the portion of the point cloud data to be labeled in the pre-selected frame after the expansion process is used as the target point cloud data.

[0074] Through the above steps, the pre-selected box is expanded, and the target point cloud data is obtained by clipping the point cloud data in the pre-selected box after the expansion. Based on the target point cloud data, the extracted local features can more accurately reflect the contour features of the annotated target and the contextual relationship between the annotated target and the surrounding environment, which helps to improve the accuracy of the target pre-selected box and thus improve the accuracy of the point cloud annotation data.

[0075] Figure 5The figure shows a flow chart of the steps of optimizing the attributes of the pre-selected box based on the target point cloud data using the attribute optimization model to obtain the target pre-selected box according to an embodiment of the present disclosure. In the above embodiment, the attribute optimization model can optimize the position attributes and / or geometric attributes of the pre-selected box. Specifically, the attribute optimization model includes a geometric attribute optimization model and / or a position attribute optimization model. The geometric attribute optimization model is used to optimize the geometric attributes of the pre-selected box, and the position attribute optimization model is used to optimize the position attributes of the pre-selected box. Figure 5 Specific implementation methods for optimizing the attributes of the pre-selected box based on a geometric attribute optimization model and / or a position attribute optimization model are introduced respectively.

[0076] like Figure 5 As shown, the embodiment of the present disclosure provides a method for optimizing the attributes of a pre-selection box based on target point cloud data using an attribute optimization model to obtain a target pre-selection box, which includes the following steps.

[0077] S2421, using the geometric attribute optimization model, performing point-to-surface feature encoding on the target point cloud data, and optimizing the geometric attributes of the pre-selection box based on the point-to-surface feature encoding result of the target point cloud data to obtain the target pre-selection box.

[0078] For any point cloud data in the target point cloud data, feature encoding is performed on the point cloud data and the surface of the pre-selected box to obtain the spatial position relationship between the point cloud data and the surface of the pre-selected box. In this way, a point-to-surface feature encoding result between the target point cloud data and the surface of the pre-selected box can be obtained, and the point-to-surface feature encoding result includes the spatial position relationship between any point in the target point cloud data and the surface of the pre-selected box.

[0079] Next, the geometric property optimization model can learn the spatial position relationship between any point in the target point cloud data and the surface of the pre-selected box, so that it can understand the geometric shape of the labeled target in the pre-selected box, and then adjust and optimize the geometric properties of the pre-selected box based on the geometric shape of the labeled target, so that the target pre-selected box can better fit the outer surface of the labeled target.

[0080] Optionally, the point-to-surface feature encoding may include geometric relationship information such as distance, angle, normal direction, etc. between the point cloud data and the surface of the pre-selected box, and the present disclosure does not impose specific limitations on this.

[0081] S2422, using the position attribute optimization model, performing point-to-corner feature encoding on the target point cloud data, and optimizing the position attribute of the pre-selection box based on the point-to-corner feature encoding result of the target point cloud data to obtain the target pre-selection box.

[0082] Similarly, for any point cloud data in the target point cloud data, feature encoding is performed on the point cloud data and the corner points of the pre-selected box to obtain the spatial position relationship between the point cloud data and the corner points of the pre-selected box. In this way, a point-to-corner feature encoding result between the target point cloud data and the corner points of the pre-selected box can be obtained, and the point-to-corner feature encoding result includes the spatial position relationship between any point in the target point cloud data and the corner points of the pre-selected box.

[0083] Next, the position attribute optimization model can learn the spatial position relationship between any point in the target point cloud data and the corner points of the pre-selected box, so that it can understand the distance relationship between the labeled target in the pre-selected box and the corner points of the pre-selected box, and then adjust and optimize the position attributes of the pre-selected box based on the distance relationship between the labeled target and the corner points of the pre-selected box, so that the position attributes of the target pre-selected box are closer to the actual position of the labeled target.

[0084] Optionally, the point-to-corner feature encoding may include the distance, angle or other geometric relationship information between the point cloud data and the corner points of the pre-selected box, and the present disclosure does not impose any specific limitation on this.

[0085] Optionally, the geometric attribute optimization model and / or the position attribute optimization model are trained based on a self-attention mechanism. The self-attention mechanism can associate the feature codes of multiple point cloud data and establish a global dependency relationship, so that the geometric attribute optimization model and / or the position attribute optimization model can capture the dependency relationship between any two point cloud data. It enables it to pay more attention to and learn the position and shape of the annotated target itself, thereby obtaining a more accurate optimization result.

[0086] Based on the above steps, the pre-selection box can be optimized through the geometric attribute optimization model and / or the position attribute optimization model, so that the optimized target pre-selection box can more accurately reflect the position information and geometric information of the labeled target, thereby improving the accuracy of the point cloud annotation data.

[0087] After obtaining the target pre-selected box, the confidence information corresponding to the target pre-selected box can also be recalculated. Specifically, after the step of obtaining the target pre-selected box by optimizing the attributes of the pre-selected box using the attribute optimization model based on the target point cloud data, the embodiment of the present disclosure also includes: determining the confidence information of the target pre-selected box based on the confidence evaluation model. The confidence information includes the category score and the intersection-over-union (IOU) score of the target pre-selected box.

[0088] The category score is the model's probability estimate of the category to which the labeled object belongs. The higher the category score, the higher the accuracy of the predicted category of the labeled object. The IOU score is an indicator of the degree of overlap between the target pre-selected box and the true box. The higher the IOU score, the more accurate the target pre-selected box is.

[0089] After determining the category score and IOU score of the target pre-selected box, the two can be combined to determine the confidence information of the target pre-selected box.

[0090] After determining the confidence information of the target pre-selection box, the confidence information can be used as part of the point cloud annotation data corresponding to the annotated target. Specifically, the step of obtaining point cloud annotation data based on tracking information and the target pre-selection box in the embodiment of the present disclosure includes: determining the point cloud annotation data based on the tracking information, and the geometric attributes, position attributes and confidence information of the target pre-selection box.

[0091] At this time, the position information and geometric information corresponding to the labeled target can be determined based on the geometric attributes and position attributes of the target pre-selection box, and the point cloud annotation data corresponding to the labeled target can be obtained by combining the tracking information and confidence information.

[0092] Based on the confidence information, the annotation quality of the point cloud annotation data can be directly judged. For point cloud annotation data with higher quality, it can be directly applied to the training, optimization and testing of the perception model. For point cloud annotation data with lower quality, it can be used as pre-annotated data and manually annotated again to improve the annotation efficiency and data flow efficiency.

[0093] The following will continue to introduce the specific implementation method of determining the tracking information corresponding to the labeled target.

[0094] Figure 6 FIG. 1 is a flow chart of the steps of predicting the motion trajectory of a pre-selected box and generating tracking information corresponding to a marked target based on the motion trajectory provided by an embodiment of the present disclosure. Figure 6 As shown, the steps of predicting the motion trajectory of the pre-selected box and generating tracking information corresponding to the marked target based on the motion trajectory provided by the embodiment of the present disclosure include the following steps.

[0095] S231, using a trajectory tracking model to determine the motion trajectories of the plurality of pre-selected boxes.

[0096] The point cloud data to be annotated includes a plurality of annotated targets. On this basis, the number of pre-selected boxes determined based on the point cloud data to be annotated is multiple, and there is a corresponding relationship between the annotated targets and the pre-selected boxes.

[0097] In this embodiment, the trajectory tracking model is used to predict the motion trajectories of the multiple pre-selected boxes. Optionally, the trajectory tracking model includes a Kalman filter model, which is an efficient autoregressive filter that can estimate the motion state of the labeled target in noisy measurement data. Moreover, it can recursively update the state estimate to achieve real-time and efficient tracking of the labeled target. Therefore, the Kalman filter model is suitable for scenarios where there are many uncertain factors and the motion state of the labeled target is relatively single, such as predicting the motion trajectory of a mine car in a mining scenario.

[0098] S232, filtering invalid pre-selected boxes among the multiple pre-selected boxes based on the respective motion trajectories of the multiple pre-selected boxes to obtain multiple valid pre-selected boxes.

[0099] In the above embodiment, in order to ensure a high recall rate, more pre-selected boxes are output in the process of determining the pre-selected boxes, and misidentification may occur in this process. Therefore, in this step, unreasonable pre-selected boxes can be filtered out based on the movement trajectories of the pre-selected boxes, thereby improving the reliability of the pre-selected boxes.

[0100] Specifically, based on the motion trajectories of the respective pre-selected boxes, it can be determined whether the motion trajectory is a reasonable motion trajectory. Exemplarily, it can be determined whether the motion trajectory of the pre-selected box conforms to the actual physical law or scene logic, for example, whether the motion trajectory has unnatural movement, whether the motion speed is abnormal, whether it is a non-drivable area during movement, etc. Alternatively, historical data and statistical models can also be used to evaluate whether the motion trajectory of the marked target meets expectations. If the motion trajectory is significantly different from the motion trajectory in most similar situations, it may be considered unreasonable.

[0101] Based on the rationality of the motion trajectories of the multiple pre-selected boxes, invalid pre-selected boxes are filtered out from the multiple pre-selected boxes, and the remaining pre-selected boxes are used as valid pre-selected boxes.

[0102] S233: Based on the motion trajectories of the multiple valid pre-selected boxes, respectively determine the tracking information corresponding to each marked target.

[0103] After filtering out invalid pre-selected boxes, the tracking information corresponding to each marked target can be determined based on the valid pre-selected boxes and their respective motion trajectories.

[0104] In the disclosed embodiment, a method for filtering unreasonable pre-selected boxes among multiple pre-selected boxes based on their respective motion trajectories is introduced. This ensures a high recall rate and avoids the phenomenon of misidentification of pre-selected boxes, thereby improving the robustness of point cloud annotation data. In addition, in this process, the tracking information corresponding to the annotation target can also be determined at the same time, thereby improving the overall efficiency of the annotation process.

[0105] After determining the motion trajectories of the multiple valid pre-selected boxes, the multiple motion trajectories may be integrated to improve the accuracy of the tracking information. The specific implementation method is as follows.

[0106] Figure 7 FIG. 1 is a flow chart of steps for determining tracking information corresponding to each marked target based on the motion trajectories of multiple valid pre-selected boxes according to an embodiment of the present disclosure. Figure 7 As shown, the steps of determining the tracking information corresponding to each marked target based on the motion trajectories of multiple valid pre-selected boxes provided in the embodiment of the present disclosure include the following steps.

[0107] S2331, for the motion trajectories of the multiple valid pre-selected boxes, split the motion trajectories that do not belong to the same marked target.

[0108] Due to the interaction and occlusion between the marked objects, or the limitations of the radar device, the motion trajectories of multiple valid pre-selected boxes may intersect or even overlap.

[0109] In this case, it is possible to determine whether the motion trajectory belongs to the same labeled target by analyzing the velocity, acceleration, direction change and other characteristics of the motion trajectory. Alternatively, it is also possible to identify whether the motion trajectory belongs to the same labeled target based on the temporal and spatial continuity of the motion trajectory.

[0110] If it is determined that a certain motion trajectory does not belong to the same labeled target, the trajectory is split, and a corresponding relationship between the split trajectory and multiple labeled targets is established.

[0111] S2332, merging the motion trajectories belonging to the same labeled target to obtain tracking information corresponding to each labeled target.

[0112] Since the same marked target may be detected multiple times by different radar devices or at different times, multiple motion trajectories are generated. In addition, in the above steps, splitting the motion trajectory may also generate discontinuous motion trajectory segments.

[0113] Therefore, in this step, the motion trajectories belonging to the same labeled target can be integrated into a single trajectory based on the motion continuity of the labeled target to ensure the integrity and continuity of the motion trajectory, thereby improving the accuracy of the tracking information and the accuracy and robustness of the point cloud annotation data.

[0114] In the disclosed embodiment, by splitting motion trajectories that do not belong to the same labeled target and merging motion trajectories that belong to the same labeled target, the accuracy of tracking information is improved, thereby improving the accuracy of point cloud annotation data corresponding to the labeled target.

[0115] The following will continue to introduce the specific implementation method of obtaining the point cloud data to be labeled.

[0116] As mentioned in the above embodiment, the point cloud data to be labeled is collected by a radar carried by a vehicle. Figure 8 Shown is a schematic diagram of a vehicle and a radar carried by the vehicle provided in an embodiment of the present disclosure.

[0117] like Figure 8 As shown, the radars carried by the vehicle include a main radar and a blind spot radar. Optionally, there are multiple blind spot radars, and the multiple blind spot radars are respectively located at the left front, right front and rear of the vehicle. Figure 8 , the main radar is located on the top of the vehicle, so the field of view of the main radar can cover a farther area. The field of view of the blind spot radar located on the left front of the vehicle covers the front and left side of the vehicle, partially overlapping with the field of view of the main radar. The field of view of the blind spot radar located on the right front of the vehicle covers the front and right side of the vehicle, partially overlapping with the field of view of the main radar. The field of view of the blind spot radar located at the rear of the vehicle covers the rear of the vehicle, partially overlapping with the field of view of the blind spot radars on the left front and right front. In this way, the vehicle's radar can look around the nearby area around the vehicle in all directions.

[0118] Optionally, the main radar includes a high-beam laser radar, and the blind spot radar includes a low-beam laser radar. The high-beam laser radar has multiple laser emission and receiving channels, which can provide denser point cloud data to help the vehicle identify long-range and detailed environmental information. The low-beam laser radar emits fewer laser beams and has a relatively low cost. Therefore, Figure 8 The layout shown ensures that the radar can detect environmental information in the far forward area and also ensures that there is no blind spot coverage around the vehicle. At the same time, it can reduce hardware costs.

[0119] Due to the main radar's field of view ( Figure 8 The oblique line area in the image) and the field of view of the blind radar ( Figure 8 There is an overlapping area (in the dotted area in the image), so the point cloud data collected by the main radar and the blind radar partially overlap. Based on this, the step of obtaining the point cloud data to be labeled provided in the embodiment of the present disclosure includes: obtaining the point cloud data collected by the main radar and the point cloud data collected by the blind radar; based on the overlapping area, densely combining at least part of the point cloud data collected by the main radar and at least part of the point cloud data collected by the blind radar to obtain the point cloud data to be labeled.

[0120] Specifically, the point cloud data collected by the main radar and the blind radar are obtained respectively. Since there is an overlapping area in the field of view of the main radar and the blind radar, the point cloud data of the main radar and the point cloud data collected by the blind radar can be transformed into a coordinate system based on the spatial position of the point cloud data, and the two can be densely combined to obtain the point cloud data to be labeled. At this time, in the point cloud data to be labeled, the point cloud density is higher in the overlapping area of ​​the field of view of the main radar and the field of view of the blind radar. Since the performance of the detection depends on the number of point clouds during the target detection process, that is, the denser the point cloud, the better the performance. Therefore, the prediction of the pre-selected box based on the densely combined point cloud data can obtain a better prediction effect.

[0121] In the disclosed embodiment, by arranging the main radar and the blind spot radar on the vehicle, the radar of the vehicle can collect point cloud data without blind spots around the vehicle. At the same time, the fields of view of the main radar and the blind spot radar overlap, so the density of the point cloud data collected by the radar can be increased, thereby improving the accuracy of the pre-selected box determined by the target detection process.

[0122] Similarly, during the training process of the target detection model, the point cloud data used as training samples can also be densely combined, and the densified point cloud data can be used for iterative sampling enhancement training. Specifically, in order to fully utilize the advantages of the target detection model and improve the accuracy of the algorithm, the present embodiment adopts a training scheme combining multiple iterations and target sampling enhancement to train the target detection model, and improves the model accuracy through cross-initialization between models; at the same time, the number of annotated targets and the category richness can be increased, thereby improving the model accuracy.

[0123] Combination of the above Figures 1 to 8 , describes in detail the method embodiment of the present disclosure, and the following is combined with Fig. 9 , describes the device embodiment of the present disclosure in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0124] Exemplary Devices

[0125] Fig. 9 Shown is a schematic structural diagram of a point cloud annotation device provided in one embodiment of the present disclosure.

[0126] like Fig. 9As shown, the point cloud annotation device 900 of the embodiment of the present disclosure includes: an acquisition module 910, configured to acquire point cloud data to be annotated, wherein the point cloud data to be annotated includes an annotation target. A determination module 920, configured to determine a pre-selection box corresponding to the annotation target based on the point cloud data to be annotated, wherein the pre-selection box is used to show the boundary of the annotation target. A first generation module 930, configured to predict the motion trajectory of the pre-selection box, and to generate tracking information corresponding to the annotation target based on the motion trajectory. A second generation module 940, configured to generate point cloud annotation data corresponding to the annotation target based on the pre-selection box corresponding to the annotation target and the tracking information.

[0127] In some embodiments, the second generation module 940 is further configured to, based on the pre-selection box, crop the point cloud data to be annotated to obtain target point cloud data; based on the target point cloud data, optimize the attributes of the pre-selection box using the attribute optimization model to obtain the target pre-selection box; and obtain point cloud annotation data based on the tracking information and the target pre-selection box.

[0128] In some embodiments, the second generating module 940 is further configured to dilate the pre-selected box; and crop at least a portion of the point cloud data to be annotated within the pre-selected box after the dilation process to obtain target point cloud data.

[0129] In some embodiments, the attribute optimization model includes a geometric attribute optimization model and / or a position attribute optimization model. The second generation module 940 is also configured to use the geometric attribute optimization model to perform point-to-surface feature encoding on the target point cloud data, and optimize the geometric attributes of the pre-selection box based on the point-to-surface feature encoding result of the target point cloud data to obtain the target pre-selection box; and / or, use the position attribute optimization model to perform point-to-corner feature encoding on the target point cloud data, and optimize the position attribute of the pre-selection box based on the point-to-corner feature encoding result of the target point cloud data to obtain the target pre-selection box. Preferably, the geometric attribute optimization model and / or the position attribute optimization model are obtained by training based on the self-attention mechanism.

[0130] In some embodiments, the point cloud annotation device 900 further includes a confidence evaluation module. The confidence evaluation module is configured to determine the confidence information of the target pre-selection box based on the confidence evaluation model, wherein the confidence information includes the category score and the intersection-over-union (IOU) score of the target pre-selection box. The second generation module 940 is further configured to determine the point cloud annotation data based on the tracking information, and the geometric attributes, position attributes and confidence information of the target pre-selection box.

[0131] In some embodiments, there are multiple labeled targets, multiple pre-selected boxes, and there is a corresponding relationship between the labeled targets and the pre-selected boxes. The first generation module 930 is further configured to determine the motion trajectories of the multiple pre-selected boxes using a trajectory tracking model; based on the motion trajectories of the multiple pre-selected boxes, filter invalid pre-selected boxes in the multiple pre-selected boxes to obtain multiple valid pre-selected boxes; based on the motion trajectories of the multiple valid pre-selected boxes, respectively determine the tracking information corresponding to each labeled target. Preferably, the trajectory tracking model includes a Kalman filter model.

[0132] In some embodiments, the first generation module 930 is further configured to, for the motion trajectories of multiple valid pre-selected boxes, split the motion trajectories that do not belong to the same marked target; merge the motion trajectories that belong to the same marked target, and obtain tracking information corresponding to each marked target.

[0133] In some embodiments, the point cloud data to be annotated is collected by a radar carried by a vehicle, and the radar carried by the vehicle includes a main radar and a blind radar, and there is an overlapping area between the field of view of the main radar and the field of view of the blind radar. The acquisition module 910 is also configured to obtain the point cloud data collected by the main radar and the point cloud data collected by the blind radar; based on the overlapping area, densely combine at least part of the point cloud data collected by the main radar and at least part of the point cloud data collected by the blind radar to obtain the point cloud data to be annotated. Preferably, the main radar includes a high-beam laser radar, and the blind radar includes a low-beam laser radar. Preferably, there are multiple blind radars, and the multiple blind radars are respectively located at the left front, right front and rear of the vehicle.

[0134] Below, reference Fig.10 To describe an electronic device embodiment of the present disclosure. Fig.10 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig.10 As shown, electronic device 1000 includes one or more processors 1010 and memory 1020 .

[0135] The processor 1010 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1000 to perform desired functions.

[0136] The memory 1020 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1010 may execute the program instructions to implement the point cloud annotation method and / or other desired functions of the various embodiments of the present disclosure described above.

[0137] In some embodiments, the electronic device 1000 may further include: an input device 1030 and an output device 1040 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0138] The input device 1030 may include, for example, a touch screen, a keyboard, a mouse, etc. The output device 1040 may output point cloud annotation data to the outside, etc. The output device 1040 may include a display, a vehicle-mounted terminal, and a communication network and a remote output device connected thereto.

[0139] Of course, to simplify, Fig.10 Only some of the components related to the present disclosure in the electronic device 1000 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 1000 may also include any other appropriate components.

[0140] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the point cloud annotation method according to various embodiments of the present disclosure described above in this specification.

[0141] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0142] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the point cloud annotation method according to various embodiments of the present disclosure described above in this specification.

[0143] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0144] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0145] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0146] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0147] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0148] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A point cloud annotation method, characterized in that: include: Acquire point cloud data to be annotated, wherein the point cloud data to be annotated includes an annotation target; Based on the point cloud data to be annotated, determining a pre-selected box corresponding to the annotated object, wherein the pre-selected box is used to show a boundary of the annotated object; Predicting a motion trajectory of the pre-selected box, and generating tracking information corresponding to the marked target based on the motion trajectory; Based on the pre-selected box corresponding to the labeled target and the tracking information, point cloud annotation data corresponding to the labeled target is generated.

2. The point cloud annotation method according to claim 1, characterized in that: The generating point cloud annotation data corresponding to the annotation target based on the pre-selected box corresponding to the annotation target and the tracking information includes: Based on the pre-selected box, the point cloud data to be annotated is cropped to obtain target point cloud data; Based on the target point cloud data, optimizing the attributes of the pre-selection box using an attribute optimization model to obtain a target pre-selection box; The point cloud annotation data is obtained based on the tracking information and the target pre-selection box.

3. The point cloud annotation method according to claim 2, characterized in that: The step of clipping the point cloud data to be annotated based on the pre-selected box to obtain target point cloud data includes: Performing expansion processing on the pre-selected box; At least part of the point cloud data to be marked within the pre-selected box after the expansion process is cropped to obtain the target point cloud data.

4. The point cloud annotation method according to claim 2, characterized in that: The attribute optimization model includes a geometric attribute optimization model and / or a position attribute optimization model. Based on the target point cloud data, the attribute optimization model is used to optimize the attributes of the pre-selection box to obtain the target pre-selection box, including: Using the geometric property optimization model, performing point-to-surface feature encoding on the target point cloud data, and optimizing the geometric properties of the pre-selection box based on the point-to-surface feature encoding result of the target point cloud data to obtain the target pre-selection box; and / or Using the position attribute optimization model, performing point-to-corner feature encoding on the target point cloud data, and optimizing the position attribute of the pre-selection box based on the point-to-corner feature encoding result of the target point cloud data to obtain the target pre-selection box; Preferably, the geometric attribute optimization model and / or the position attribute optimization model is trained based on a self-attention mechanism.

5. The point cloud annotation method according to claim 2 or 4, characterized in that: After optimizing the attributes of the pre-selected box based on the target point cloud data by using the attribute optimization model to obtain the target pre-selected box, the method further includes: Based on the confidence evaluation model, determine the confidence information of the target pre-selected box, wherein the confidence information includes the category score and the intersection-over-union (IOU) score of the target pre-selected box; The obtaining of the point cloud annotation data based on the tracking information and the target pre-selection box includes: The point cloud annotation data is determined based on the tracking information, the geometric attributes, the position attributes and the confidence information of the target pre-selection box.

6. The point cloud annotation method according to claim 1, characterized in that: There are multiple marked targets, there are multiple pre-selected boxes, and there is a corresponding relationship between the marked targets and the pre-selected boxes; The predicting the motion trajectory of the pre-selected box and generating tracking information corresponding to the marked target based on the motion trajectory includes: Determine the motion trajectory of each of the plurality of pre-selected boxes using a trajectory tracking model; Based on the motion trajectories of the plurality of pre-selected boxes, filtering out invalid pre-selected boxes from the plurality of pre-selected boxes to obtain a plurality of valid pre-selected boxes; Based on the motion trajectories of the plurality of valid pre-selected boxes, respectively determining the tracking information corresponding to each of the marked targets; Preferably, the trajectory tracking model includes a Kalman filter model.

7. The point cloud annotation method according to claim 6, characterized in that: The determining, based on the respective motion trajectories of the plurality of valid pre-selected boxes, the tracking information corresponding to each of the marked targets comprises: For the motion trajectories of the plurality of valid pre-selected boxes, the motion trajectories that do not belong to the same marked object are split; The motion trajectories belonging to the same marked object are merged to obtain tracking information corresponding to each of the marked objects.

8. The point cloud annotation method according to claim 1, characterized in that: The point cloud data to be annotated is collected by a radar carried by a vehicle, the radar carried by the vehicle includes a main radar and a blind spot radar, and there is an overlapping area between the field of view of the main radar and the field of view of the blind spot radar; The step of obtaining the point cloud data to be annotated includes: Acquire the point cloud data collected by the main radar and the point cloud data collected by the blind spot radar; Based on the overlapping area, densely combining at least part of the point cloud data collected by the main radar and at least part of the point cloud data collected by the blind spot radar to obtain the point cloud data to be labeled; Preferably, the main radar includes a high-beam laser radar, and the blind spot radar includes a low-beam laser radar; Preferably, there are multiple blind spot radars, and the multiple blind spot radars are respectively located at the left front, right front and rear of the vehicle.

9. A point cloud annotation device, characterized in that: include: An acquisition module is configured to acquire point cloud data to be annotated, wherein the point cloud data to be annotated includes an annotation target; A determination module is configured to determine a pre-selected box corresponding to the annotation target based on the point cloud data to be annotated, wherein the pre-selected box is used to show the boundary of the annotation target; A first generating module is configured to predict a motion trajectory of the pre-selected box and generate tracking information corresponding to the marked target based on the motion trajectory; The second generating module is configured to generate point cloud annotation data corresponding to the annotation target based on the pre-selected box corresponding to the annotation target and the tracking information.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the point cloud annotation method described in any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud annotation method according to any one of claims 1 to 8 is implemented.

Images