Target labeling method, device, and equipment, and storage medium

Abstract

The application discloses a target labeling method, device and equipment and a storage medium. The method comprises the following steps: acquiring continuous multiple frames of point clouds of a target and target labeling information of a first frame of point clouds; for each current frame in the continuous multiple frames, determining a current frame expansion frame of the target according to a current frame bounding box of the current frame point cloud containing the target and an expansion coefficient of the current frame bounding box; projecting the current frame expansion frame onto a next frame point cloud to obtain a next frame prediction box; clustering the next frame point cloud in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box; performing target tracking on the current frame bounding boxes corresponding to the current frame point clouds in the continuous multiple frames of point clouds until a tracking termination condition is reached, and performing target labeling on the continuous multiple frames of point clouds by using the target labeling information, so that a bounding box with an adaptive size is matched according to the motion information of the target, and the target labeling accuracy based on the bounding box is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving perception, and in particular to a target labeling method and device, equipment and a storage medium. BACKGROUND

[0002] In vehicle automatic driving, a vehicle collects point cloud data through a sensor, and processes the point cloud data to output target information visually perceived in an automatic driving scene. When processing the point cloud information, the point cloud information needs to be labeled, and then target recognition is performed according to the labeled point cloud information.

[0003] In the existing target labeling related technology, an artificial labeling method is mainly used. This method has high accuracy, but artificial labeling consumes a large amount of manpower and financial resources. Therefore, automatic labeling methods have gradually emerged.

[0004] The current automatic labeling method has the following problems: for the point cloud of a target in consecutive frames, a boundary box of a previous frame is usually projected onto a next frame to obtain a boundary box of the point cloud of the next frame, that is, a fixed-size boundary box is used for target tracking and labeling. However, for a target in a motion state, the size of the boundary box may change, and using a fixed-size boundary box for target labeling of consecutive frame point clouds may result in a decrease in target labeling accuracy. SUMMARY

[0005] The present application provides a target labeling method, device, equipment and storage medium to solve the problem that using a fixed-size boundary box for target tracking and labeling may result in low target labeling accuracy, and to realize matching of a self-adaptive-size boundary box according to motion information of a target and improve the accuracy of target labeling based on a boundary box.

[0006] According to an aspect of the present application, a target labeling method is provided, comprising:

[0007] obtaining target labeling information of a first frame point cloud and consecutive frame point clouds of a target;

[0008] For each current frame in the consecutive frames, a current frame expansion box of the target is determined according to a current frame boundary box of the current frame point cloud containing the target and an expansion coefficient of the current frame boundary box obtained;

[0009] projecting the current frame expansion box onto a next frame point cloud to obtain a next frame prediction box;

[0010] performing clustering on the next frame point cloud in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame boundary box;

[0011] The target labeling information is used to label the continuous multiple frames of point clouds.

[0012] According to another aspect of the present application, a labeling device of a target is provided, comprising:

[0013] A point cloud obtaining module is configured to obtain continuous multiple frames of point clouds of a target and target labeling information of a first frame of point clouds.

[0014] An expansion module is configured to, for each current frame in the continuous multiple frames, determine a current frame expansion bounding box of the target according to a current frame bounding box of a current frame of point clouds containing the target and an expansion coefficient of the current frame bounding box obtained.

[0015] A projection module is configured to project the current frame expansion bounding box onto a next frame of point clouds to obtain a next frame prediction bounding box.

[0016] A clustering module is configured to cluster a next frame of point clouds in the next frame prediction bounding box to obtain a next frame clustering bounding box of the target, and take the next frame clustering bounding box as a new current frame bounding box.

[0017] A labeling module is configured to perform target association and tracking on current frame bounding boxes corresponding to each current frame of point clouds in the continuous multiple frames of point clouds until a tracking termination condition is reached, and use the target labeling information to label the continuous multiple frames of point clouds.

[0018] According to another aspect of the present application, a method for completing a bounding box is provided, comprising:

[0019] A current frame bounding box corresponding to a current frame of point clouds of a target is obtained.

[0020] If the current frame bounding box is a defective bounding box, a preset completion strategy is used to complete the current frame bounding box to obtain a current frame completed bounding box of the target.

[0021] According to another aspect of the present application, a device for completing a bounding box is provided, comprising:

[0022] A bounding box obtaining module is configured to obtain a current frame bounding box corresponding to a current frame of point clouds of a target.

[0023] A completion module is configured to, if the current frame bounding box is a defective bounding box, use a preset completion strategy to complete the current frame bounding box to obtain a current frame completed bounding box of the target.

[0024] According to another aspect of the present application, an electronic device is provided, comprising:

[0025] at least one processor; and

[0026] a memory in communication with the at least one processor; wherein,

[0027] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the object labeling method according to the first to fourth embodiments of the present application, or perform the boundary box completion method according to the fifth embodiment of the present application.

[0028] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the object labeling method according to the first to fourth embodiments of the present application, or implement the boundary box completion method according to the fifth embodiment of the present application.

[0029] The technical solution of the embodiments of the present application comprises the following steps: obtaining continuous multiple frames of point clouds of a target and target labeling information of a first frame of point clouds; for each current frame in the continuous multiple frames, determining a current frame extended boundary box of the target according to a current frame boundary box of the current frame of point clouds containing the target and an extension coefficient of the current frame boundary box obtained; projecting the current frame extended boundary box onto a next frame of point clouds to obtain a next frame prediction box; clustering the next frame of point clouds in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame boundary box; performing target tracking on the current frame boundary box corresponding to each current frame of point clouds in the continuous multiple frames of point clouds until a tracking termination condition is reached, and performing target labeling on the continuous multiple frames of point clouds using the target labeling information. The method of projecting the current frame extended boundary box obtained by extending the target boundary box onto the next frame of point clouds and clustering to obtain the next frame boundary box can reasonably predict the position and size of the next frame boundary box according to the motion information of the target, so that the next frame boundary box can contain the complete point cloud of the target and fit the target point cloud, and the boundary box of adaptive size is matched according to the motion information of the target, thereby solving the problem that the target tracking and labeling using the boundary box of fixed size will result in low target labeling accuracy, and achieving the beneficial effect of improving the target labeling accuracy based on the boundary box.

[0030] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0032] Figure 1 The flow chart of the target labeling method provided by the embodiment of the present application;

[0033] Figure 2A The flow chart of the target labeling method provided by the embodiment of the present application;

[0034] Figure 2B The schematic diagram of the selected judgment point on the minimum bounding box when the type of the target is a motor vehicle;

[0035] Figure 2C The schematic diagram of the selected judgment point on the minimum bounding box when the type of the target is a non-motor vehicle;

[0036] Figure 3 The flow chart of the target labeling method provided by the embodiment of the present application;

[0037] Figure 4A The flow chart of the target labeling method provided by the embodiment of the present application;

[0038] Figure 4B The schematic diagram of the incomplete bounding box completion method;

[0039] Figure 4C The schematic diagram of another incomplete bounding box completion method;

[0040] Figure 5 The flow chart of the bounding box completion method provided by the embodiment of the present application;

[0041] Figure 6 The structural schematic diagram of the target labeling device provided by the embodiment of the present application;

[0042] Figure 7 The structural schematic diagram of the bounding box completion device provided by the embodiment of the present application;

[0043] Figure 8 The structural schematic diagram of the electronic device for implementing the target labeling method or the bounding box completion method of the embodiment of the present application. DETAILED DESCRIPTION

[0044] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0045] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0046] Figure 1 A flowchart of a target labeling method is provided for an embodiment of the present application. The embodiment can be applicable to the automatic labeling of target point clouds of consecutive frames. The method can be performed by a target labeling device, which can be realized in the form of hardware and / or software. The target labeling device can be configured in an electronic device, which can be a vehicle-mounted device used on a vehicle or a device used independently of the vehicle. As shown in Figure 1 The method includes:

[0047] S110, acquiring target continuous multi-frame point clouds and target labeling information of a first frame point cloud.

[0048] The type of the target can include movable targets such as vehicles, pedestrians and animals, and other stationary targets that can be perceived, such as roadblocks and railings. The target continuous multi-frame point clouds can be continuous multi-frame point clouds containing targets collected by a laser sensor or a radar. The continuous multi-frame point clouds can have the same or different sampling intervals. The first frame point cloud refers to the first frame point cloud in the continuous multi-frame point clouds. The target labeling information is obtained by manually labeling the target of the first frame point cloud. The target labeling information can include the bounding box of the target corresponding to the first frame point cloud and the target type and other labeling information.

[0049] Specifically, the acquired target continuous multi-frame point clouds are obtained. The target labeling information is obtained by manually labeling the first frame point cloud in the continuous multi-frame point clouds.

[0050] For example, the way of obtaining the continuous multiple frames of point clouds of the target can be that a laser sensor or a radar arranged on the vehicle collects the continuous multiple frames of point clouds of each target contained in the sensing range, so as to label each target in the surrounding environment of the vehicle during the driving of the vehicle; or the sensor arranged at a fixed position on the road can collect the continuous multiple frames of point clouds of each target in the sensing range, so as to label each target appearing in the fixed region.

[0051] S120, for each current frame in the continuous multiple frames, determining a current frame expansion box of the target according to a current frame bounding box of the current frame point cloud containing the target and an expansion coefficient of the obtained current frame bounding box.

[0052] The current frame in the continuous multiple frames can include a first frame and a non-first frame in the continuous multiple frames, that is, the first frame to the last frame in the continuous multiple frames. The current bounding box refers to the bounding box corresponding to the current frame point cloud containing the target. The expansion coefficient is a numerical value representing the expansion degree of the bounding box, and the current frame expansion box is obtained by expanding the current bounding box according to the expansion coefficient.

[0053] Specifically, for each current frame in the continuous multiple frames, the current frame bounding box of the target is determined according to the current frame point cloud containing the target, and the expansion coefficient corresponding to the current frame bounding box is obtained. Then, the current frame bounding box is expanded by using the expansion coefficient to obtain the current frame expansion box of the target.

[0054] For example, the way of determining the current frame expansion box of the target according to the current frame bounding box of the current frame point cloud containing the target and the expansion coefficient of the obtained current frame bounding box can be that the size of the current bounding box is expanded by the expansion coefficient times in a set expansion direction with the center point of the current bounding box as the center to obtain the size and position of the current frame bounding box; or the size of the current bounding box is expanded by the length of the expansion coefficient in a set expansion direction to obtain the size and position of the current frame bounding box.

[0055] For example, the way of obtaining the current frame bounding box can be that the current frame bounding box of the current frame point cloud containing the target is set by using a manual labeling method for the first frame point cloud; and the detection box in the current frame point cloud is set as the current frame bounding box by using a conventional target detection algorithm for the non-first frame point cloud, which is not described and limited herein. The way of obtaining the expansion coefficient of each current frame bounding box can be that the expansion coefficient is obtained according to at least one of the frame number of the current frame, the state and type of the target, and the change amount between adjacent frame point clouds.

[0056] S130, projecting the current frame expansion box onto the next frame point cloud to obtain a next frame prediction box.

[0057] The latter frame prediction box is a prediction box obtained by predicting the latter frame point cloud of the target in the latter frame.

[0058] Specifically, when labeling the target in the continuous frame point cloud, the current frame extended bounding box corresponding to the current frame point cloud of the target needs to be projected onto the latter frame point cloud, so that the latter frame prediction box corresponding to the latter frame point cloud of the target can be obtained in the latter frame.

[0059] Specifically, the step of projecting the current frame extended bounding box onto the latter frame point cloud to obtain the latter frame prediction box can be: obtaining first position information of the current frame extended bounding box in the current frame coordinate system, projecting the current frame extended bounding box in the world coordinate system according to the first local position information of the current frame extended bounding box to obtain the world position information of the current frame extended bounding box in the world coordinate system. Then, according to the world position information of the current frame extended bounding box, the current frame extended bounding box in the world coordinate system is projected onto the latter frame coordinate system to obtain second position information of the current frame extended bounding box in the latter frame coordinate system. The second position information of the current frame extended bounding box is determined as the position information of the latter frame prediction box in the latter frame, so as to obtain the latter frame prediction box.

[0060] In the case that the target is in motion, for the traditional method of directly projecting the current frame bounding box onto the latter frame point cloud to obtain the latter frame prediction box, due to the motion of the target and the distortion of the point cloud, it cannot be guaranteed that the latter frame prediction box projected onto the latter frame can contain the complete latter frame point cloud of the target, thereby affecting the accuracy of the target labeling. However, by projecting the current frame extended bounding box obtained by expanding the target bounding box onto the latter frame point cloud to obtain the latter frame prediction box, the position and size of the latter frame prediction box can be reasonably predicted according to the motion information of the target, so that the latter frame prediction box can contain the complete point cloud of the target, thereby the size of the prediction box of the target can be adaptively matched with the motion information of the target, and the accuracy of the target labeling is improved.

[0061] S140, clustering the latter frame point cloud in the latter frame prediction box to obtain a latter frame clustering box of the target, and taking the latter frame clustering box as a new current frame bounding box.

[0062] The current frame bounding box can be understood as a bounding box containing the current frame point cloud of the target.

[0063] Specifically, since the latter frame prediction box is obtained by projecting the current frame extended bounding box onto the latter frame point cloud, and the current frame extended bounding box is obtained by extending the bounding box, the latter frame prediction box contains the complete latter frame point cloud corresponding to the target while making the prediction box not fit the target point cloud, which may contain background noise or other target noise, and thus it is necessary to further cluster the latter frame point cloud in the latter frame prediction box, and take the latter frame cluster box obtained by clustering as a new current frame bounding box. For the new current frame bounding box, the current frame extended bounding box of the target is determined according to the current frame bounding box of the current frame point cloud containing the target and the extended coefficient of the current frame bounding box obtained, until the current frame bounding box corresponding to each frame is determined.

[0064] For example, the manner of clustering the latter frame point cloud in the latter frame prediction box can be to select different clustering manners to cluster the latter frame point cloud in the latter frame prediction box according to the type of the target. The clustering manner can be clustering based on minimum distance or minimum average distance, and the embodiments of the present application do not limit this.

[0065] It should be noted that the position of the new current frame bounding box corresponding to the latter frame and the current corresponding current frame bounding box can not be completely aligned, but the new current frame bounding box can contain the complete latter frame point cloud of the target. If the new current frame bounding box does not completely contain the complete latter frame point cloud of the target in the prediction process, it can be caused by inaccurate motion state (such as motion speed) of the target. In order to ensure that the current frame bounding box can contain the complete point cloud of the target, it can be verified whether the current frame bounding box contains the complete point cloud of the target, if the verification fails, the current frame bounding box can be manually deleted and re-annotated, or the motion state information of the target is re-predicted and the current frame bounding box is re-determined in steps S120 to S140 to ensure that the current frame bounding box of each frame can include the complete target point cloud.

[0066] This step makes the current frame bounding box in the current frame contain the complete current frame point cloud corresponding to the target while making the size of the current frame bounding box more fit the current frame point cloud of the target, and reduces the point cloud noise.

[0067] S150, track the current frame bounding box corresponding to each current frame point cloud in the continuous multiple frame point clouds until a tracking termination condition is reached, and label the target in the continuous multiple frame point clouds using the target annotation information.

[0068] Specifically, after the current frame bounding boxes corresponding to each current frame point cloud in the continuous multiple frames of point clouds are obtained by the expansion, projection and clustering manner of steps S120 to S140, the same unique tracking identification numbers are constructed for the current frame bounding boxes corresponding to the current frame point clouds of the same target, such as target 1, target 2 and target 3, etc. The current frame bounding boxes with the same bounding box identification numbers are associated and tracked until the tracking termination condition is reached. The target annotation information manually annotated on the first frame point cloud is used to annotate the point clouds contained in the bounding boxes of each frame associated with the current frame bounding box corresponding to the first frame point cloud, so as to realize the target annotation of the continuous multiple frames of point clouds.

[0069] For example, the tracking termination condition can include at least one of the following: the prediction box does not contain the target point cloud, the prediction box exceeds the preset size, the prediction box exceeds the region of interest, or the prediction box is manually modified.

[0070] The technical scheme of the embodiment of the present application comprises the following steps: obtaining the target annotation information of the continuous multiple frames of point clouds and the first frame point cloud of a target; for each current frame in the continuous multiple frames, determining the current frame expansion box of the target according to the current frame bounding box of the current frame point cloud containing the target and the expansion coefficient of the current frame bounding box obtained; projecting the current frame expansion box onto the next frame point cloud to obtain the next frame prediction box; clustering the next frame point cloud in the next frame prediction box to obtain the next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box; performing target tracking on the current frame bounding boxes corresponding to each current frame point cloud in the continuous multiple frames of point clouds until the tracking termination condition is reached, and using the target annotation information to perform target annotation on the continuous multiple frames of point clouds. By projecting the target bounding box after expansion onto the next frame point cloud and clustering to obtain the next frame bounding box, the position and size of the next frame bounding box can be reasonably predicted according to the motion information of the target, so that the next frame bounding box can contain the complete point cloud of the target and fit the target point cloud, thereby realizing the matching of the self-adaptive size bounding box according to the motion information of the target, and improving the target annotation accuracy based on the bounding box.

[0071] Figure 2A A flowchart of a target annotation method provided by the embodiment of the present application, which further limits the step S140 "clustering the next frame point cloud in the next frame prediction box to obtain the next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box" of the above-mentioned embodiment. As shown in the figure, the method comprises the following steps: Figure 2A

[0072] S210, obtaining the target annotation information of the continuous multiple frames of point clouds and the first frame point cloud of a target.

[0073] ​S220, for each current frame in the continuous multiple frames, determining a current frame extended bounding box of the target according to a current frame bounding box of a current frame point cloud containing the target and an extended coefficient of the current frame bounding box obtained.

[0074] S230, projecting the current frame extended bounding box onto the next frame point cloud to obtain a next frame prediction box.

[0075] Optionally, after projecting the current frame extended bounding box onto the next frame point cloud to obtain the next frame prediction box, and before clustering the next frame point cloud in the next frame prediction box to obtain the next frame clustering box of the target, the method further comprises:

[0076] performing noise point filtering on the next frame point cloud in the next frame prediction box; wherein the noise points include ground noise and abnormal point clouds in the next frame prediction box.

[0077] Specifically, since the next frame prediction box is obtained by projecting the current frame extended bounding box onto the next frame point cloud, and the current frame extended bounding box is obtained by extending the bounding box, the next frame prediction box contains the complete next frame point cloud corresponding to the target and noise not belonging to the target point cloud. Therefore, before clustering the next frame point cloud in the next frame prediction box, noise point filtering can be performed on the next frame point cloud in the next frame prediction box to filter the ground noise and abnormal point clouds in the next frame prediction box. The abnormal point clouds may be, for example, sporadic isolated points obviously deviating from the point cloud cluster. The method of noise point filtering in the embodiments of the present application is not limited, and a suitable filtering method can be selected according to actual needs.

[0078] S240, if the type of the target is a vehicle, performing iterative rotation on the next frame point cloud in the next frame prediction box to obtain a clustering direction, and clustering the next frame point cloud in the next frame prediction box according to the clustering direction to obtain a next frame clustering box of the target.

[0079] The next frame clustering box can be understood as a clustering box obtained by clustering the next frame prediction box in the next frame.

[0080] Specifically, a suitable clustering method is used to cluster the next frame point cloud in the next frame prediction box according to the characteristics of the type of the target. In the automatic driving scene, the movable targets mainly include vehicles (including motor vehicles or non-motor vehicles) and pedestrians. If the label information corresponding to the first frame point cloud of the target indicates that the type of the target is a vehicle, the next frame point cloud in the next frame prediction box is clustered by using the prediction box direction method. The basic idea of the prediction box direction method is to use the direction of the prediction box obtained by multiple iterative rotations as the clustering direction for clustering.

[0081] An example is to create a next frame prediction box coordinate system with the center point of the next frame prediction box as the origin and the direction of the next frame prediction box as the positive direction of the Y axis. In the next frame prediction box coordinate system, the next frame point cloud in the next frame prediction box is iteratively rotated according to a preset rotation angle and a preset number of rotations to obtain a next frame prediction box direction. The next frame prediction box direction is determined as a clustering direction, and the next frame point cloud in the next frame prediction box is clustered according to the clustering direction to obtain a next frame clustering box of the target.

[0082] In S250, if the type of the target is a pedestrian, the next frame point cloud in the next prediction box is clustered according to the center point offset value of the next frame prediction box to obtain a next frame clustering box of the target.

[0083] Specifically, if the label information corresponding to the first frame point cloud of the target indicates that the type of the target is a pedestrian, the next frame point cloud in the next prediction box is clustered by using a speed direction method. The basic idea of the speed direction method is to determine a speed direction of the prediction box according to the center point offset value of the next frame prediction box relative to the current frame prediction box, determine the speed direction of the prediction box as a clustering direction, and cluster the next frame point cloud in the next frame prediction box according to the clustering direction to obtain a next frame clustering box of the target.

[0084] In S260, the next frame clustering box is taken as a new current frame bounding box.

[0085] In S270, the target tracking is performed on the current frame bounding boxes corresponding to the current frame point clouds in the continuous multiple frame point clouds until a tracking termination condition is reached, and the target label information is used to label the continuous multiple frame point clouds.

[0086] The technical scheme of the embodiment of the application adopts the manner of projecting the current frame extended box obtained by extending the target bounding box to the next frame point cloud and clustering to obtain the next frame bounding box, which can reasonably predict the position and size of the next frame bounding box according to the motion information of the target, so that the next frame bounding box can contain the complete point cloud of the target and fit the point cloud, thereby realizing the matching of the bounding box with an adaptive size according to the motion information of the target. Moreover, the point cloud in the prediction box is clustered to obtain a clustering box by using a suitable clustering method according to the characteristics of the type of the target, which can improve the clustering accuracy and further make the bounding box with an adaptive size more fit the target point cloud, thereby further improving the target labeling accuracy based on the bounding box.

[0087] In an optional embodiment, for step S240, the next frame point cloud in the next frame prediction box is iteratively rotated to obtain a clustering direction, including:

[0088] S241, performing iterative rotation on the next frame point cloud in the next frame prediction box based on a preset iterative rotation strategy; wherein the preset iterative rotation strategy comprises: rotating the next frame point cloud in the next frame prediction box by a preset unit rotation angle, respectively, in a clockwise direction and an anticlockwise direction for a preset number of times.

[0089] Specifically, if the type of the target is a vehicle, the process of clustering the next frame point cloud in the next frame prediction box by using the prediction box direction method can specifically include: first, rotating the next frame point cloud in the next frame prediction box by a preset unit rotation angle, respectively, in a clockwise direction and an anticlockwise direction for a preset number of times.

[0090] Exemplarily, taking the origin of the next frame prediction box coordinate system as the rotation center point, the next frame point cloud in the next frame prediction box is iteratively rotated by 0.3 degrees as a unit rotation angle, respectively, in a clockwise direction and an anticlockwise direction for 20 times.

[0091] S242, constructing a minimum bounding box of the next frame point cloud after each rotation, and selecting a preset number of judgment points on the minimum bounding box to determine the average minimum distance of each judgment point and the next frame point cloud; the average minimum distance is the average value of the minimum distance of each judgment point and the next frame point cloud.

[0092] Specifically, after each rotation, a minimum bounding box that can completely enclose the next frame point cloud is constructed. According to the further divided subtypes of the target, a preset number of judgment points are selected on the minimum bounding box, a preset number of judgment points are selected on the minimum bounding box, and the minimum distance between each judgment point and the next frame point cloud is determined in turn, and the average value of the minimum distance between each judgment point and the next frame point cloud is calculated as the average minimum distance.

[0093] Optionally, if the type of the target is a motor vehicle, the preset number of judgment points includes: four vertices of the minimum bounding box and three equidistant points on each frame line of the minimum bounding box.

[0094] If the type of the target is a non-motor vehicle, the preset number of judgment points includes: four vertices of the minimum bounding box and two equidistant points on any two parallel frame lines in the minimum bounding box.

[0095] Exemplarily, Figure 2B is a schematic diagram of the judgment points selected on the minimum bounding box when the type of the target is a motor vehicle. As Figure 2B shown, 12-point direction method is adopted for the motor vehicle, that is, 12 judgment points are selected on the minimum bounding box, which are four vertices of the minimum bounding box and three equidistant points on each frame line of the minimum bounding box in turn.

[0096] Figure 2Cis a schematic view of the selected judgment points on the minimum bounding box in the case that the type of the target is a non-motor vehicle. As shown in Figure 2C Since most non-motor vehicles are two-wheeled, a 6-point direction method is adopted for non-motor vehicles, that is, 6 judgment points are selected on the minimum bounding box, which are successively the four vertices of the minimum bounding box and the two midpoints on any two parallel frame lines in the minimum bounding box.

[0097] S243, after completing the iterative rotation, determining the angle corresponding to the average minimum distance obtained in each rotation, and determining the minimum angle among the determined angles as the clustering direction.

[0098] The angle corresponding to the average minimum distance can be understood as the angle of the target point on the minimum bounding box with the distance of the next frame of point cloud being the average minimum distance.

[0099] Specifically, after completing the preset number of iterative rotations, the average minimum distance obtained in each rotation can be obtained. For the average minimum distance obtained in each rotation, the angle of the target point on the minimum bounding box with the distance of the next frame of point cloud being the average minimum distance is determined as the angle corresponding to the average minimum distance. The direction corresponding to the minimum angle is determined as the clustering direction.

[0100] For example, the number of rotations is set to 20, and 20 average minimum distances can be obtained; for each average minimum distance, the angle of the target point on the minimum bounding box with the distance of the next frame of point cloud being the average minimum distance is determined, and 20 angles can be obtained, and the direction corresponding to the minimum angle among the 20 angles is determined as the clustering direction.

[0101] Figure 3 A flowchart of a target labeling method provided by the embodiment of the present application, the embodiment further limits the step S120 "for each current frame in the continuous multiple frames, determining the current frame expansion bounding box of the target according to the current frame bounding box of the current frame point cloud containing the target and the expansion coefficient of the current frame bounding box obtained" of the above-mentioned embodiment. As shown in Figure 3 The method comprises:

[0102] S310, obtaining the target labeling information of the continuous multiple frames of point clouds and the first frame of point clouds of the target.

[0103] S320, for each current frame in the continuous multiple frames, obtaining the current frame bounding box containing the target of the current frame point cloud.

[0104] Specifically, for each current frame in the continuous multiple frames, the current frame bounding box containing the target of the current frame point cloud is determined by using a conventional target detection method, and the embodiment of the present application does not elaborate and limit this.

[0105] S330, if the current frame bounding box is the bounding box corresponding to the first frame point cloud, obtaining a first expansion coefficient of the current frame bounding box.

[0106] The first expansion coefficient can be understood as an expansion coefficient corresponding to the current frame bounding box. The first expansion coefficient can be determined according to the type and dynamic state of the target and the like.

[0107] Specifically, if the current frame is the first frame in a plurality of continuous frames, that is, the current frame bounding box is the bounding box corresponding to the first frame point cloud, the first expansion coefficient corresponding to the current frame bounding box is obtained. The way to obtain the first expansion coefficient can be to obtain the corresponding first expansion coefficient according to the motion state of the target, or to obtain the corresponding first expansion coefficient according to the motion state and type of the target.

[0108] Optionally, in step S330, obtaining the first expansion coefficient of the current frame bounding box comprises: determining the type of the target according to the target annotation information of the first frame point cloud; determining the dynamic state of the target according to the current frame point cloud and the next frame point cloud; determining the first expansion coefficient of the target according to the dynamic state and type of the target.

[0109] The dynamic state of the target includes a motion state and a static state.

[0110] Specifically, since the position and shape of the point cloud of the target will not change basically in the static state, but will cause the position of the point cloud to change and the shape to distort in the motion state, it is necessary to determine the dynamic state of the target according to the current frame point cloud and the next frame point cloud. And for different types of targets, the size of the bounding box will also have a big difference, so it is also necessary to determine the type of the target according to the target annotation information of the first frame point cloud, and to determine the first expansion coefficient of the target according to the dynamic state and type of the target.

[0111] S340, if the current frame bounding box is the bounding box corresponding to the non-first frame point cloud, determining a second expansion coefficient of the current frame bounding box according to a previous frame offset of the current frame bounding box relative to a previous frame bounding box.

[0112] Specifically, for the non-first frame point cloud in a plurality of continuous frames, the second expansion coefficient of the current frame bounding box corresponding to the non-first frame point cloud is mainly determined according to the motion speed of the target. The motion speed of the target can be represented by the previous frame offset of the current frame bounding box relative to the previous frame bounding box.

[0113] Optionally, in step S340, determining the second expansion coefficient of the current frame bounding box according to the previous frame offset of the current frame bounding box relative to the previous frame bounding box comprises:

[0114] S341, calculating a current frame offset of the current frame bounding box relative to the previous frame bounding box.

[0115] S342, inputting the current frame offset into the target function to obtain a second expansion coefficient of the current frame bounding box.

[0116] The target function is used to express the functional relationship between the offset and the second expansion coefficient. The target function can be determined according to a large amount of actual data analysis and learning, etc. Generally, the greater the offset, the greater the target motion speed, and the greater the second expansion coefficient.

[0117] Specifically, the current frame offset of the current frame bounding box relative to the previous frame bounding box is calculated, the current frame offset is input into the target function expressing the functional relationship between the offset and the second expansion coefficient, so as to obtain the second expansion coefficient of the current frame bounding box corresponding to the target function output.

[0118] For example, the way to calculate the current frame offset of the current frame bounding box relative to the previous frame bounding box can be: determining the offset between the center point of the current frame bounding box and the center point of the previous frame bounding box as the current frame offset of the current frame bounding box relative to the previous frame bounding box. The current frame offset of the current frame bounding box relative to the previous frame bounding box can be obtained according to the offset speed of the current frame bounding box multiplied by the time interval between the current frame bounding box and the previous frame bounding box.

[0119] S350, determining a current frame expansion box of the target according to the current frame bounding box and the expansion coefficient.

[0120] S360, projecting the current frame expansion box onto the next frame point cloud to obtain a next frame prediction box.

[0121] S370, clustering the next frame point cloud in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box.

[0122] S380, performing target tracking on the current frame bounding box corresponding to each current frame point cloud in the continuous multiple frame point clouds until a tracking termination condition is reached, and performing target labeling on the continuous multiple frame point clouds using target labeling information.

[0123] The technical scheme of the embodiment of the present application obtains a first edge expansion coefficient of a current frame bounding box through a bounding box corresponding to a first frame of point cloud; obtains a second edge expansion coefficient of the current frame bounding box according to a previous frame offset of the current frame bounding box relative to a previous frame bounding box through a bounding box corresponding to a non-first frame of point cloud; and then determines a current frame edge expansion box of the target according to the current frame bounding box and the edge expansion coefficient. The method realizes determining a suitable edge expansion coefficient according to the motion state and type of the target, appropriately expanding the current frame bounding box corresponding to the current frame of point cloud containing the target to obtain the current frame edge expansion box, and projecting and clustering the current frame edge expansion box to obtain a next frame bounding box, so that the position and size of the next frame bounding box are reasonably predicted according to the motion information of the target, the size of the next frame bounding box is selected on the basis of the complete point cloud containing the target, the noise contained in the bounding box is reduced, and the subsequent clustering effect is improved, thereby realizing matching the bounding box with an adaptive size according to the motion information of the target, and improving the target labeling accuracy based on the bounding box.

[0124] In the process of calculating the current frame offset of the current frame bounding box, the point cloud acquisition time interval between adjacent frames may not be a fixed time interval. In view of this problem, the step of calculating the current frame offset of the current frame bounding box relative to the previous frame bounding box is further optimized. In an optional embodiment, the step S341 of calculating the current frame offset of the current frame bounding box relative to the previous frame bounding box comprises:

[0125] S3411, if the previous frame of point cloud is a non-first frame of point cloud, obtaining a previous frame offset corresponding to a previous frame bounding box of the previous frame of point cloud and a previous frame time; wherein the previous frame offset is an offset of the previous frame bounding box relative to a second previous frame bounding box; and the previous frame time is an acquisition time interval between the previous frame of point cloud and a second previous frame of point cloud.

[0126] S3412, determining a product of the previous frame offset and the previous frame time as a current frame offset speed of the current frame bounding box.

[0127] S3413, obtaining a current frame time corresponding to the current frame bounding box, the current frame time being an acquisition time interval between the current frame of point cloud and the previous frame of point cloud.

[0128] S3414, determining a product of the current frame offset speed and the current frame time as the current frame offset of the current frame bounding box corresponding to the previous frame bounding box.

[0129] Specifically, the calculation formula of the current frame offset is:

[0130] L i =T i ×V i =T i ×(Li-1 / T i-1 );

[0131] T i = t1-t i-1 ;

[0132] L i-1 = l i-1 -l i-2 ;

[0133] T i-1 = t i-1 -t i-1 ;

[0134] wherein, L i is a current frame offset (i.e., i-th frame offset), T i is a current frame time (i.e., i-th frame time), V i is a current frame offset speed (i.e., i-th frame offset speed); L i-1 is a previous frame offset of a position l i-1 of a previous frame bounding box relative to a position l i-2 of a second previous frame bounding box, T i-1 is a collection time interval between a collection time t i-1 of the previous frame point cloud and a collection time t i-2 of the second previous frame point cloud, i.e., a previous frame time; T i is a collection time interval between a collection time t i of the current frame point cloud and the collection time t i-1 of the second previous frame point cloud, i.e., a current frame time.

[0135] In the process of calculating the previous frame offset corresponding to the previous frame bounding box in step S3411, the previous frame bounding box can be a defective box, which can be caused by false detection due to the sparsity of the point cloud, and is not conducive to subsequent identification and labeling of the point cloud in the current frame bounding box. Therefore, the defective bounding box can be completed to obtain a completed box, and then the previous frame offset of the completed box is calculated as the previous frame offset of the previous frame bounding box. In an optional embodiment, the previous frame offset corresponding to the previous frame bounding box of the previous frame point cloud is obtained, comprising:

[0136] Step 1), if the previous frame bounding box corresponding to the previous frame point cloud is a defective box, obtaining a previous frame completed box corresponding to the previous frame bounding box; wherein the previous frame completed box is obtained by completing the previous frame bounding box using a preset completion strategy.

[0137] wherein, the preset completion strategy is a completion strategy prepared in advance for the defective bounding box.

[0138] Specifically, if the previous frame bounding box corresponding to the previous frame point cloud is a defective box, a preset completion strategy is used to complete the previous frame bounding box to obtain a previous frame completed box.

[0139] Step 2), if the previous second frame bounding box corresponding to the previous frame point cloud is a defective box, a current frame completed box corresponding to the current frame bounding box is obtained by using the preset completion strategy to complete the previous second frame bounding box.

[0140] Specifically, if the previous second frame bounding box corresponding to the previous frame point cloud is a defective box, a preset completion strategy is used to complete the previous second frame bounding box to obtain a previous second frame completed box.

[0141] Step 3), a previous frame center offset of the previous frame completed box relative to the previous second frame completed box is calculated, and the previous frame center offset of the previous frame completed box is determined as a previous frame offset of the previous frame bounding box.

[0142] Specifically, a position difference between a center point position of the previous frame completed box and a center point position of the previous second frame completed box is determined as the previous frame center offset of the previous frame completed box relative to the previous second frame completed box, and the previous frame center offset is determined as the previous frame offset of the previous frame bounding box.

[0143] It should be noted that the preset completion strategy used for completing the previous frame bounding box to obtain the previous frame completed box and completing the previous second frame bounding box to obtain the previous second frame completed box can be the preset completion strategy described in Embodiment 4.

[0144] The embodiment can reduce errors caused by target motion by calculating the center offset of the completed box to determine the expansion coefficient, and can reduce false detection caused by the sparsity of point clouds by calculating the offset of the bounding box after completing the defective bounding box, which is beneficial to subsequent identification and labeling of point clouds in the current frame bounding box.

[0145] Figure 4A A flowchart of a target labeling method provided by the embodiment of the present application, the embodiment further limits the case where the current bounding box is defective in the “for each current frame in the continuous multiple frames, determining a current frame expansion box of the target according to a current frame bounding box of a current frame point cloud containing the target and an expansion coefficient of the current frame bounding box obtained” in any of the above embodiments. As shown in the figure, the method comprises: Figure 4A

[0146] S410, obtaining continuous multiple frames of point clouds of a target and target labeling information of a first frame of point clouds.

[0147] S420, for each current frame in the continuous multiple frames, obtaining a current frame bounding box of a current frame point cloud containing the target.​

[0148] S430, if the current frame bounding box is a defective box, a preset completion strategy is used to complete the current frame bounding box of the current frame point cloud containing the target to obtain a current frame completion box of the target.

[0149] The current frame completion box can be understood as a box obtained by completing the defective current frame bounding box in the current frame.

[0150] Specifically, in the case that the target point cloud is occluded, the current frame bounding box determined may be a defective box, which will inevitably affect the recognition of the target, so it is necessary to complete the defective current frame bounding box. Specifically, the size of the current frame bounding box and the preset size corresponding to the type of the target can be compared to determine whether there is a large size difference between them. If the size difference is greater than a preset value, it is considered that the current frame bounding box is a defective box, and a preset completion strategy is used to complete the current frame bounding box, thereby obtaining the current frame completion box of the target. Otherwise, it is considered that the current frame bounding box is not a defective box, and no completion operation is needed.

[0151] This step completes the defective current frame bounding box, reduces the false detection caused by the sparsity of the point cloud, and can obtain more stable target size and speed state recognition, which is beneficial to the recognition and labeling of the point cloud in the current frame bounding box.

[0152] S440, according to the expansion coefficient of the obtained current frame bounding box, the current frame completion box is expanded to obtain a current frame expansion box containing the target.

[0153] Specifically, the expansion coefficient corresponding to the current frame bounding box is obtained, and then the current frame completion box is expanded using the expansion coefficient to obtain the current frame expansion box of the target.

[0154] It should be noted that the expansion coefficient used to expand the current frame completion box in this step is the same as the expansion coefficient used to expand the current frame bounding box, and the acquisition method is also the same. The present embodiment will not be repeated here.

[0155] S450, projecting the current frame expansion box onto the next frame point cloud to obtain a next frame prediction box.

[0156] S460, clustering the next frame point cloud in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box;

[0157] S470, tracking the target in the current frame bounding box corresponding to each current frame point cloud in the continuous multiple frames of point clouds until a tracking termination condition is reached, and labeling the target in the continuous multiple frames of point clouds using the target labeling information.

[0158] The technical scheme of the embodiment of the present application is that, by the incomplete current frame boundary box, a preset completion strategy is used to complete the current frame boundary box of the current frame point cloud containing the target to obtain a current frame completion box of the target; the current frame completion box is expanded according to an expansion coefficient to obtain a current frame expansion box containing the target, which is beneficial to subsequent projection and clustering of the current frame expansion box to obtain a next frame boundary box, can realize reasonable prediction of the position and size of the next frame boundary box according to the motion information of the target, makes the size of the next frame boundary box selected on the basis of the complete point cloud containing the target, reduces the noise contained in the boundary box, can obtain more stable target size and speed state recognition, reduces the false detection caused by the sparse characteristics of the point cloud, improves the subsequent clustering effect, realizes matching of the boundary box with the adaptive size according to the motion information of the target, and improves the target labeling accuracy based on the boundary box.

[0159] In an optional embodiment, the preset completion strategy comprises:

[0160] Step 1), obtaining a self-vehicle coordinate system, and determining a connecting line between each corner point of the incomplete boundary box and the origin of the self-vehicle coordinate system;

[0161] Step 2), obtaining a target connecting line without intersection with other boundary boxes in the frame in which the incomplete boundary box is located, and determining a corner point corresponding to the target connecting line as a credible corner point;

[0162] Step 3), determining a to-be-completed frame line and a completion direction of the incomplete boundary box according to the number of the credible corner points;

[0163] Step 4), determining a completion length of the to-be-completed frame line of the incomplete boundary box in the completion direction according to a preset size corresponding to the type of the target;

[0164] Step 5), completing the incomplete boundary box along the completion direction of the to-be-completed frame line according to the completion length.

[0165] In the present application, the incomplete boundary box can be any one of the above-mentioned existing incomplete current frame boundary box, previous frame boundary box or previous two frame boundary box. It can be understood that, due to the occlusion of the target and other reasons, the detected boundary box is an incomplete boundary box, but it is still a rectangular boundary box, only the size of the boundary box determined according to the actual target will have some difference.

[0166] Specifically, the basic idea of the preset completion strategy is to determine the to-be-completed frame, the completion direction and the completion length of the incomplete boundary box, and complete the to-be-completed frame line of the incomplete boundary box in the completion direction by using the completion length.

[0167] Wherein, a self-vehicle coordinate system can be constructed with the setting position of the radar collecting the point cloud as the origin, and the coordinate axis of the self-vehicle coordinate system can be the lateral center line or the longitudinal center line of the self-vehicle body, and the present embodiment does not limit this.

[0168] The trusted corner point can be understood as a corner point that is not blocked by other bounding boxes, that is, this corner point can be trusted to represent the intersection of the real bounding box, rather than the intersection point obtained by extending two non-intersecting frame lines, and therefore is called a trusted corner point.

[0169] For example, the main idea of determining the frame line to be completed is to take the frame line in the incomplete bounding box that is not blocked and closest to the origin of the self-vehicle coordinate system as the frame line to be completed. Specifically, whether the line connecting the corner point and the origin of the self-vehicle coordinate system intersects with other bounding boxes in the frame in which the incomplete bounding box is located can be used to determine whether the two frame lines connected by the corner point are blocked by other bounding boxes. If there is an intersection, it means that the frame line connected by the corner point corresponding to the intersection line can be blocked by other bounding boxes and cannot be used as the frame line to be completed. Therefore, the line connecting the corner point corresponding to the target line that does not intersect with other bounding boxes is determined as the target line, and the corner point corresponding to the target line is determined as the trusted corner point. There are four corner points in an incomplete bounding box, and the number of blocked corner points is 0-4, that is, the number of trusted corner points can be 0-4. If the number of trusted corner points is at least one, it means that at least one corner point of the incomplete bounding box is not blocked. If the number of trusted corner points is 0, it means that all four corner points of the incomplete bounding box are blocked. Therefore, the frame line to be completed of the incomplete bounding box and the corresponding completion direction can be further determined according to the number of trusted corner points. There can be one or two completion directions corresponding to the frame line to be completed. For example, the total completion length can be obtained by obtaining the corresponding preset size according to the type of the target obtained from the target labeling information of the first frame of point cloud, such as a first preset size corresponding to a motor vehicle, a second preset size corresponding to a pedestrian, and then comparing the preset size and the size of the incomplete bounding box to determine the total completion length. The total completion length can be a length value or a length range. Finally, the completion length of the frame line to be completed in each completion direction is determined according to the completion direction corresponding to the frame line to be completed and the total completion length.

[0170] Optionally, step 3) comprises:

[0171] Step 311) If there is at least one trusted corner point, the trusted corner point closest to the origin is determined as the first target corner point.

[0172] Step 312), connecting two frame lines connected by the first target corner point in the incomplete boundary frame as a first to-be-completed frame line and a second to-be-completed frame line;

[0173] Step 313), determining a direction of the first target corner point towards another end point of the first to-be-completed frame line as a completion direction of the first to-be-completed frame line;

[0174] Step 13), determining a direction of the first target corner point towards another end point of the second to-be-completed frame line as a completion direction of the second to-be-completed frame line. Specifically, if there is at least one trusted corner point, the trusted corner point closest to the origin (i.e., the shortest connecting line) is determined as the first target corner point. It can be understood that if there is only one trusted corner point, the trusted corner point is the first target corner point. Since the first target corner point is the unobstructed corner point closest to the origin, the two frame lines connected by the first target corner point in the incomplete boundary frame can be determined as the first to-be-completed frame line and the second to-be-completed frame line, respectively.

[0175] In determining the to-be-completed frame line, the completion direction of the to-be-completed frame line needs to be further determined. Since the first target corner point is a trusted real corner point in the incomplete boundary frame, and the first target corner point is a common end point of the first to-be-completed frame line and the second to-be-completed frame line, a direction of the first target corner point towards another end point of the first to-be-completed frame line is determined as a completion direction of the first to-be-completed frame line; a direction of the first target corner point towards another end point of the second to-be-completed frame line is determined as a completion direction of the second to-be-completed frame line.

[0176] Exemplarily, Figure 4B is a schematic diagram of a method for completing an incomplete boundary frame. As shown in Figure 4B , the incomplete boundary frame 41, i.e., the rectangular frame ABCD, obtains the connecting lines of the four corner points A, B, C and D of the incomplete boundary frame 41 and the origin O of the ego coordinate system, i.e., the dashed lines OA, OB, OC and OD. As can be seen, the connecting line OB of the corner point B and the origin O intersects with another boundary frame 40 in the frame where the incomplete boundary frame 41 is located, and therefore, it is considered that the corner point B is obstructed, while the corner points A, C and D are considered to be trusted corner points. The corner point A is determined as the first target corner point from the trusted corner points because it is closest to the origin. The two frame lines AB and AD connected by the first target corner point A are the first to-be-completed frame line 42 and the second to-be-completed frame line 43, respectively. The completion direction of the first to-be-completed frame line 42 is determined as the AB direction; and the completion direction of the second to-be-completed frame line 43 is determined as the AD direction.

[0177] If the target type corresponds to a preset size of 5cm x 8cm and the size of the incomplete bounding box 41 is 4cm x 6cm, the first to-be-completed frame line 42 is completed by 1cm along the AB direction, and the second to-be-completed frame line 43 is completed by 2cm along the AD direction, to obtain the completed bounding box 44.

[0178] Optionally, step 3) comprises:

[0179] Step 321) If the number of the trusted corner points is zero, the corner point closest to the origin on the incomplete bounding box is determined as a second target corner point.

[0180] Step 322) The two frame lines connected by the second target corner point in the incomplete bounding box are determined as a first to-be-completed frame line and a second to-be-completed frame line, respectively.

[0181] Step 323) The direction of the second target corner point towards the other end point of the first to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the first to-be-completed frame line, respectively.

[0182] Step 324) The direction of the second target corner point towards the other end point of the second to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the second to-be-completed frame line, respectively.

[0183] Specifically, if there are zero trusted corner points, that is, all four corner points of the incomplete bounding box are occluded, the corner point closest to the origin (that is, the shortest connecting line) on the incomplete bounding box is determined as a second target corner point. The two frame lines connected by the second target corner point in the incomplete bounding box can be determined as a first to-be-completed frame line and a second to-be-completed frame line, respectively. Since the second target corner point is actually an occluded corner point, it is not an actual corner point in the bounding box, and therefore the first to-be-completed frame line and the second to-be-completed frame line can be completed towards the two ends. Accordingly, the direction of the second target corner point towards the other end point of the first to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the first to-be-completed frame line, respectively; the direction of the second target corner point towards the other end point of the second to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the second to-be-completed frame line, respectively.

[0184] Exemplarily, Figure 4C is another schematic view of a method for completing an incomplete bounding box. As Figure 4CAs shown, the broken bounding box 41, i.e. the rectangular box ABCD, obtains the lines connecting the four corner points A, B, C and D of the broken bounding box 41 to the origin O of the ego vehicle coordinate system, i.e. the dashed lines OA, OB, OC and OD. It can be seen that the lines connecting the four corner points A, B, C and D to the origin O all intersect with another bounding box 40 in the frame where the broken bounding box 41 is located, and therefore it is considered that the four corner points are all occluded, and the number of reliable corner points is zero. The corner point A is determined to be the second target corner point from the four corner points of the broken bounding box 41 because it is closest to the origin O. The two frame lines AB and AD connected by the second target corner point A are the first to-be-completed frame line 42 and the second to-be-completed frame line 43 respectively. The first completion direction of the first to-be-completed frame line 42 is determined to be the AB direction, and the second completion direction is determined to be the BA direction; the first completion direction of the second to-be-completed frame line 43 is determined to be the AD direction, and the second direction is determined to be the DA direction.

[0185] If the preset size corresponding to the type of the target is 5cm x 8cm, and the size of the broken bounding box 41 is 4cm x 6cm, then the first to-be-completed frame line 42 is completed by 0.5cm along the AB direction and 0.5cm along the BA direction; the second to-be-completed frame line 43 is completed by 1cm along the AD direction and 1cm along the DA direction, to obtain the completed bounding box 44.

[0186] Since the collected point cloud of the target can be a two-dimensional point cloud or a three-dimensional point cloud, if the target point cloud is a three-dimensional point cloud, i.e. the determined bounding box is a three-dimensional solid frame, the height of the broken bounding box can also be completed by using a preset completion strategy. In an optional embodiment, the preset completion strategy further comprises:

[0187] Step 5), if the broken bounding box is a three-dimensional solid frame, determining a target completion height of the broken bounding box according to the preset size corresponding to the type of the target;

[0188] Step 6), expanding a height frame line of the broken bounding box in the direction of the ground by a first actual completion height, so that any frame line of the broken bounding box contacts the ground level or the first actual completion height is equal to the target completion height;

[0189] Step 7), if any frame line of the broken bounding box contacts the ground level and the first actual completion height is less than the target completion height, expanding the height frame line of the broken bounding box in the direction opposite to the ground by a second actual completion height, and the second actual completion height is the difference between the target completion height and the first actual completion height.

[0190] Specifically, the basic idea of completing the incomplete bounding box as a three-dimensional frame is to determine the actual completion height and the height completion direction, and to complete the height of the incomplete bounding box according to the actual completion height and the height completion direction. The height completion direction includes: completing towards the ground and completing in the opposite direction of the ground.

[0191] The way to determine the actual completion height is to determine the target completion height of the incomplete bounding box according to the preset size corresponding to the type of the target, for example, the type of the target is a motor vehicle, and the first preset size corresponding to the motor vehicle is obtained; the type of the target is a pedestrian, and the second preset size corresponding to the pedestrian is obtained; and then the target completion height is determined according to the preset size. The target completion height can be a length value or a length range.

[0192] When completing the height frame line of the incomplete bounding box, the height frame line of the incomplete bounding box is first expanded by a first actual completion height towards the ground; if the first actual completion height expanded towards the ground is equal to the target completion height, and any frame line of the incomplete bounding box does not contact the ground level, the completion of the height frame line of the incomplete bounding box is completed. If the first actual completion height expanded towards the ground is less than the target completion height, but any frame line of the incomplete bounding box has contacted the ground level. Since the target should be at least above the ground, it cannot continue to expand towards the ground, and at this time the height frame line of the incomplete bounding box is expanded by a second actual completion height towards the ground in the opposite direction, that is, the second actual completion height is calculated according to the difference between the target completion height and the first actual completion height, to avoid the expansion of the bounding box to the underground.

[0193] The embodiment of the present application completes the height of the incomplete bounding box of the three-dimensional frame, and considers the actual existing scene of the target in the completion process, so that the completed bounding box more truly reflects the actual position of the target.

[0194] Figure 5 A flowchart of a bounding box completion method provided by the embodiment of the present application, the embodiment can be applicable to the case of independently completing the incomplete bounding box, and the method can be executed by a bounding box completion device. The bounding box completion device can be realized in the form of hardware and / or software, and can be configured in an electronic device. The electronic device can be a vehicle-mounted device arranged on a vehicle, or a device independent of the vehicle. As shown in the figure, the method comprises: Figure 5

[0195] S510, obtaining a current frame bounding box corresponding to a current frame point cloud of a target.

[0196] ​For example, the first frame point cloud is manually labeled to set the current frame bounding box of the current frame point cloud containing the target. For non-first frame point cloud, a regular target detection algorithm is used to set the detection box in the current frame point cloud as the current frame bounding box, which is not described and limited herein.

[0197] In S520, if the current frame bounding box is a defective box, a preset completion strategy is used to complete the current frame bounding box to obtain the current frame completion box of the target.

[0198] The current frame completion box can be understood as a box obtained by completing the defective current frame bounding box in the current frame.

[0199] Specifically, due to occlusion or point cloud sparsity, the outline may be incomplete, so that the determined current frame bounding box deviates from the actual box, that is, the current frame bounding box is a defective box. The size-unstable bounding box will affect the speed state calculation, so it is necessary to complete the defective current frame bounding box. Specifically, the current frame bounding box can be completed by using a preset completion strategy, so as to obtain the current frame completion box of the target.

[0200] It should be noted that the preset completion strategy used in the embodiment of the present application can use the preset completion strategy shown in the above embodiment, which is not described herein. In addition, the completion method of the bounding box provided by the embodiment of the present application is limited to the scene of completing the current frame bounding box to obtain the current frame completion box. The defective bounding box obtained in any bounding box detection scene can be applied.

[0201] The completion method of the bounding box of the embodiment of the present application does not need to rely on the matching and association of multiple frames, and directly completes the size completion of the defective bounding box of the target on a single frame. Compared with the method of labeling and generating training samples by manually completing the size of the target, and then sending them into a neural network to learn and infer to realize the completion of the bounding box, the automatic completion based on the preset completion strategy can be realized without relying on manual completion of each frame, which saves more manpower and financial resources.

[0202] The technical scheme of the embodiment of the present application, by obtaining the current frame bounding box corresponding to the current frame point cloud of the target; if the current frame bounding box is a defective box, the current frame bounding box is completed by using a preset completion strategy to obtain the current frame completion box of the target; completing the defective current frame bounding box can reduce the false detection caused by occlusion or point cloud sparsity characteristics, obtain more stable target size and speed state recognition, and is beneficial to the recognition and labeling of the point cloud in the current frame bounding box.

[0203] Figure 6 A structure diagram of a target labeling device provided by the embodiment of the present application is shown in FIG. 1. Figure 6 As shown in the figure, the device comprises:

[0204] The point cloud acquisition module 610 is configured to acquire continuous multiple frames of point clouds of a target and target labeling information of a first frame of point clouds;

[0205] The expansion module 620 is configured to, for each current frame in the continuous multiple frames, determine a current frame expansion bounding box of the target according to a current frame bounding box of the current frame of point clouds containing the target and an expansion coefficient of the current frame bounding box acquired;

[0206] The projection module 630 is configured to project the current frame expansion bounding box onto a next frame of point clouds to obtain a next frame prediction bounding box;

[0207] The clustering module 640 is configured to cluster the next frame of point clouds in the next frame prediction bounding box to obtain a next frame clustering bounding box of the target, and take the next frame clustering bounding box as a new current frame bounding box;

[0208] The labeling module 650 is configured to perform target association and tracking on the current frame bounding box corresponding to each current frame of point clouds in the continuous multiple frames of point clouds until a tracking termination condition is reached, and perform target labeling on the continuous multiple frames of point clouds by using the target labeling information.

[0209] Optionally, the clustering module 640 comprises:

[0210] The first clustering unit is configured to, if the type of the target is a vehicle, perform iterative rotation on the next frame of point clouds in the next frame prediction bounding box under a next frame coordinate system to obtain a clustering direction, and cluster the next frame of point clouds in the next frame prediction bounding box according to the clustering direction to obtain the next frame clustering bounding box of the target; the next frame prediction bounding box coordinate system is a coordinate system constructed with the center of the next frame prediction bounding box as the origin;

[0211] The second clustering unit is configured to, if the type of the target is a pedestrian, cluster the next frame of point clouds in the next prediction bounding box according to a center point offset value of the next frame prediction bounding box to obtain the next frame clustering bounding box of the target.

[0212] Optionally, the first clustering unit is specifically configured to:

[0213] perform iterative rotation on the next frame of point clouds in the next frame prediction bounding box based on a preset iterative rotation strategy; wherein the preset iterative rotation strategy comprises rotating by a preset unit rotation angle in a clockwise direction and an anticlockwise direction for a preset number of times;

[0214] construct a minimum bounding box of the next frame point cloud after each rotation, and select a preset number of judgment points on the minimum bounding box to determine the average minimum distance between each judgment point and the next frame point cloud; the average minimum distance is the average value of the minimum distance between each judgment point and the next frame point cloud;

[0215] After completing the iterative rotation, the average minimum distance corresponding to each rotation is determined, and the minimum angle in the determined angles is determined as the clustering direction.

[0216] Optionally, if the type of the target is a motor vehicle, the preset number of judgment points includes: four vertices of the minimum bounding box and three equidistant points on each frame line of the minimum bounding box;

[0217] If the type of the target is a non-motor vehicle, the preset number of judgment points includes: four vertices of the minimum bounding box and two equidistant points on any two parallel frame lines in the minimum bounding box.

[0218] Optionally, the device further comprises:

[0219] The filtering module is configured to perform noise point filtering on the next frame point cloud in the next frame prediction box after projecting the current frame extended bounding box onto the next frame point cloud and before clustering the next frame point cloud in the next frame prediction box to obtain the next frame clustering box of the target; wherein the noise points include ground noise points and abnormal point clouds in the next frame prediction box.

[0220] Optionally, the extension module 620 comprises:

[0221] The first coefficient acquisition unit is configured to acquire a first extension coefficient of the current frame bounding box if the current frame bounding box is a bounding box corresponding to the first frame point cloud.

[0222] The second coefficient acquisition unit is configured to determine a second extension coefficient of the current frame bounding box according to a previous frame offset of the current frame bounding box relative to a previous frame bounding box if the current frame bounding box is a bounding box corresponding to a non-first frame point cloud.

[0223] Optionally, the first coefficient acquisition unit is specifically configured to:

[0224] determine the type of the target according to the target annotation information of the first frame point cloud;

[0225] determine the dynamic state of the target according to the current frame point cloud and the next frame point cloud;

[0226] determine the first extension coefficient of the target according to the dynamic state and the type of the target.

[0227] Optionally, the second coefficient obtaining unit comprises:

[0228] an offset amount calculating subunit configured to calculate a current frame offset amount of the current frame bounding box relative to the previous frame bounding box;

[0229] a coefficient determining subunit configured to input the current frame offset amount into a target function to obtain a second expansion coefficient of the current frame bounding box; wherein the target function is configured to express a functional relationship between the offset amount and the second expansion coefficient.

[0230] Optionally, the offset amount calculating subunit is specifically configured to:

[0231] if the previous frame point cloud is a non-first frame point cloud, obtain a previous frame offset amount and a previous frame time corresponding to a previous frame bounding box of the previous frame point cloud; wherein the previous frame offset amount is an offset amount of the previous frame bounding box relative to a second previous frame bounding box; and the previous frame time is a time interval between the previous frame point cloud and a second previous frame point cloud;

[0232] determine a ratio of the previous frame offset amount and the previous frame time as a current frame offset speed of the current frame bounding box;

[0233] obtain a current frame time corresponding to the current frame bounding box; wherein the current frame time is a time interval between the current frame point cloud and the previous frame point cloud;

[0234] determine a product of the current frame offset speed and the current frame time as the current frame offset amount of the current frame bounding box relative to the previous frame bounding box.

[0235] Optionally, the offset amount calculating subunit is specifically configured to:

[0236] if the previous frame bounding box corresponding to the previous frame point cloud is a defective bounding box, obtain a previous frame complete bounding box corresponding to the previous frame bounding box; wherein the previous frame complete bounding box is obtained by complementing the previous frame bounding box according to a preset complementing strategy;

[0237] if the second previous frame bounding box corresponding to the previous frame point cloud is a defective bounding box, obtain a current frame complete bounding box corresponding to the current frame bounding box; wherein the second previous frame complete bounding box is obtained by complementing the second previous frame bounding box according to the preset complementing strategy;

[0238] calculate a previous center offset amount of the previous frame complete bounding box relative to the second previous frame complete bounding box;

[0239] determine the previous center offset amount of the previous frame complete bounding box as the previous frame offset amount of the previous frame bounding box.

[0240] Optionally, the expansion module 620 comprises:

[0241] the current frame bounding box is a defective box, a preset completion strategy is adopted to complete the current frame bounding box of the current frame point cloud containing the target to obtain a current frame completion box of the target;

[0242] the current frame completion box is expanded according to the expansion coefficient of the current frame bounding box to obtain a current frame expansion box containing the target.

[0243] Optionally, the preset completion strategy comprises:

[0244] an ego-vehicle coordinate system is obtained, and a line connecting each corner point of the defective bounding box and an origin of the ego-vehicle coordinate system is determined;

[0245] a target line without intersection with other bounding boxes in the frame in which the defective bounding box is located is obtained, and a corner point corresponding to the target line is determined as a trusted corner point;

[0246] a to-be-completed box line of the defective bounding box and a corresponding completion direction are determined according to the number of trusted corner points;

[0247] a completion length of the to-be-completed box line of the defective bounding box in the completion direction is determined according to a preset size corresponding to the type of the target;

[0248] the defective bounding box is completed along the completion direction of the to-be-completed box line according to the completion length.

[0249] Optionally, the determining the to-be-completed box line of the defective bounding box and the corresponding completion direction according to the number of trusted corner points comprises: if there is at least one trusted corner point, a trusted corner point closest to the origin is determined as a first target corner point.

[0250] two box lines connected by the first target corner point in the defective bounding box are determined as a first to-be-completed box line and a second to-be-completed box line;

[0251] a direction of the first target corner point towards another end point of the first to-be-completed box line is determined as a completion direction of the first to-be-completed box line.

[0252] a direction of the first target corner point towards another end point of the second to-be-completed box line is determined as a completion direction of the second to-be-completed box line. Optionally, the determining the to-be-completed box line of the defective bounding box and the corresponding completion direction according to the number of trusted corner points comprises:

[0253] if the number of trusted corner points is zero, a corner point closest to the origin on the defective bounding box is determined as a second target corner point.

[0254] The two frame lines connected by the second target corner point in the incomplete bounding box are determined as a first to-be-completed frame line and a second to-be-completed frame line respectively.

[0255] The direction of the second target corner point towards the other end point of the first to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the first to-be-completed frame line respectively.

[0256] The direction of the second target corner point towards the other end point of the second to-be-completed frame line and the corresponding opposite direction are determined as a first completion direction and a second completion direction of the second to-be-completed frame line respectively.

[0257] Optionally, the preset completion strategy further comprises:

[0258] If the incomplete bounding box is a three-dimensional solid box, a target completion height of the incomplete bounding box is determined according to a preset size corresponding to the type of the target.

[0259] The height frame line of the incomplete bounding box is extended in the direction of the ground by a first actual completion height, so that any frame line of the incomplete bounding box contacts the horizon or the first actual completion height is equal to the target completion height.

[0260] If any frame line of the incomplete bounding box contacts the horizon and the first actual completion height is less than the target completion height, the height frame line of the incomplete bounding box is extended in the direction opposite to the ground by a second actual completion height, and the second actual completion height is the difference between the target completion height and the first actual completion height.

[0261] The target labeling device provided by the embodiment of the application can execute the target labeling method provided by any embodiment of the application, has the corresponding function modules and beneficial effects of the execution method.

[0262] Figure 7 A structure diagram of a bounding box completion device provided by the embodiment of the application is shown in FIG. 7. Figure 7 As shown in the figure, the device comprises:

[0263] The bounding box acquisition module 710 is configured to acquire a current frame bounding box corresponding to a current frame point cloud of a target.

[0264] The completion module 720 is configured to, if the current frame bounding box is an incomplete box, complete the current frame bounding box by using a preset completion strategy to obtain a current frame completed box of the target.

[0265] Optionally, the preset completion strategy comprises:

[0266] A completion length of the bounding box is determined according to a preset size corresponding to the type of the target.

[0267] determine a target quadrant and a target angle interval of the bounding box in a self-vehicle coordinate system, wherein the self-vehicle coordinate system is a coordinate system constructed with a setting position of a radar collecting the point cloud as an origin, and the target angle interval is a preset angle interval to which an included angle between a center line of the bounding box and a coordinate axis of the self-vehicle coordinate system belongs;

[0268] determine a to-be-completed frame line of the bounding box and a corresponding completion direction according to the target quadrant and the target angle interval corresponding to the bounding box;

[0269] complete the to-be-completed frame line on the bounding box according to the completion direction and the completion length.

[0270] Optionally, the determining the frame line of the bounding box and the corresponding completion direction according to the target quadrant and the target angle interval corresponding to the bounding box comprises:

[0271] determining a to-be-completed frame line of the bounding box in the target angle interval, wherein the to-be-completed frame line is two adjacent frame lines of the bounding box closest to the origin of the self-vehicle coordinate system and falling in the target angle interval;

[0272] querying a completion direction table according to the target quadrant and the to-be-completed frame line corresponding to the bounding box to obtain the completion direction of the bounding box.

[0273] Optionally, the preset completion strategy further comprises:

[0274] if the bounding box is a three-dimensional solid box, determining a target completion height of the bounding box according to a preset size corresponding to the type of the target;

[0275] extending a height frame line of the bounding box in a ground direction by a first actual completion height, so that any frame line of the bounding box contacts a ground level or the first actual completion height is equal to the target completion height;

[0276] if any frame line of the bounding box contacts the ground level and the first actual completion height is less than the target completion height, extending the height frame line of the bounding box in a direction opposite to the ground by a second actual completion height, wherein the second actual completion height is a difference between the target completion height and the first actual completion height.

[0277] The completion device of the bounding box provided in the embodiments of the application can execute the completion method of the bounding box provided in any of the embodiments of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0278] Figure 8A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, vehicle-mounted devices, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0279] As shown in Figure 8 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected in communication with the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0280] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0281] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the labeling method of the target, or the completion method of the bounding box.

[0282] In some embodiments, the object labeling method, or the bounding box completion method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded onto the electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the object labeling method, or the bounding box completion method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the object labeling method, or the bounding box completion method by any other suitable means, e.g., by means of firmware.

[0283] The various implementations of the system and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0284] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flowcharts and / or the block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0285] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0286] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0287] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0288] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0289] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and this is not limited herein.

[0290] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for labeling a target, characterized by, The method comprises the following steps: obtaining a plurality of continuous frames of point clouds of a target and target annotation information of a first frame of point clouds; for each current frame in the plurality of continuous frames, determining a current frame extended bounding box of the target according to a current frame bounding box of a current frame of point clouds containing the target and an extension coefficient of the current frame bounding box obtained; projecting the current frame extended bounding box onto a next frame of point clouds to obtain a next frame prediction box; performing clustering on the next frame of point clouds in the next frame prediction box to obtain a next frame clustering box of the target, and taking the next frame clustering box as a new current frame bounding box; tracking the target according to the current frame bounding box corresponding to each current frame of point clouds in the plurality of continuous frames of point clouds until a tracking termination condition is reached, and annotating the target according to the target annotation information; the step of performing clustering on the next frame of point clouds in the next frame prediction box to obtain the next frame clustering box of the target comprises: if the type of the target is a vehicle, performing iterative rotation on the next frame of point clouds in the next frame prediction box to obtain a clustering direction, and performing clustering on the next frame of point clouds in the next frame prediction box according to the clustering direction to obtain the next frame clustering box of the target.

2. The method of claim 1, wherein, the step of performing clustering on the next frame of point clouds in the next frame prediction box to obtain the next frame clustering box of the target comprises: if the type of the target is a pedestrian, performing clustering on the next frame of point clouds in the next frame prediction box according to a center point offset value of the next frame prediction box to obtain the next frame clustering box of the target.

3. The method of claim 2, wherein, the step of performing iterative rotation on the next frame of point clouds in the next frame prediction box to obtain a clustering direction comprises: performing iterative rotation on the next frame of point clouds in the next frame prediction box based on a preset iterative rotation strategy; wherein the preset iterative rotation strategy comprises: rotating by a preset unit rotation angle in a clockwise direction and a counterclockwise direction for a preset number of times; after each rotation, constructing a minimum bounding box of the next frame of point clouds, and selecting a preset number of judgment points on the minimum bounding box to determine the average minimum distance between each judgment point and the next frame of point clouds; the average minimum distance is the average value of the minimum distance between each judgment point and the next frame of point clouds; after completing the iterative rotation, determining the angle corresponding to the average minimum distance obtained in each rotation, and determining the minimum angle among the determined angles as the clustering direction.

4. The method of claim 3, wherein, if the type of the target is a motor vehicle, the preset number of judgment points comprises: four vertices of the minimum bounding box and three equidistant points on each frame line of the minimum bounding box; if the type of the target is a non-motor vehicle, the preset number of judgment points comprises: four vertices of the minimum bounding box and two equidistant points on any two parallel frame lines in the minimum bounding box.

5. The method of claim 1, wherein, the step of obtaining the extension coefficient of the current frame bounding box comprises: if the current frame bounding box is the bounding box corresponding to the first frame of point clouds, obtaining a first extension coefficient of the current frame bounding box; if the current frame bounding box is the bounding box corresponding to a non-first frame of point clouds, determining a second extension coefficient of the current frame bounding box according to a previous frame offset of the current frame bounding box relative to a previous frame bounding box.

6. The method of claim 1, wherein, For each current frame in the continuous multiple frames, a current frame bounding box of a current frame point cloud containing the target and an expansion coefficient of the current frame bounding box are obtained, and a current frame expansion bounding box of the target is determined according to the current frame bounding box and the expansion coefficient of the current frame bounding box. For each current frame in the continuous multiple frames, a current frame bounding box of a current frame point cloud corresponding to the target is obtained. If the current frame bounding box is a defective bounding box, a preset completion strategy is used to complete the current frame bounding box to obtain a current frame completion bounding box of the target. The current frame completion bounding box is expanded according to the expansion coefficient of the obtained current frame bounding box to obtain a current frame expansion bounding box containing the target.

7. An apparatus for labeling a target, comprising: The device comprises: a point cloud acquisition module configured to acquire target annotation information of a continuous multiple frames of point clouds and a first frame of point clouds of a target; an expansion module configured to, for each current frame in the continuous multiple frames, determine a current frame expansion bounding box of the target according to a current frame bounding box of a current frame point cloud containing the target and an expansion coefficient of the current frame bounding box; a projection module configured to project the current frame expansion bounding box onto a next frame point cloud to obtain a next frame prediction bounding box; a clustering module configured to cluster a next frame point cloud in the next frame prediction bounding box to obtain a next frame clustering bounding box of the target, and take the next frame clustering bounding box as a new current frame bounding box; an annotation module configured to perform target association and tracking on current frame bounding boxes corresponding to each current frame point cloud in the continuous multiple frames of point clouds until a tracking termination condition is reached, and perform target annotation on the continuous multiple frames of point clouds using the target annotation information; the clustering module comprises: a first clustering unit configured to, if the type of the target is a vehicle, perform iterative rotation on a next frame point cloud in the next frame prediction bounding box in a next frame coordinate system to obtain a clustering direction, and cluster the next frame point cloud in the next frame prediction bounding box according to the clustering direction to obtain a next frame clustering bounding box of the target; the next frame prediction bounding box coordinate system is a coordinate system constructed with the center of the next frame prediction bounding box as the origin.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the target annotation method of any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the target annotation method of any one of claims 1-5 when executed.

Images