Target labeling method of three-dimensional point cloud and target tracking method in point cloud continuous frames

Abstract

Embodiments of the present application provide a target labeling method of a three-dimensional point cloud and a target tracking method in a point cloud continuous frame. The labeling method comprises: for a to-be-labeled point cloud in a first region in the three-dimensional point cloud, performing ground point cloud filtering on the to-be-labeled point cloud based on the density of points in different parts of the to-be-labeled point cloud, to obtain filtered point cloud from which ground point cloud is removed; performing target segmentation on the filtered point cloud to determine target point cloud; and outputting position information of the determined target point cloud. This scheme has small calculation amount and can significantly improve labeling efficiency. At the same time, the scheme of performing ground point cloud filtering according to the density of points in different parts of the to-be-labeled point cloud can accurately remove ground point cloud, thereby improving the accuracy of the determined target point cloud, and the accuracy of target labeling is also high. Moreover, this scheme does not need to be trained, and the implementation cost is also low, so the scheme has good implementability and strong expandability.

Description

Technical Field

[0001] This application relates to the field of point cloud processing technology, specifically to a target annotation method for three-dimensional point clouds, a target tracking method in continuous frames of point clouds, a target annotation system for three-dimensional point clouds, a target tracking system in continuous frames of point clouds, an electronic device, and a storage medium. Background Technology

[0002] Point clouds are discrete sets of points distributed in N-dimensional space, primarily in three dimensions. They are discrete samples of information about the surface of an object, typically acquired by LiDAR scanning, and play an important role in fields such as autonomous driving.

[0003] In the field of point cloud processing technology, it is often necessary to identify and label targets in 3D point clouds. Taking autonomous driving as an example, autonomous vehicles mainly rely on LiDAR to sense the external environment to drive safely on the road. The 3D point cloud images generated by LiDAR, after annotation, can be used to train autonomous driving systems. As the performance of autonomous driving systems continues to improve, the amount of training sample data required also increases. Therefore, autonomous driving systems require a large amount of labeled 3D point cloud data.

[0004] Existing methods for labeling 3D point clouds typically include traditional manual labeling and intelligent labeling. Manual labeling usually involves manually marking points or drawing bounding boxes using labeling tools. This method is entirely reliant on manual labor, resulting in low efficiency and a high risk of errors. Intelligent labeling typically utilizes trained deep learning models to segment and label point clouds, achieving automated labeling and segmentation. However, this deep learning approach often requires a large amount of data to train the model, making it highly dependent on training samples. It struggles to achieve accurate labeling and segmentation on untrained data. Therefore, this deep learning-based intelligent point cloud labeling method is costly to implement and has relatively low accuracy. Summary of the Invention

[0005] To at least partially address the aforementioned problems in the prior art, according to one aspect of this application, a target annotation method for three-dimensional point clouds is provided, comprising: for a point cloud to be annotated in a first region of a three-dimensional point cloud, performing ground point cloud filtering on the point cloud to be annotated based on the density of points in different parts of the point cloud to obtain a filtered point cloud after removing ground point cloud; performing target segmentation on the filtered point cloud to determine a target point cloud; and outputting the location information of the determined target point cloud.

[0006] For example, ground point cloud filtering of the point cloud to be labeled is performed based on the point density of different parts of the point cloud to be labeled, including: dividing the point cloud to be labeled into a sparse point cloud part and a dense point cloud part based on the point density of different parts of the point cloud to be labeled; performing progressive morphological filtering on the sparse point cloud part to obtain a filtering result, wherein the filtering result includes a filtered sparse point cloud part after removing ground point clouds from the sparse point cloud part; merging the dense point cloud part and the filtered sparse point cloud part to obtain a first merged point cloud; and determining the filtered point cloud based on the first merged point cloud.

[0007] For example, the filtering result further includes: the height range of the ground point cloud in the sparse point cloud portion; determining the filtered point cloud based on the first merged point cloud includes: deleting ground points in the first merged point cloud based on the height range to obtain the filtered point cloud, wherein the height of the ground points is within the height range.

[0008] For example, dividing the point cloud to be labeled into a sparse point cloud part and a dense point cloud part includes: using a radius outlier filter to filter the point cloud to be labeled, and taking the filtered point cloud as the dense point cloud part; and determining the other point clouds in the point cloud to be labeled other than the dense point cloud part as the sparse point cloud part.

[0009] For example, before performing ground point cloud filtering on the point cloud to be labeled, the method further includes: binding and storing the coordinates and index information of each point in the point cloud to be labeled; outputting the location information of the determined target point cloud, including: outputting the index information of each point in the determined target point cloud.

[0010] For example, before outputting the determined location information of the target point cloud, the method further includes: calculating the location information of the true bounding box of each determined target point cloud as the location information of the target point cloud.

[0011] For example, the calculation of the position information of the true bounding box of each target point cloud includes: for each target point cloud, determining the initial bounding box of the target point cloud based on the points of the target point cloud; determining the center point of the initial bounding box, rotating the target point cloud 180° along the height direction with the center point as the rotation center to obtain a rotated point cloud; and merging the rotated point cloud and the target point cloud, and calculating the position information of the bounding box of the merged point cloud as the position information of the true bounding box of the target point cloud.

[0012] For example, before performing ground point cloud filtering on the point cloud to be labeled, the method further includes: performing downsampling on the point cloud to be labeled to obtain the downsampled point cloud to be labeled.

[0013] For example, target segmentation is performed on the filtered point cloud to determine the target point cloud, including: target segmentation is performed on the filtered point cloud to obtain at least one segmented point cloud in the filtered point cloud; for each segmented point cloud, a random sampling consistency segmentation method is used to determine the straight line representing the segmented point cloud; the proportion of the number of points in the segmented point cloud that fall on the straight line is determined; if the proportion is less than a preset proportion threshold, the segmented point cloud is determined as the target point cloud.

[0014] For example, performing region target segmentation on the filtered point cloud includes: performing target segmentation on the filtered point cloud using a Euclidean clustering method.

[0015] For example, before performing ground point cloud filtering on the point cloud to be labeled based on the density of points in the point cloud to be labeled, the method further includes: in response to a first operation by the user on the three-dimensional point cloud displayed in the visualization interface, determining a first region; and determining a set of points in the first region of the three-dimensional point cloud as the point cloud to be labeled.

[0016] For example, the method further includes: when the number of points in the point cloud to be labeled is less than a preset threshold, using a random sampling consistency filtering method to perform ground point cloud filtering on the point cloud to be labeled, so as to obtain a filtered point cloud; wherein, based on the density of points in different parts of the point cloud to be labeled, the ground point cloud filtering on the point cloud to be labeled is performed when the number of points in the point cloud to be labeled is not less than a preset threshold.

[0017] According to another aspect of this application, a method for target tracking in consecutive frames of a point cloud is provided, comprising: acquiring consecutive frames of a 3D point cloud of a target to be tracked and position information of the target point cloud of the target to be tracked in the current frame of the consecutive frames, wherein the position information includes the position information of the bounding box of the target point cloud; for each other frame in the consecutive frames other than the current frame, determining the selected point cloud in the other frame based at least on the position information of the bounding box, wherein the position of the bounding box in the other frame corresponds to the point cloud within the box being a part of the selected point cloud; and performing target segmentation on the selected point cloud to determine... The process involves segmenting the point cloud; performing tracking operations on the point cloud within the outer bounding box to determine the predicted bounding box of the target to be tracked in the other frame; and determining the target tracking point cloud of the target to be tracked in the segmented point cloud based on the predicted bounding box; wherein the position information of the target point cloud of the target to be tracked in the current frame of consecutive frames is obtained using the target annotation method of the above-mentioned three-dimensional point cloud; and / or segmenting the selected point cloud to determine the segmented point cloud, including: using the selected point cloud as the point cloud to be annotated, using the segmented point cloud as the target point cloud, and determining the segmented point cloud using the target annotation method of the above-mentioned three-dimensional point cloud.

[0018] According to another aspect of this application, a target annotation system for three-dimensional point clouds is provided, comprising: a filtering module for performing ground point cloud filtering on the point cloud to be annotated in a first region of the three-dimensional point cloud based on the density of points in different parts of the point cloud to be annotated, so as to obtain a filtered point cloud after removing the ground point cloud; a segmentation module for performing target segmentation on the filtered point cloud to determine the target point cloud; and an output module for outputting the location information of the determined target point cloud.

[0019] According to another aspect of this application, a target tracking system in continuous frames of a point cloud is provided, comprising: an acquisition module, configured to acquire continuous frames of a 3D point cloud of a target to be tracked and position information of the target point cloud of the target to be tracked in the current frame of the continuous frames, wherein the position information includes the position information of the bounding box of the target point cloud; a first determination module, configured to, for each other frame in the continuous frames other than the current frame, determine, at least based on the position information of the bounding box, a selected point cloud in that other frame, wherein the position of the bounding box in that other frame corresponds to the point cloud within the box being a part of the selected point cloud; and a segmentation module, configured to segment the selected point cloud. The system comprises: a target segmentation module to determine a segmented point cloud; a tracking operation module to perform tracking operations on the point cloud within the bounding box to determine the predicted bounding box of the target to be tracked in the other frame; and a second determination module to determine the target tracking point cloud of the target to be tracked in the segmented point cloud based on the predicted bounding box; wherein the acquisition module obtains the position information of the target point cloud of the target to be tracked in the current frame of consecutive frames using the target annotation method of the above-mentioned three-dimensional point cloud; and / or the segmentation module is specifically used to: use the selected point cloud as the point cloud to be annotated, use the segmented point cloud as the target point cloud, and use the target annotation method of the above-mentioned three-dimensional point cloud to determine the segmented point cloud.

[0020] According to another aspect of this application, an electronic device is provided, including a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, are used to perform the aforementioned target annotation method for three-dimensional point clouds and / or the aforementioned target tracking method in consecutive frames of point clouds.

[0021] According to another aspect of this application, a storage medium is provided, on which program instructions are stored, which, when executed, are used to perform the target annotation method of the above-described three-dimensional point cloud and / or the target tracking method in the above-described continuous frames of the point cloud.

[0022] According to the above scheme, the target point cloud can be quickly and accurately determined by filtering the ground point cloud in the point cloud to be labeled and then segmenting the target point cloud after removing the ground. Finally, the target point cloud is labeled by outputting its position information. This scheme has low computational cost and can significantly improve labeling efficiency. At the same time, the scheme of filtering the ground point cloud according to the density of points in different parts of the point cloud to be labeled can remove the ground point cloud more accurately, thereby improving the accuracy of the determined target point cloud and the accuracy of target labeling. Moreover, this scheme does not require training and has low implementation cost, so it has good feasibility and strong scalability.

[0023] The invention description introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This description is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0024] The advantages and features of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0025] The following drawings, which are incorporated herein by reference and are used to understand this application, illustrate embodiments of the invention and their descriptions to explain the principles of the invention. In the drawings,

[0026] Figure 1 A schematic flowchart illustrating a target annotation method for a 3D point cloud according to an embodiment of this application is shown.

[0027] Figure 2a A schematic diagram showing the initial bounding box of a target point cloud according to an embodiment of this application;

[0028] Figure 2b A schematic diagram of a rotated point cloud of a target point cloud according to an embodiment of this application is shown;

[0029] Figure 2c A schematic diagram showing the actual bounding box of a target point cloud according to an embodiment of this application;

[0030] Figure 3 A schematic flowchart illustrating a target annotation method for a three-dimensional point cloud according to another embodiment of this application is shown.

[0031] Figure 4 A schematic flowchart illustrating a target tracking method in consecutive frames of a point cloud according to an embodiment of this application is shown.

[0032] Figure 5 A schematic block diagram of a target annotation system for a three-dimensional point cloud according to an embodiment of this application is shown.

[0033] Figure 6 A schematic block diagram of a target annotation system for a 3D point cloud according to an embodiment of this application is shown; and

[0034] Figure 7 A schematic block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0035] In the following description, numerous details are provided to enable a thorough understanding of this application. However, those skilled in the art will appreciate that the following description merely illustrates preferred embodiments of the application, and that the application can be implemented without one or more of these details. Furthermore, to avoid confusion with this application, some technical features well-known in the art have not been described in detail.

[0036] To at least partially solve the above-mentioned technical problems, embodiments of this application provide a target annotation method for three-dimensional point clouds.

[0037] Figure 1 A schematic flowchart illustrating a target annotation method 100 for a 3D point cloud according to an embodiment of this application is shown. Figure 1 As shown, the target annotation method 100 for 3D point clouds includes steps S120, S140 and S160.

[0038] Step S120: For the point cloud to be labeled in the first region of the 3D point cloud, ground point cloud filtering is performed on the point cloud to be labeled based on the density of points in different parts of the point cloud to be labeled, so as to obtain the filtered point cloud after removing the ground point cloud.

[0039] According to embodiments of this application, the 3D point cloud can contain at least one target. The target can be any suitable object, which can be static or dynamic. In a specific example, the target can be a moving car. The 3D point cloud is, for example, one frame from a series of frames obtained by the LiDAR of a currently autonomously driving car A in real-time acquisition of its external environment. It is understood that the external environment of car A may include pedestrians, roads, and other vehicles around it. Therefore, the 3D point cloud can also include objects in this environment. In this example, the target can be other vehicles located around car A. If multiple vehicles are located around car A, the target can be one or more of them, depending on the actual annotation requirements.

[0040] 3D point clouds can be raw point cloud data directly acquired using any existing or future suitable point cloud acquisition device, or point cloud data after preprocessing. This preprocessing can include various suitable operations to facilitate subsequent processing, such as coordinate transformation, point cloud downsampling, and point cloud filtering.

[0041] According to embodiments of this application, the first region in the 3D point cloud can be a region of arbitrary size and shape. In one example, the first region can be a preset fixed region. For example, the first region can be a region within a preset coordinate range at a preset calibration position. Specifically, in the example where the 3D point cloud is a point cloud of the surrounding environment of an autonomously driving car A, the first region is, for example, a cube-shaped region centered on the origin of the 3D point cloud (e.g., the radar acquisition center corresponding to the top of car A), with a length, width, and height of 5 meters each. In another example, the first region can also be determined based on region parameters input by the user. Optionally, the user can directly input the size parameters of the first region to be labeled in the human-computer interaction interface, such as the coordinates of the corner points or center point of the region, as well as the length, width, and height of the region. Alternatively, a human-computer interaction interface can be provided that displays a visualization window of the 3D point cloud, in which the user can input the first region to be labeled through a toolbar such as "Region Selection". Specifically, in the example above where the 3D point cloud represents the point cloud of the surrounding environment of a car A in autonomous driving, the 3D point cloud of the surrounding environment of car A from a top-down perspective can be displayed in the human-computer interaction interface. The user can select a rectangular area as the first area using a rectangle, such as one in the interface. For example, if the user wants to mark the position of car B around car A, they can roughly draw a rectangle around car B by controlling the left mouse button, placing car B within the rectangle, and then input the information into the system by clicking the "OK" control in the interface.

[0042] According to embodiments of this application, the point cloud to be labeled can be a portion of a 3D point cloud falling within a first region. For example, in the example described above where the user selects a rectangular region as the first region using a rectangle in an interface, the two sides of the rectangular region can be parallel to the x-axis and y-axis directions of the 3D point cloud, respectively. This rectangular region can be represented, for example, as "200≤x≤700 and 100≤y≤330", then the point cloud to be labeled can be a point cloud composed of all points in the 3D point cloud whose x and y coordinates fall within the corresponding coordinate range. Specifically, in the example described above where the user defines a rectangular frame around car B as the first region, the point cloud to be labeled can include all points of car B within the rectangular frame, as well as other points within the rectangular frame, such as ground points.

[0043] It can be understood that the first region is the approximate area to be labeled. The point cloud within the first region includes not only the target point cloud but also the ground point cloud surrounding the target and the point clouds of other objects. To accurately label the target's location, this step involves filtering the point cloud to be labeled using ground point clouds to obtain a filtered point cloud.

[0044] Typically, in 3D point clouds acquired by point cloud acquisition devices such as LiDAR, the density of points in the ground portion of the point cloud is relatively low, meaning the point distribution is sparse. Conversely, the density of points in the target point cloud is relatively high, meaning the point distribution is dense. Therefore, ground point cloud filtering can be performed on the point cloud to be labeled based on the point density of different parts of the point cloud. Any suitable filtering logic can be used, and any existing or future suitable point cloud filtering algorithm can be employed for ground point cloud filtering of the point cloud to be labeled.

[0045] By way of example and not limitation, the point cloud to be labeled can first be divided into multiple point cloud parts according to the density of points in different parts; then, at least one of the multiple point cloud parts can be filtered to remove ground points from the point cloud part; finally, the filtered point cloud after removing ground points can be obtained from the filtered point cloud parts.

[0046] For example, the point cloud to be labeled can be divided into an appropriate number of point cloud segments. Optionally, the point cloud to be labeled can be divided into two types of point cloud segments. For example, the point cloud to be labeled can be divided into a point cloud segment with a high point density and a point cloud segment with a low point density. The segment with a low point density can be called a sparse point cloud segment with a sparse point distribution. The segment with a high point density can be called a dense point cloud segment with a dense point distribution. Alternatively, the point cloud to be labeled can be divided into three or more types of point cloud segments. For example, the point cloud to be labeled can be divided into a low-density point cloud segment, a medium-density point cloud segment, and a high-density point cloud segment, etc.

[0047] Taking the above example of dividing the point cloud to be labeled into a dense point cloud portion and a sparse point cloud portion, various suitable division criteria can be used for the division. Optionally, the number of points in a preset neighborhood of each point in the point cloud to be labeled can be determined. If this number is not less than a first number threshold, the point can be regarded as a dense point; conversely, if the number is less than the first number threshold, the point can be regarded as a sparse point. Furthermore, the set of the determined dense points can be regarded as the dense point cloud portion of the point cloud to be labeled, and the set of the determined sparse points can be regarded as the sparse point cloud portion of the point cloud to be labeled. Exemplarily, but not limitingly, a radius outlier filter can be used for division. Alternatively, the first region can be divided into multiple unit regions, and the number of points in the point cloud to be labeled falling within each unit region and the average distance between adjacent points in that unit region can be determined. Points in regions where the number of points is less than a second number threshold and the average distance is large can be determined as sparse points, and the set of the determined sparse points can be determined as the sparse point cloud portion, while the set of other points in the point cloud to be labeled can be regarded as the dense point cloud portion.

[0048] Taking the example of dividing the point cloud to be labeled into dense and sparse point cloud components, as described above, alternatively, after determining the dense and sparse point cloud components, appropriate filtering methods can be applied to each component to remove ground point clouds. Finally, the filtered components are merged to obtain a filtered point cloud with ground point clouds removed. Since the point cloud of a target (e.g., a vehicle) is usually dense, while the point cloud of the target's background (e.g., the road surface) is usually sparse, the dense point cloud component in the 3D point cloud can be left unfiltered for more accurate target labeling. Alternatively, only one type of point cloud component can be filtered; for example, only the sparse point cloud component can be filtered to obtain the filtered sparse point cloud component. Then, the filtered sparse point cloud component and the divided dense point cloud component can be merged, and the filtered point cloud with ground point clouds removed can be determined based on the merged point cloud. In this step, various suitable filtering algorithms can be used to filter the ground point clouds in the divided point cloud components. By way of example and not limitation, the filtering algorithm may include random sampling consistency filtering algorithms, progressive morphological filtering algorithms, etc.

[0049] Step S140: Perform target segmentation on the filtered point cloud to determine the target point cloud.

[0050] After obtaining the point cloud after removing the ground point cloud filter, target segmentation can be further performed on the point cloud. According to the embodiments of this application, the target segmentation can be implemented using any existing or future-developed point cloud segmentation algorithm. These point cloud segmentation algorithms may include, but are not limited to, random sample consensus segmentation algorithm, Euclidean clustering segmentation algorithm, region growing segmentation algorithm, etc.

[0051] According to a preferred embodiment of this application, Euclidean clustering is used to segment the filtered point cloud. The Euclidean clustering algorithm requires less computation and produces more accurate segmentation results. This ensures the accuracy of the target criteria.

[0052] It is understandable that after ground point cloud filtering of the point cloud to be labeled in the first region, the point cloud to be labeled can include target point clouds of each target falling within the first region. For example, if the first region only includes car B, the point cloud after ground point cloud filtering can include the target point cloud corresponding to car B, or it can also include a small number of interfering point clouds around it. In this step, using a method such as Euclidean clustering to segment the point cloud after ground point cloud filtering can remove the interfering point clouds with a small number of points, resulting in a target point cloud that only includes car B. Similarly, if the first region includes car B and car C, the point cloud after ground point cloud filtering can include target point clouds corresponding to car B and target point clouds corresponding to car C. In this step, segmenting the point cloud after ground point cloud filtering using a method such as Euclidean clustering can separate the target point cloud that only includes car B and the target point cloud that only includes car C, thus creating two target point clouds.

[0053] It is understandable that by successively passing the ground point cloud filtering step in step S120 and the target segmentation step in step S140, a relatively accurate target point cloud corresponding to each target in the first region can be obtained.

[0054] Step S160: Output the location information of the determined target point cloud.

[0055] After determining the target point cloud of each target contained in the first region, the positional information of each target point cloud can be determined as annotation data. Optionally, the positional information of the target point cloud may include the positional information of each point in the target point cloud. In this step, the positional coordinates of each point in each target point cloud can be directly output. Alternatively, the correspondence between the positional coordinates and positional index numbers of each point in the point cloud to be annotated can be established or obtained in advance, and the positional index numbers can be stored. In this step, the positional coordinates of the points can be omitted, and only the positional index numbers of each point in the target point cloud can be output. Alternatively, the positional information of the target point cloud can also be represented by the positional information of the minimum bounding box of the target point cloud. Before this step, any suitable method can be used to determine the minimum bounding box of each target point cloud. The minimum bounding box is, for example, an axis-aligned bounding box, a directed bounding box, and a point cloud convex hull. Preferably, the minimum bounding box can be the minimum directed bounding box of the target point cloud. In this step, the positional information of each minimum bounding box can be output. Taking the minimum directed bounding box (OBB) as an example, the position information can include the center coordinates of the OBB, the unit vectors of the OBB in the three directions (length, width, and height), and the dimensions of the OBB in the three directions.

[0056] According to embodiments of this application, corresponding location information can also be output based on the user's desired annotation mode. Exemplarily, but not limitingly, the annotation method may include bounding box annotation and segmentation annotation. Prior to this step, a human-computer interaction interface can be provided to the user to obtain the user's desired annotation mode, and then the location information of the corresponding target point cloud can be determined based on this annotation mode. For example, the human-computer interaction interface may include a first selection control named "stereoscopic bounding box annotation mode" and a second selection control named "segmentation annotation mode". Exemplarily, if a user's selection instruction for the first selection control is received, it can be determined that the current annotation mode is bounding box annotation, and the location information of the minimum bounding box of each target point cloud can be output in this step; if a user's selection instruction for the second selection control is received, it can be determined that the current annotation mode is segmentation annotation, and the location coordinates or location index number of each point in each target point cloud can be directly output in this step.

[0057] According to the embodiments of this application, the above annotation method 100 can be implemented in any existing or future suitable point cloud basic processing platform. For example, a browser / server (B / S architecture) model can be adopted, providing HTTP API calls, and the point cloud processing algorithms involved in each step can be implemented through various related basic processing algorithms in the Point Cloud Library (PCL). The solution based on a mature point cloud processing library has a smaller computational load, higher processing efficiency, and stronger scalability.

[0058] According to the above scheme, the target point cloud can be quickly and accurately determined by filtering the ground point cloud in the point cloud to be labeled and then segmenting the target point cloud after removing the ground. Finally, the target point cloud is labeled by outputting its position information. This scheme has low computational cost and can significantly improve labeling efficiency. At the same time, the scheme of filtering the ground point cloud according to the density of points in different parts of the point cloud to be labeled can remove the ground point cloud more accurately, thereby improving the accuracy of the determined target point cloud and the accuracy of target labeling. Moreover, this scheme does not require training and has low implementation cost, so it has good feasibility and strong scalability.

[0059] For example, before performing ground point cloud filtering on the point cloud to be labeled based on the density of points in the point cloud in step S120, method 100 further includes steps S111 and S112.

[0060] Step S111: In response to the user's first operation on the 3D point cloud displayed in the visualization interface, a first region is determined. As mentioned earlier, annotation can be implemented based on a B / S architecture. For example, the 3D point cloud to be annotated can be displayed in the client's visualization interface. Exemplarily, if the 3D point cloud to be annotated consists of multiple frames, one frame of the 3D point cloud can be displayed at a time. The user can use operable controls to perform human-computer interaction operations on the 3D point cloud displayed in the interface in real time, thereby determining the range of the region to be annotated in the 3D point cloud. The first operation can be any suitable human-computer interaction operation. Exemplarily, but not limitingly, when the user wants to annotate a target within a certain region of the displayed 3D point cloud, the user can use a toolbar such as "Region Selection" in the interface to input the first region to be annotated. For example, in the top view of the 3D point cloud, the user can select the rectangle tool and use the left mouse button to select the rectangular area to be annotated as the first region in the interface, and can confirm the operation by clicking the right mouse button. Of course, the first operation can also include other suitable human-computer interaction operations, which are not limited in this application.

[0061] Step S112: Determine the point set consisting of all points in the first region of the 3D point cloud, as the point cloud to be labeled. For example, after determining the first region, the client can transmit the position coordinates of each point in the 3D point cloud falling within the first region to the server. The server can create a point cloud data structure based on the received point set of all points, as the point cloud to be labeled. For example, a PCL point cloud data structure can be created based on the coordinates of each point in the first region, as the point cloud to be labeled.

[0062] The above approach accurately determines the first region based on user interaction, and then identifies the point cloud within that region to be labeled. This approach fully considers users' personalized needs, making labeling more user-friendly and efficient, thus improving the method's usability.

[0063] As mentioned earlier, when using segmentation annotation mode for annotation, the position information of each point in the determined target point cloud can be output as annotation data. This position information can be the position coordinates of the point, but the amount of data of points in the point cloud may be large, and the method of outputting the position coordinates of the points also occupies a large amount of storage.

[0064] For example, in the case of using the segmentation annotation mode, before performing ground point cloud filtering on the point cloud to be annotated in step S120, the method 100 further includes step S113.

[0065] Step S113: Bind the coordinates and index information of each point in the point cloud to be labeled and store the index information. For example, as described above, while the client transmits the position coordinates of each point in the three-dimensional point cloud that falls within the first region to the server, it can also transmit the index information of each point to the server. The index information of the point is, for example, an extended field representing the point's sequence number, such as "data[3]". The server can bind the received position coordinates and index information of each point. That is, the xyz coordinates of the point and its index number can be combined into a union. Then, a PCL point cloud data structure can be created based on the combined information of the bound points as the point cloud to be labeled. At the same time, the index information of each point is saved.

[0066] Step S160 outputs the determined location information of the target point cloud, including step S161. In step S161, the index information of each point in the determined target point cloud is output. After filtering the point cloud to be labeled using ground point cloud technology and segmenting the filtered point cloud to obtain the target point cloud, the position coordinates of each point in the target point cloud can be determined. Since the position coordinates and index information of each point have been bound in step S113, the index information of the point can be determined based on the position coordinates of the point. In this step, the position information of the point can be represented by the index information of the point, and the index information of each point in the determined target point cloud can be output. For example, the server can return the index information of each point in the determined target point cloud as labeling information to the client. This can significantly improve the data transmission speed.

[0067] In the above scheme, the coordinates and index information of each point in the point cloud to be labeled are bound together, and the index information of the determined target point cloud points is output as the labeling data. This can save the space required for data storage and transmission while ensuring labeling accuracy, significantly improve data transmission speed, and thus effectively improve labeling efficiency.

[0068] For example, before performing ground point cloud filtering on the point cloud to be labeled in step S120, method 100 further includes step S114. In step S114, downsampling is performed on the point cloud to be labeled to obtain a downsampled point cloud to be labeled. Any existing or future suitable point cloud downsampling method can be used to downsample the point cloud to be labeled. For example, but not limitingly, downsampling algorithms such as random downsampling, moving least squares downsampling, radius filtering downsampling, and Poisson disk downsampling can be used. It can be understood that by downsampling the point cloud, the density of the point cloud to be labeled can be reduced while maintaining the overall geometric features of the point cloud, thereby reducing the amount of data and algorithm complexity of point cloud processing in steps such as ground point cloud filtering and target segmentation, and thus significantly improving labeling efficiency.

[0069] As mentioned earlier, when using the bounding box annotation mode, it is not necessary to output the position information of each point; instead, the bounding box of the determined target point cloud can be output as annotation data. In this case, when processing and annotating the point cloud to be annotated, as long as the overall geometric features of the point cloud remain unchanged, its bounding box can be accurately determined. Therefore, for the case of using the bounding box annotation mode, it is particularly suitable to perform downsampling on the point cloud to be annotated in step S114 above, so as to reduce the computational load of subsequent processing steps and improve annotation efficiency while ensuring the accuracy of the output results.

[0070] For example, method 100 further includes step S120'. In step S120', if the number of points in the point cloud to be labeled is less than a preset threshold, a random sampling consistency filtering method is used to filter the point cloud to be labeled using ground point cloud filtering to obtain a filtered point cloud. Specifically, step S120 filters the point cloud to be labeled based on the density of points in different parts of the point cloud to be labeled, and is performed if the number of points in the point cloud to be labeled is not less than the preset threshold.

[0071] According to the embodiments of this application, before performing ground point cloud filtering on the point cloud to be labeled, the number of points in the point cloud to be labeled can be counted first, and a suitable ground point cloud filtering algorithm can be determined based on the counted number to balance the accuracy and efficiency of ground point cloud filtering. In other words, depending on the number of points in the point cloud to be labeled, the filtered point cloud targeted by the target segmentation in step S140 can be obtained through step S120' or step S120.

[0072] Specifically, when the number of points in the point cloud to be labeled is less than a preset threshold, the random sample-consistency filtering algorithm can be directly used to filter the ground point cloud. The preset threshold can be an empirical value obtained from data testing, and different preset thresholds can be set for different point cloud data. For example, the preset threshold can be 1500. That is, when the number of points in the point cloud to be labeled is less than 1500, the random sample-consistency filtering algorithm can be directly used to remove ground point clouds from the point cloud to be labeled. The scheme of using the random sample-consistency filtering algorithm to remove ground point clouds from the point cloud has a faster processing speed and better filtering accuracy for point clouds with a small number of points.

[0073] When the number of points in the point cloud to be labeled is greater than or equal to a preset threshold, directly using the random sample consistency filter algorithm to remove ground point clouds from the point cloud to be labeled may result in relatively poor filtering accuracy. In this case, ground point cloud filtering can be performed on the point cloud to be labeled based on the density of points in different parts of the point cloud.

[0074] In the above scheme, when the amount of point cloud data to be labeled is small, the random sampling consistency filtering algorithm is directly used to remove ground point clouds, resulting in faster processing speed. However, when the amount of point cloud data to be labeled is large, ground point cloud filtering is performed based on the density of points in different parts of the point cloud to ensure both processing efficiency and accuracy. This scheme, which uses a matching ground point cloud filtering method based on the size of the point cloud data to be labeled, has a more reasonable execution logic and can better balance the processing speed and accuracy of point clouds.

[0075] For example, step S120 performs ground point cloud filtering on the point cloud to be labeled based on the density of points in different parts of the point cloud to be labeled, including steps S121 to S124.

[0076] Step S121: Based on the point density of different parts of the point cloud to be labeled, divide the point cloud to be labeled into a sparse point cloud part and a dense point cloud part.

[0077] In one example, a radius filter can be used to filter the point cloud to be labeled to determine the sparse and dense point cloud portions. Specifically, step S121 divides the point cloud to be labeled into sparse and dense point cloud portions, including steps S121.1 and S121.2. Step S121.1 uses a Radius Outlier Removal (ROR) filter to filter the point cloud to be labeled, and the filtered point cloud is considered the dense point cloud portion. Step S121.2 identifies the remaining point cloud portions outside the dense point cloud portion as the sparse point cloud portion.

[0078] According to the embodiments of this application, step S121.1 can traverse each point in the point cloud to be labeled, use a radius outlier filter, and set an appropriate filter radius and point number threshold to filter out sparse points in the point cloud to be labeled, obtaining a dense point cloud portion composed of dense points. This method can quickly and accurately remove sparse points from the point cloud to be labeled. The filtered point cloud is the dense point cloud portion. In step S121.2, by comparing the point cloud to be labeled and the filtered dense point cloud portion, the point set of other points outside the dense point cloud portion can be determined as the sparse point cloud portion. This partitioning method is simpler and more accurate. In summary, based on the above technical solution, the point cloud to be labeled can be accurately divided into a sparse point cloud portion and a dense point cloud portion with less computational cost.

[0079] Step S122 involves performing progressive morphological filtering on the sparse point cloud portion to obtain a filtering result. This result includes a filtered sparse point cloud portion after removing ground point clouds from the sparse point cloud portion. It is understood that since the distribution of points in the ground point cloud is typically sparser, the sparse point cloud portion obtained in step S121 can include most of the ground point clouds. Therefore, in step S122, ground point cloud filtering can be performed on the sparse point cloud portion. Specifically, a progressive morphological filtering method can be used in this step to remove ground point clouds from the sparse point cloud portion. Appropriate parameters such as an elevation threshold and a gradually increasing window size can be set, and based on these parameters, a progressive morphological filtering algorithm from the PCL library can be used to obtain the ground point clouds and non-ground point clouds in the sparse point cloud portion. Furthermore, the ground point clouds in the sparse point cloud portion can be deleted, resulting in a filtered sparse point cloud portion after removing the ground point clouds.

[0080] Progressive morphological filtering is applicable to ground point cloud filtering in various scenarios and achieves good filtering results, removing most ground point clouds from sparse point cloud regions. However, this method is slow in processing dense point clouds. This step, by applying progressive morphological filtering only to sparse point cloud regions, ensures both faster processing speed and better results.

[0081] Step S123: Merge the dense point cloud portion and the filtered sparse point cloud portion to obtain the first merged point cloud. Alternatively, the dense point cloud portion obtained in step S121 and the filtered sparse point cloud portion obtained in step S122 can be directly merged into a single point cloud to obtain the first merged point cloud.

[0082] Step S124: Determine the filtered point cloud based on the first merged point cloud. Optionally, the first merged point cloud can be directly used as the filtered point cloud. Alternatively, the first merged point cloud can be further processed. For example, various suitable methods can be used to perform secondary ground point cloud filtering on the first merged point cloud, and the point cloud after secondary ground point cloud filtering can be used as the filtered point cloud for subsequent target segmentation.

[0083] In the above scheme, the point cloud to be labeled is divided into sparse and dense point cloud regions based on the density of points in different parts. Morphological techniques are then used to filter only the sparse point cloud region, achieving the goal of quickly and accurately removing most of the ground point cloud from the point cloud to be labeled. Furthermore, the filtered sparse and dense point cloud regions are merged, and the filtered ground point cloud is determined based on the merged point cloud, ensuring the integrity of the point cloud other than the ground point cloud in the point cloud to be labeled. Therefore, this scheme, on the one hand, ensures good processing efficiency and accuracy of ground point cloud filtering, and on the other hand, guarantees the integrity of the target point cloud in the point cloud to be labeled, thus ensuring the target labeling effect.

[0084] For example, the filtering result also includes the height range of the ground point cloud in the sparse point cloud portion. It can be understood that a digital terrain model (DTM) of the ground point cloud in the sparse point cloud portion can also be output using algorithms such as progressive morphological filtering. Based on this digital terrain model, the height range of the ground point cloud can also be obtained. This height range is, for example, the range of Z-coordinate values ​​of the ground point cloud in the sparse point cloud portion.

[0085] Step S124, determining the filtered point cloud based on the first merged point cloud, may include the following steps: deleting ground points from the first merged point cloud based on a height range to obtain the filtered point cloud. The height of the ground points is within the height range of the ground point cloud in the sparse point cloud portion.

[0086] It's understandable that for large-scale 3D point clouds, the ground is not an ideal plane but exhibits height undulations within a certain range. Therefore, the aforementioned ground point cloud filtering method may not be able to completely remove the ground point clouds. For example, dense point clouds may also include ground point clouds, which are included in the first merged point cloud. In this approach, the ground height range represented in the digital terrain model output by the progressive morphological filtering algorithm can be used to further filter out the remaining ground point clouds contained in the first merged point cloud. Taking a ground height range of 0 meters ≤ z ≤ 1 meter represented in the digital terrain model as an example, we can iterate through each point in the first merged point cloud and remove points with z-coordinate values ​​within this range. Specifically, based on this height range, a pass-through filter in PCL can be used to filter out points within this ground height range, resulting in a filtered point cloud.

[0087] This scheme can further improve the thoroughness of ground point cloud filtering, avoid interference from residual ground point clouds to subsequent target identification, and thus help improve the efficiency and accuracy of target annotation.

[0088] For example, step S140 performs target segmentation on the filtered point cloud to determine the target point cloud, including step S141. In step S141, target segmentation is performed on the filtered point cloud to obtain at least one segmented point cloud in the filtered point cloud. Step S140 further includes: for each segmented point cloud, performing the following steps S142 to S144. Step S142: for each segmented point cloud, a random sampling consistency segmentation method is used to determine the straight line representing the segmented point cloud. Step S143: the proportion of points in the segmented point cloud that fall on the straight line is determined. Step S144: if the proportion is less than a preset proportion threshold, the segmented point cloud is determined as the target point cloud.

[0089] It is understood that in step S141, any suitable segmentation algorithm can be used to achieve the target segmentation of the point cloud, such as Euclidean clustering segmentation algorithm, region growing segmentation algorithm, etc., to obtain one or more segmented point clouds. Each segmented point cloud can be a different segmentation result of the point cloud segmentation. For example, the Euclidean clustering segmentation algorithm can be used to segment the filtered point cloud into a first segmented point cloud corresponding to object D and a second segmented point cloud corresponding to object E. Object D corresponding to the first segmented point cloud may not be the target to be segmented. For example, object D may not be the target car, but may be part of other interfering objects, such as trees. That is, the first segmented point cloud is not the target point cloud to be labeled, but an invalid segmentation result. Therefore, the invalid segmentation results in the first segmented point cloud can be filtered out by the methods in steps S142 to S144. Specifically, in step S142, a straight line (or plane) representing each segmented point cloud can be obtained by using the random sampling consistency method. And in step S143, the ratio of the number of points in the segmented point cloud that fall on the straight line to the total number of points in the segmented point cloud can be calculated. Then, in step S144, invalid segmented point clouds are deleted based on their proportion and a preset proportion threshold. The preset proportion threshold can be set according to actual processing needs. For example, 50%. That is, if the calculated proportion of a certain segmented point cloud is greater than or equal to 50%, the segmented point cloud can be directly removed.

[0090] In the above scheme, the linear fitting method with random sampling consistency can filter out invalid segmentation results, thereby further improving the accuracy of the determined target point cloud and ensuring good annotation accuracy.

[0091] As mentioned above, in the case of using the bounding box annotation mode, the position information of the actual bounding box of the target point cloud can be output. For example, before outputting the determined position information of the target point cloud in step S160, the method 100 further includes step S150.

[0092] Step S150: Calculate the position information of the true bounding box for each determined target point cloud, using it as the position information of that target point cloud. Any suitable method can be used to calculate the position information of the true bounding box for each target point cloud. The true bounding box can be various suitable bounding boxes, such as axis-aligned bounding boxes, oriented bounding boxes, fixed-direction convex hulls, etc. For example, the true bounding box of a target point cloud can be the smallest cubic bounding box that encloses the target point cloud. If the target point cloud is, for example, the point cloud of a car, its bounding box can be the smallest cuboid bounding box with its base parallel to the ground. Optionally, the bounding box of each target point cloud can be determined directly based on the 3D coordinates of each point in the target point cloud using various suitable bounding box fitting algorithms. Alternatively, existing methods in point cloud basic processing platforms such as PCL can be used to obtain an initial bounding box, and then the desired bounding box can be obtained by adjusting the initial bounding box, serving as the true bounding box.

[0093] The above-mentioned method for determining and outputting the true bounding box of the target point cloud is relatively simple and has high annotation efficiency.

[0094] For example, step S150 calculates the position information of the true bounding box of each determined target point cloud, including performing steps S151 to S154 for each determined target point cloud.

[0095] Step S151: For each determined target point cloud, determine the initial bounding box of the target point cloud based on its points. Step S152: Determine the center point of the initial bounding box. Step S153: Rotate the target point cloud 180° along the height direction using the center point as the rotation center to obtain a rotated point cloud. Step S154: Merge the rotated point cloud and the target point cloud, and calculate the position information of the bounding box of the merged point cloud as the position information of the true bounding box of the target point cloud.

[0096] Figure 2a This diagram illustrates the initial bounding box of a target point cloud according to an embodiment of this application. In this embodiment, the target is a car. As shown, the initial minimum directed bounding box (shown as the gray bounding box in the figure) of each car point cloud can be determined using the minimum directed bounding box determination method in PCL.

[0097] It is understandable that during radar scanning of a car, some faces of the car may be obscured and not scanned. Therefore, the resulting point cloud is not a complete car point cloud. For example, data for the right side of the car may be missing, causing the initial directed bounding box of the target point cloud determined in the PCL to deviate from its true position. For example, the bottom surface of the determined rectangular directed bounding box may not be flush with the ground, or the angle deviation may be large. Therefore, according to the embodiments of this application, the initial bounding box of each determined target point cloud is also adjusted to obtain the true bounding box of each target point cloud. Specifically, the center of the initial bounding box can be determined first using various suitable methods. It is assumed that the center of the initial bounding box corresponds to the center of the target point cloud. Therefore, the target point cloud can be repositioned based on the center of the initial bounding box, so that the repositioned target point cloud can roughly fill in the missing point cloud parts.

[0098] Figure 2b This diagram illustrates a rotated point cloud of a target point cloud according to an embodiment of this application. Since targets such as cars are often symmetrical objects, as shown, the target point cloud can be rotated 180° along the height direction with the center point of the determined initial bounding box as the rotation center to obtain a rotated point cloud. The height direction can be the height direction of the target, such as the height direction of a car, i.e., the direction perpendicular to the ground. It can be understood that for a 3D point cloud of a target in a large scene, the z-axis direction of the point cloud usually corresponds to the height direction of the target. Therefore, the target point cloud, such as a car point cloud, can be rotated 180° around the z-axis with the center point as the rotation center to obtain a rotated point cloud. Taking a car point cloud as an example, it can be understood that a car is generally a symmetrical structure. The left side of the rotated car point cloud can be aligned with the right side of the original car point cloud.

[0099] Figure 2c This diagram illustrates the true bounding box of a target point cloud according to an embodiment of this application. As shown, the rotated car point cloud is merged with the original car point cloud. The merged point cloud can roughly complete the point cloud data of the missing side of the original point cloud. Specifically, the point cloud of the left side of the car visible to radar can roughly represent the right side of the car invisible to radar after rotation, i.e., it can be used as the right side point cloud of the car. Thus, the point cloud of the car is completed, avoiding the negative impact of the radar view. Finally, the directed bounding box determination method in PCL can be used to determine the minimum directed bounding box of the merged point cloud. Thus, the length direction of this minimum directed bounding box can accurately correspond to the vehicle body direction, and the height direction can effectively correspond to the height direction of the car. Finally, this minimum directed bounding box can be used as the true bounding box of the target point cloud, and the position information of the true bounding box is output as annotation information.

[0100] The true bounding box of each target point cloud determined by the above scheme can more accurately represent the vehicle's location information, thus resulting in more accurate annotation information. Furthermore, this scheme has simple execution logic, low computational load, and therefore high processing efficiency and good real-time performance.

[0101] Figure 3 This diagram illustrates a schematic flowchart of a target annotation method 300 for a 3D point cloud according to another embodiment of this application. The 3D point cloud may be a point cloud of the external environment of a car A in autonomous driving. Figure 3As shown, a B / S architecture can be adopted, providing HTTP API calls, and the point cloud processing algorithm in PCL can be used to achieve target annotation of 3D point clouds. First, in response to the user's first operation on the display interface of the 3D point cloud, the point cloud to be annotated in the first region of the 3D point cloud can be determined, i.e., region segmentation. For example, the user can use operable controls to select a rectangular region in the 3D point cloud, i.e., the first region. The client can send an HTTP request to the server, which includes information about the location of the rectangular region. The server can calculate the point cloud falling within the rectangular region in the current frame based on the location of the rectangular region and construct PCL point cloud data as the point cloud to be annotated. In this step, the position information and index information of each point in the point cloud can also be bound and stored. Furthermore, the current annotation mode can be determined based on the user's selection of the annotation mode. This annotation method can include bounding box annotation and segmentation annotation. If the current annotation mode is determined to be bounding box annotation, the point cloud to be annotated can first be downsampled to reduce the number of points in the point cloud to reduce the amount of computation and improve processing efficiency. Typically, the number of points in the downsampled point cloud can be reduced to below a preset threshold, allowing for further processing such as ground removal. If the current annotation method is determined to be segmentation rather than bounding box annotation, point cloud downsampling is not performed; instead, ground removal can be directly applied to the point cloud to be annotated. Specifically, the number of points in the point cloud to be annotated or the downsampled point cloud can be calculated. If the number is less than or equal to the preset threshold, the random sample consistency filter method in PCL can be directly used to remove ground points, resulting in a filtered point cloud. If the number is greater than the preset threshold, ground point cloud filtering can be applied to the point cloud to be annotated based on the density of points in different parts of the point cloud, resulting in a filtered point cloud with ground points removed. Specifically, a radius filter can be used to divide the point cloud to be annotated into dense and sparse point cloud portions. A progressive morphological filtering method is then used to filter the sparse point cloud portion, removing ground points from this portion, resulting in the filtered sparse point cloud portion. Simultaneously, the height range of the ground point cloud can be determined based on the results of progressive morphological filtering, for example, determining the z-coordinate range as [10, 200]. The filtered sparse and dense point cloud portions are merged to obtain a merged point cloud. Based on the determined height range (z-coordinate range) of the ground point cloud, the remaining ground point clouds in the merged point cloud are filtered out to obtain a filtered point cloud. A pass-through filter can be used to remove points in the merged point cloud whose z-coordinate values ​​are in the interval [10, 200]. Then, methods such as Euclidean clustering can be used to segment the filtered point cloud to determine the car point clouds in the point cloud to be labeled. Post-segmentation operations can be performed on each segmented point cloud obtained from Euclidean clustering.Furthermore, the bounding box of each car point cloud, such as the minimum directed bounding box of a rectangle, can be determined as the annotation bounding box. Specifically, for each segmented point cloud, the straight lines contained in each segmented point cloud can be obtained using the random sampling consistency method. The ratio of the number of points in the segmented point cloud falling on the straight lines to the total number of points in the segmented point cloud is calculated. Then, invalid segmented point clouds are deleted based on the ratio and a preset ratio threshold. The preset ratio threshold can be set according to actual processing needs, for example, 50%. That is, if the calculated ratio of a certain segmented point cloud is greater than or equal to 50%, that segmented point cloud can be directly removed. Thus, each remaining segmented point cloud can be used as a target point cloud to determine the position information of each target point cloud. For the case of using region segmentation annotation, the index information of the points in each target point cloud can be directly output. For the case of using bounding box annotation mode, the position information of the actual bounding box of the target point cloud can be determined and output. Specifically, for each determined target point cloud, the initial bounding box of the target point cloud can first be determined based on the points in the target point cloud. Then, the center point of the initial bounding box can be determined. The target point cloud can be rotated 180° around the z-axis with the center point as the rotation center to obtain a rotated point cloud. The rotated point cloud and the target point cloud are then merged, and the position information of the bounding box of the merged point cloud is calculated as the position information of the true bounding box of the target point cloud. For example, after all post-processing asynchronous tasks are completed, the position information of all target point clouds can be output as annotation data.

[0102] According to another aspect of this application, a method for tracking a target in consecutive frames of a point cloud is also provided. Figure 4 A schematic flowchart of a target tracking method 400 in consecutive frames of a point cloud according to an embodiment of this application is shown. Figure 4 As shown, the tracking method 400 includes steps S410, S420, S430, S440 and S450.

[0103] Step S410: Obtain consecutive frames of the 3D point cloud of the target to be tracked, and the position information of the target point cloud of the target to be tracked in the current frame of the consecutive frames. The position information includes the position information of the bounding box of the target point cloud.

[0104] According to the embodiments of this application, in this step, continuous frames of a 3D point cloud can be directly acquired using a point cloud acquisition device, or pre-stored continuous frames can be indirectly invoked. For example, a user can send the Uniform Resource Locator (URL) of the continuous frames to be tracked to the server using a client, and in this step, the continuous frames can be retrieved from network storage using multiple threads based on the received continuous frame URL. Retrieving continuous frames and obtaining the position information of the target point cloud in the current frame of the continuous frames of the target to be tracked can be performed in various suitable execution orders. The position information of the target point cloud includes the position information of the bounding box of the target point cloud. Optionally, the bounding box of the target point cloud can be pre-annotated and can be called an annotated bounding box. In this step, the annotated bounding box of the target point cloud and the URL of the continuous frames to be tracked can be obtained first, and then the continuous frames can be retrieved based on the continuous frame URL. Alternatively, the continuous frames to be tracked can be retrieved first, and then the annotated bounding box in one frame of the continuous frames of the target to be tracked can be obtained. For example, after fetching consecutive frames, the bounding box of the target to be tracked in the current frame can be received from the user using client input, or the bounding box can be determined using a suitable target annotation algorithm.

[0105] In one example, the current frame of a series of frames can be any frame in the series, as long as it includes the target to be tracked. In another example, the current frame can also be a specific frame in the series. This specific frame could be, for example, the middle frame or the first frame in the series. According to an embodiment of this application, the labeled bounding box can be a stereo bounding box of the target point cloud corresponding to the target to be tracked in the current frame. The labeled bounding box can be any suitable stereo bounding box, including but not limited to axis-aligned bounding boxes, directed bounding boxes, and point cloud convex hulls. Preferably, the labeled bounding box can be the minimum directed bounding box of the target point cloud corresponding to the target to be tracked in the current frame.

[0106] Steps S420, S430, S440 and S450 are performed for each other frame in the consecutive frames acquired in step S410, except for the current frame, to determine the target tracking point cloud in these other frames.

[0107] Step S420: For each other frame in a continuous frame other than the current frame, at least based on the position information of the bounding box, determine the selected point cloud in the other frame, wherein the bounding box is the bounding box of the target point cloud of the target to be tracked in the aforementioned current frame, and the position of the bounding box in the other frame corresponds to the point cloud within the box being a part of the selected point cloud.

[0108] It is understood that the bounding box of the target to be tracked marked in the current frame is the true bounding box of the target to be tracked in the current frame. In a series of frames acquired over a period of time, the position of the target to be tracked may move. For example, for a moving car A, its external environment is constantly changing. Specifically, taking car B located next to car A as the target to be tracked, the relative positional relationship between car A and car B may change in a short period of time. Therefore, for a series of frames capturing the external environment of car A, the position of car B may be different in different frames. It is understood that the positional change of the same target in consecutive frames is usually gradual. Therefore, after determining the position of the target in a certain frame of a series of frames, the approximate position range of the target in other frames can be estimated based on the position of the target in that frame. According to the embodiments of this application, after determining the bounding box of the target to be tracked in the current frame in step S410, the target to be tracked in each other frame of the series of frames other than the current frame can be tracked based on the position parameters of the bounding box in the current frame. Specifically, in step S420, the selected point cloud in each other frame can be determined based on the position parameters of the bounding box in the current frame. The selected point cloud can be a point cloud representing the selected area where the target to be tracked may be located. The selected area can be a region that includes and is larger than the actual location area of ​​the target to be tracked in that other frame.

[0109] According to embodiments of this application, the selected point cloud in each other frame can be determined using various suitable determination logics based on the position parameters of the bounding box in the current frame. Exemplarily, but not limitingly, a selected region in each other frame that is larger than the area of ​​the corresponding frame of the bounding box can be determined based on the position parameters of the bounding box in the current frame, and the point cloud falling within the determined selected region is defined as the selected point cloud. For example, in the embodiment of consecutive frames of the external environment of car A described above, if the bounding box of car B in the current frame is represented as "200≤x≤700, 100≤y≤330 and 100≤z≤330", then a selected region can be determined in each other frame. This selected region can be an area in other frames that includes and is larger than the corresponding frame "200≤x≤700, 100≤y≤330 and 100≤z≤330". This selected region is, for example, a large area obtained by extending along a preset direction of the corresponding frame. This preset direction is, for example, the determined movement direction of the target to be tracked. For example, the movement direction of the target to be tracked can be determined first using any suitable method. Specifically, the bounding box can be a rectangle, and the movement direction can be the direction of the long side of the rectangle. After determining the movement direction, the corresponding position box can be expanded outward along that direction, and the expanded area can be used as the selection region. Then, the selected point cloud within this selection region can be determined in each current frame. This preset direction can also be the direction of the three-dimensional coordinate axes of the point cloud, such as expanding the corresponding position box outward along at least one dimension to obtain the selection region and determining the selected point cloud within it.

[0110] Step S430: Perform target segmentation on the selected point cloud to determine the segmented point cloud.

[0111] It is understood that the selected point cloud may include not only the target point cloud of the target to be tracked, but also non-target point clouds surrounding the target. For example, in the embodiment of consecutive frames of the external environment of car A mentioned above, the selected point cloud in each other frame may include not only the target point cloud corresponding to car B, but also non-target point clouds near car B. For example, the selected point cloud may also include ground point clouds located on the ground and point clouds corresponding to car C. In this step, any suitable method can be used to segment the selected point cloud to determine the segmented point cloud as the segmentation result. Optionally, segmentation can be performed directly using a suitable point cloud segmentation algorithm, either existing or developed in the future, to obtain the segmentation result. These point cloud segmentation algorithms may include, but are not limited to, any one of the following: random sample consensus segmentation algorithm, Euclidean clustering segmentation algorithm, and region growing segmentation algorithm. For example, Euclidean clustering segmentation can be used directly to segment the selected point cloud to obtain the segmented point cloud. Since the computational cost of using the Euclidean clustering segmentation algorithm is lower, the segmentation result is also more accurate. Alternatively, the selected point cloud can be first filtered using ground point cloud techniques, and then a segmentation algorithm such as Euclidean clustering can be used to segment the filtered point cloud to obtain the segmented point cloud. This approach can more accurately segment the desired point cloud.

[0112] It is understandable that the number of segmented point clouds determined in this step may be one or more. In one example, if the selected point cloud only includes the target point cloud corresponding to car B and a small number of surrounding interfering point clouds, then in this step, a method such as Euclidean clustering can be used to segment the selected point cloud to obtain a segmented point cloud corresponding to car B. In another example, if the selected point cloud includes the target point cloud corresponding to car B, the point cloud corresponding to car C, and the nearby ground point cloud, then in this step, the selected point cloud can first be filtered for ground point clouds to obtain filtered point clouds. It is understood that the filtered point cloud can include the point clouds corresponding to car B and the point clouds corresponding to car C. Then, a method such as Euclidean clustering can be used to segment the point cloud after the ground point cloud filtering, resulting in a segmented point cloud containing only the points of car B and a segmented point cloud containing only the points of car C, thus obtaining two segmented point clouds. Similarly, if the segmented point cloud includes more than two cars, then more than two segmented point clouds can be obtained in this step. Of course, in some cases, the number of segmented point clouds determined in this step may also be 0. For example, there may be no target to be tracked in some other frames, in which case the number of segmented point clouds determined in this step may be 0.

[0113] Step S440: Perform tracking calculations on the point cloud in the outer bounding box to determine the predicted bounding box of the target to be tracked in the other frame.

[0114] The point cloud within the labeled bounding box is tracked to determine the position of the target in each other frame of a continuous sequence, thereby determining the predicted bounding box of the target in each other frame. For example, the points contained in the labeled bounding box of the target in the current frame can be calculated, and point cloud data can be constructed as the point cloud to be tracked. Then, the point cloud to be tracked can be input into the PCL tracking algorithm module, and the tracking task for continuous frames can be started. Optionally, the tracking of the point cloud to be tracked can be performed directly on the point cloud of each other frame. For example, the point cloud of each other frame can be input into the PCL tracking algorithm module, and the predicted bounding box of the target in each frame can be output. Alternatively, the tracking of the point cloud to be tracked can be performed on the selected point cloud of each other frame determined in step S420 above. For example, only the selected point cloud of each other frame can be input into the PCL tracking algorithm module, and the predicted bounding box of the target in the selected point cloud can be output. This approach can significantly reduce the amount of computation, improve the efficiency of target tracking, and improve the tracking accuracy.

[0115] According to the embodiments of this application, any existing or future suitable target tracking algorithm can be used to implement target tracking in this step. This includes, but is not limited to, target tracking methods based on the Hungarian algorithm, target tracking methods based on Kalman filtering, and target tracking methods based on correlation filtering.

[0116] Step S450: Based on the predicted bounding box, determine the target tracking point cloud of the target to be tracked in the segmented point cloud.

[0117] According to embodiments of this application, a target tracking point cloud can be determined based on the relative positional relationship between the segmented point cloud determined by target segmentation in step S430 and the predicted bounding box. Specifically, the target tracking point cloud can be a group of the determined segmented point clouds. In other words, the segmented point clouds obtained from the target segmentation result can be filtered using the predicted bounding box obtained from the target tracking result to determine the final target tracking point cloud. For example, for each other frame, segmented point clouds that meet preset criteria can be filtered out as target tracking point clouds based on the relative positional relationship between the predicted bounding box and each segmented point cloud. This relative positional relationship can be determined by various methods.

[0118] In one example, the relative positional relationship can be represented by the number of points in each segmented point cloud that fall within the predicted bounding box, thereby determining the target tracking point cloud. For example, if multiple segmented point clouds are determined in step S430, the number of points in each segmented point cloud that fall within the predicted bounding box can be determined separately, and then the segmented point cloud with the largest number of points can be directly determined as the target tracking point cloud. In particular, in this case, if only one segmented point cloud is determined in step S150, it can be directly determined as the target tracking point cloud. Alternatively, a number threshold can be set. If the number of points is greater than or equal to the number threshold, the segmented point cloud can be used as a candidate target tracking point cloud, and then the one with the largest number of points among the candidate target tracking point clouds can be determined as the target tracking point cloud. This number threshold can be set arbitrarily according to actual needs, for example, it can be set to 3, 5, 10, etc. In this example, if the number of points in each segmented point cloud that fall within the predicted bounding box is less than the preset threshold, a prompt message such as "pending review" can be returned, and manual review can be performed to determine whether a target tracking point cloud exists in other frames.

[0119] In another example, the bounding box of each segmented point cloud can be determined first based on the target segmentation result in step S430. Then, the target tracking point cloud can be determined based on the relative positional relationship between the bounding box and the predicted bounding box of each segmented point cloud. For example, the area of ​​the intersection region between the bounding box and the predicted bounding box of each segmented point cloud can be calculated. The segmented point cloud with the largest intersection area can be determined as the target tracking point cloud. Alternatively, an area threshold can be set, and segmented point clouds with intersection areas greater than the threshold can be used as candidate segmented point clouds, from which the segmented point cloud with the largest intersection area can be determined as the target tracking point cloud.

[0120] Of course, any other suitable method can be used to determine the relative positional relationship between the predicted bounding box and each segmented point cloud, thereby determining the target tracking point cloud.

[0121] In step S410, obtaining the position information of the target point cloud of the target to be tracked in the current frame of the continuous frames can be achieved using the aforementioned 3D point cloud target annotation method 100. Specifically, the ground point cloud filtering method in step S120 can be used to filter the point cloud to be annotated in the first region of the current frame, obtaining the filtered point cloud. Then, the segmentation method in step S140 is used to determine each target point cloud in the current frame, and the position information of each target point cloud is output. This position information includes the position information of the bounding box of the target point cloud.

[0122] Step S430 involves target segmentation of the selected point cloud to determine the segmented point cloud. This segmentation may include using the selected point cloud as the point cloud to be labeled and the segmented point cloud as the target point cloud, and utilizing the aforementioned 3D point cloud target labeling method 100 to determine the segmented point cloud. Similarly, the ground point cloud filtering method in step S120 can be used to filter out ground point clouds in the selected point cloud. Furthermore, the segmentation method in step S140 can be used to determine the segmented point cloud in each other frame.

[0123] The above-described scheme can significantly improve the tracking accuracy of targets in continuous frames of point clouds. Furthermore, the scheme has simple execution logic and low computational cost, resulting in high tracking efficiency. Moreover, the scheme has good applicability and can be applied to target tracking of continuous frames of 3D point clouds in various scenarios.

[0124] According to another aspect of this application, a target annotation system for three-dimensional point clouds is also provided. Figure 5 A schematic block diagram of a 3D point cloud target annotation system 500 according to an embodiment of this application is shown. As shown, the system 500 includes a filtering module 510, a segmentation module 520, and an output module 530. The filtering module 510 is used to perform ground point cloud filtering on the point cloud to be annotated in a first region of the 3D point cloud, based on the density of points in different parts of the point cloud to be annotated, to obtain a filtered point cloud with ground point cloud removed. The segmentation module 520 is used to perform target segmentation on the filtered point cloud to determine target point clouds. The output module 530 is used to output the location information of the determined target point cloud.

[0125] According to another aspect of this application, a target tracking system in continuous frames of point cloud is also provided. Figure 6This diagram illustrates a schematic block diagram of a 3D point cloud target annotation system 600 according to an embodiment of this application. The system 600 includes an acquisition module 610, a first determination module 620, a segmentation module 630, a tracking calculation module 640, and a second determination module 650. The acquisition module 610 is used to acquire consecutive frames of the 3D point cloud of the target to be tracked, and the position information of the target point cloud of the target to be tracked in the current frame of the consecutive frames, wherein the position information includes the position information of the bounding box of the target point cloud. The first determination module 620 is used to determine, for each other frame in the consecutive frames other than the current frame, the selected point cloud in that other frame, at least based on the position information of the bounding box. The position of the bounding box in that other frame corresponds to the point cloud within the bounding box being a part of the selected point cloud. The segmentation module 630 is used to perform target segmentation on the selected point cloud to determine segmented point clouds. The tracking calculation module 640 is used to perform tracking calculations on the point cloud within the bounding box to determine the predicted bounding box of the target to be tracked in that other frame. The second determining module 650 is used to determine the target tracking point cloud of the target to be tracked in the segmented point cloud based on the predicted bounding box. Specifically, the acquisition module 610 acquires the position information of the target point cloud of the target to be tracked in the current frame of consecutive frames using the aforementioned 3D point cloud target annotation method 100; and / or the segmentation module 630 is specifically used to: use the selected point cloud as the point cloud to be annotated, use the segmented point cloud as the target point cloud, and determine the segmented point cloud using the aforementioned 3D point cloud target annotation method 100.

[0126] According to another aspect of this application, an electronic device is also provided. Figure 7 A schematic block diagram of an electronic device 700 according to one embodiment of this application is shown. Figure 7 As shown, the electronic device 700 includes a processor 710 and a memory 720, wherein the memory 720 stores computer program instructions, which are executed by the processor 710 to perform the target annotation method 100 of the three-dimensional point cloud and / or the target tracking method 400 in the continuous frames of the point cloud.

[0127] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the target annotation method 100 of the three-dimensional point cloud and / or the target tracking method 400 in the continuous frames of the point cloud.

[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0130] Similarly, it should be understood that, in order to streamline this application and aid in understanding one or more of the various inventive aspects, features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with features fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0131] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0132] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in a target annotation system for 3D point clouds or a target tracking system in continuous frames of point clouds according to embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0133] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0134] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A target labeling method of a three-dimensional point cloud, characterized by comprising: The method comprises: performing ground point cloud filtering on a to-be-labeled point cloud in a first region in a three-dimensional point cloud based on the density of points in different parts of the to-be-labeled point cloud, to obtain a filtered point cloud from which the ground point cloud is removed; performing target segmentation on the filtered point cloud to determine a target point cloud; and outputting position information of the determined target point cloud. The ground point cloud filtering based on the density of points in different parts of the to-be-labeled point cloud comprises: dividing the to-be-labeled point cloud into a sparse point cloud part and a dense point cloud part based on the density of points in different parts of the to-be-labeled point cloud; performing gradual morphological filtering on the sparse point cloud part to obtain a filtering result, wherein the filtering result comprises a filtered sparse point cloud part from which the ground point cloud in the sparse point cloud part is removed; merging the dense point cloud part and the filtered sparse point cloud part to obtain a first merged point cloud; and determining the filtered point cloud based on the first merged point cloud. The filtering result further comprises a height range of the ground point cloud in the sparse point cloud part. 2.The method of claim 1, wherein, The determination of the filtered point cloud based on the first merged point cloud comprises: based on the height range, deleting ground points in the first merged point cloud whose height is within the height range to obtain the filtered point cloud. The division of the to-be-labeled point cloud into a sparse point cloud part and a dense point cloud part comprises: 3.The method of claim 1, wherein, filtering the to-be-labeled point cloud using a radius outlier filter, and taking the filtered point cloud as the dense point cloud part; and determining other point clouds in the to-be-labeled point cloud other than the dense point cloud part as the sparse point cloud part. Before the ground point cloud filtering on the to-be-labeled point cloud, the method further comprises:

4. The method of claim 1 to 3, wherein, binding the coordinates of each point in the to-be-labeled point cloud with index information of each point and storing the index information. The outputting of the position information of the determined target point cloud comprises: outputting the index information of each point in the determined target point cloud.

5. The target labeling method of a three-dimensional point cloud according to any one of claims 1 to 3, wherein Before the outputting of the position information of the determined target point cloud, the method further comprises: calculating the position information of a real outer bounding box of each determined target point cloud as the position information of the target point cloud. The calculation of the position information of the real outer bounding box of each determined target point cloud comprises: 6.The method of claim 5, wherein, for each determined target point cloud, determining an initial outer bounding box of the target point cloud according to the points of the target point cloud; determining a center point of the initial outer bounding box, rotating the target point cloud by 180° along the height direction with the center point as the rotation center to obtain a rotated point cloud; and merging the rotated point cloud and the target point cloud, and calculating the position information of the outer bounding box of the merged point cloud as the position information of the real outer bounding box of the target point cloud. Before the ground point cloud filtering on the to-be-labeled point cloud, the method further comprises: 7.The method of claim 5, wherein, ​ performing down-sampling on the to-be-labeled point cloud to obtain a down-sampled to-be-labeled point cloud.

8. The method of claim 1 to 3, wherein, The target segmentation on the filtered point cloud comprises: performing target segmentation on the filtered point cloud to obtain at least one segmented point cloud from the filtered point cloud; For each segmented point cloud, adopting a random sample consensus segmentation method to determine a straight line representing the segmented point cloud; determining a proportion of the number of points in the segmented point cloud that fall on the straight line; if the proportion is less than a preset proportion threshold, determining the segmented point cloud as a target point cloud.

9. The method of claim 1 to 3, wherein, The region target segmentation on the filtered point cloud comprises: adopting a Euclidean clustering segmentation method to perform target segmentation on the filtered point cloud.

10. The method of claim 1 to 3, wherein, Before the ground point cloud filtering on the to-be-labeled point cloud based on the density of points in the to-be-labeled point cloud, the method further comprises: in response to a first operation of a user on the three-dimensional point cloud displayed in a visual interface, determining the first region; and determining a point set composed of each point in the first region of the three-dimensional point cloud as the to-be-labeled point cloud.

11. The method of claim 1 to 3, wherein, The method further comprises: in a case where the number of points in the to-be-labeled point cloud is less than a preset threshold, performing ground point cloud filtering on the to-be-labeled point cloud by adopting a random sample consensus filtering method to obtain the filtered point cloud; wherein the ground point cloud filtering on the to-be-labeled point cloud based on the density of points in different parts of the to-be-labeled point cloud is performed in a case where the number of points in the to-be-labeled point cloud is not less than a preset threshold. 12.A method for object tracking in consecutive frames of point clouds, the method comprising: comprises: obtaining a continuous frame of a three-dimensional point cloud of a to-be-tracked target and position information of a target point cloud of the to-be-tracked target in a current frame of the continuous frame, wherein the position information comprises position information of an outer bounding box of the target point cloud; for each other frame of the continuous frame except the current frame, based at least on the position information of the outer bounding box, determining a selected point cloud in the other frame, wherein a point cloud in a position corresponding frame of the outer bounding box in the other frame is part of the selected point cloud; performing target segmentation on the selected point cloud to determine a segmented point cloud; performing tracking operation on the point cloud in the outer bounding box to determine a predicted bounding box of the to-be-tracked target in the other frame; and based on the predicted bounding box, determining a target tracking point cloud of the to-be-tracked target in the segmented point cloud; wherein the position information of the target point cloud of the to-be-tracked target in the current frame of the continuous frame is obtained by using the target labeling method of the three-dimensional point cloud according to any one of claims 1 to 11; and / or the target segmentation on the selected point cloud to determine a segmented point cloud comprises: taking the selected point cloud as a to-be-labeled point cloud, taking the segmented point cloud as a target point cloud, and determining the segmented point cloud by using the target labeling method of the three-dimensional point cloud according to any one of claims 1 to 11.

13. A target labeling system of a three-dimensional point cloud, comprising: The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud in a first region in the three-dimensional point cloud based on densities of points in different parts of the to-be-labeled point cloud, to obtain a filtered point cloud from which the ground point cloud is removed. The segmentation module is configured to perform target segmentation on the filtered point cloud to determine a target point cloud. The output module is configured to output position information of the determined target point cloud. The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud based on densities of points in different parts of the to-be-labeled point cloud, including: The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud based on densities of points in different parts of the to-be-labeled point cloud, including: The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud based on densities of points in different parts of the to-be-labeled point cloud, including: The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud based on densities of points in different parts of the to-be-labeled point cloud, including: The filtering module is configured to perform ground point cloud filtering on the to-be-labeled point cloud based on densities of points in different parts of the to-be-labeled point cloud, including:

14. A target tracking system in continuous frames of point clouds, comprising: The acquisition module is configured to acquire continuous frames of three-dimensional point clouds of a to-be-tracked target and position information of a target point cloud of the to-be-tracked target in a current frame of the continuous frames, wherein the position information includes position information of an outer bounding box of the target point cloud. The first determination module is configured to determine, for each other frame of the continuous frames except the current frame, a selected point cloud in the other frame based at least on the position information of the outer bounding box, wherein a point cloud in a position corresponding to the outer bounding box in the other frame is part of the selected point cloud. The segmentation module is configured to perform target segmentation on the selected point cloud to determine a segmented point cloud. The tracking operation module is configured to perform tracking operation on the point cloud in the outer bounding box to determine a predicted bounding box of the to-be-tracked target in the other frame. The second determination module is configured to determine, based on the predicted bounding box, a target tracking point cloud of the to-be-tracked target in the segmented point cloud. The acquisition module acquires the position information of the target point cloud of the to-be-tracked target in the current frame of the continuous frames by using the target labeling method for three-dimensional point clouds according to any one of claims 1 to 11; and / or The segmentation module is specifically configured to use the target labeling method for three-dimensional point clouds according to any one of claims 1 to 11 to determine the segmented point cloud, with the selected point cloud as the to-be-labeled point cloud and the segmented point cloud as the target point cloud. The memory stores computer program instructions, which, when executed by the processor, are configured to implement the target labeling method for three-dimensional point clouds according to any one of claims 1 to 11 and / or the target tracking method in continuous frames of point clouds according to claim 12.

15. An electronic device comprising a processor and a memory, wherein, ​ 16. A storage medium having stored thereon program instructions which, when executed by a processor, cause the execution of a method for object annotation of a three-dimensional point cloud according to any one of claims 1 to 11 and / or a method for object tracking in consecutive frames of a point cloud according to claim 12.

Images