A target labeling method and device, electronic equipment and storage medium

Abstract

Embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for object labeling, relating to the field of image processing technology. The method comprises: in response to a user determining a first vertex and a second vertex in a first view of raw point cloud data, clustering a target point cloud determined by the first and second vertices to obtain a target bounding box; determining a third vertex and a fourth vertex based on the target bounding box; and determining a three-dimensional bounding box for the target based on the first, second, third, and fourth vertices. Embodiments of the present invention achieve highly efficient object labeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a target labeling method and device, electronic equipment and storage medium. BACKGROUND

[0002] Three-dimensional point cloud generally refers to a plurality of points in a three-dimensional space obtained by a three-dimensional scanning device such as a laser radar. Three-dimensional point cloud data can reflect information of each point in a three-dimensional space, such as coordinate position information, color information, intensity information, etc. Based on three-dimensional point cloud data, specific information of a three-dimensional target composed of each point in a three-dimensional space can be restored, such as the shape and scale of the target. The three-dimensional target includes vehicles, pedestrians, advertising signs, trees and any object in a three-dimensional space.

[0003] Therefore, in the application of three-dimensional point cloud, artificial labeling of a three-dimensional target based on three-dimensional point cloud is often required, that is, in the collected three-dimensional image, a three-dimensional bounding box is determined based on three-dimensional point cloud, and then the target is labeled by using the three-dimensional bounding box, so as to facilitate subsequent processing. In the labeling process, the labeling personnel need to switch between multiple views for observation, so that the three-dimensional bounding box manually constructed can fit the real contour of the target as much as possible. Due to the large number of point clouds and targets, the labeling efficiency is low. SUMMARY

[0004] The purpose of the embodiment of the present application is to provide a target labeling method, device, electronic equipment and storage medium, so as to improve the efficiency and accuracy of point cloud labeling. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiment of the present application provides a target labeling method, which comprises:

[0006] In response to a first vertex and a second vertex determined by a user in a first view of original point cloud data, clustering target point clouds determined by the first vertex and the second vertex to obtain a target bounding box;

[0007] Determining a third vertex and a fourth vertex according to the target bounding box;

[0008] Determining a three-dimensional bounding box of a target to be labeled according to the first vertex, the second vertex, the third vertex and the fourth vertex.

[0009] In one embodiment of the present application, the above method further comprises:

[0010] Determine, based on the first vertex and the second vertex, a first plane perpendicular to the first edge and passing through the first vertex, and a second plane perpendicular to the first edge and passing through the second vertex, to obtain a point cloud coverage range, wherein the point cloud coverage range is an area between the first plane and the second plane in the original point cloud data, and the first edge is an edge formed by the first vertex and the second vertex;

[0011] A point cloud within the coverage range of the point cloud and having a distance from the first edge less than a preset threshold is determined as a target point cloud.

[0012] In one embodiment of the present invention, the target bounding box includes a three-dimensional bounding box or a two-dimensional bounding box, and clustering the target point cloud determined by the first vertex and the second vertex to obtain the target bounding box includes:

[0013] Using a preset three-dimensional clustering algorithm, the target point cloud is clustered to obtain the three-dimensional bounding box;

[0014] or,

[0015] Projecting the target point cloud onto a second view direction to obtain a first point cloud projection image; the second view does not include the first vertex;

[0016] A preset two-dimensional clustering algorithm is used to determine a two-dimensional bounding box in the first point cloud projection image.

[0017] In one embodiment of the present invention, determining the third vertex and the fourth vertex according to the target bounding box includes:

[0018] In response to the user determining the second vertex, switching the display interface from the first view to the second view, and displaying the third vertex and the fourth vertex on the second view, wherein the second view does not include the first vertex;

[0019] The positions of the third vertex and the fourth vertex are updated according to the correction operation of the user.

[0020] In one embodiment of the present invention, updating the positions of the third vertex and the fourth vertex according to the correction operation of the user includes:

[0021] The region of interest is re-determined based on the first vertex, the second vertex and the third vertex corrected by the user, the second target point cloud within the region of interest is clustered to obtain an updated target bounding box, and the fourth vertex is re-determined based on the updated target bounding box.

[0022] In one embodiment of the present invention, the above method further includes:

[0023] In response to the user's modification operation on the third vertex, the re-determined fourth vertex is displayed in the second view.

[0024] In one embodiment of the present application, the target bounding box includes a three-dimensional bounding box and a two-dimensional bounding box, and the method further comprises:

[0025] determining a first proposed third vertex according to the two-dimensional bounding box;

[0026] determining a second proposed third vertex according to the three-dimensional bounding box;

[0027] determining the third vertex according to the first proposed third vertex and the second proposed third vertex.

[0028] In a second aspect, an embodiment of the present application further provides a target labeling device, which comprises:

[0029] a target bounding box determination module, configured to cluster a target point cloud determined by first and second vertices determined by a user in a first view of original point cloud data, to obtain a target bounding box;

[0030] a vertex determination module, configured to determine a third vertex and a fourth vertex according to the target bounding box;

[0031] a three-dimensional bounding box determination module, configured to determine a three-dimensional bounding box of a target to be labeled according to the first vertex, the second vertex, the third vertex and the fourth vertex.

[0032] In one embodiment of the present application, the device further comprises:

[0033] a point cloud coverage range determination module, configured to determine a first plane perpendicular to a first edge and passing through the first vertex, and a second plane perpendicular to the first edge and passing through the second vertex, according to the first vertex and the second vertex, to obtain a point cloud coverage range, wherein the point cloud coverage range is a range of an area between the first plane and the second plane in the original point cloud data, and the first edge is an edge formed by the first vertex and the second vertex;

[0034] a target point cloud determination module, configured to determine, as a target point cloud, a point cloud in the point cloud coverage range and having a distance to the first edge less than a preset threshold.

[0035] In one embodiment of the present application, the target bounding box includes a three-dimensional bounding box or a two-dimensional bounding box, and the target bounding box determination module is specifically configured to:

[0036] perform target clustering on the target point cloud by using a preset three-dimensional clustering algorithm, to obtain the three-dimensional bounding box;

[0037] or,

[0038] Projecting the target point cloud onto a second view direction to obtain a first point cloud projection image; the second view does not include the first vertex;

[0039] A preset two-dimensional clustering algorithm is used to determine a two-dimensional bounding box in the first point cloud projection image.

[0040] In one embodiment of the present invention, the vertex determination module includes:

[0041] a view switching submodule, configured to, in response to the user determining the second vertex, switch the display interface from the first view to the second view, and display the third vertex and the fourth vertex on the second view, wherein the second view does not include the first vertex;

[0042] The vertex position updating submodule is used to update the positions of the third vertex and the fourth vertex according to the user's correction operation.

[0043] In one embodiment of the present invention, the vertex position updating submodule is specifically configured to:

[0044] The region of interest is re-determined based on the first vertex, the second vertex and the third vertex corrected by the user, the second target point cloud within the region of interest is clustered to obtain an updated target bounding box, and the fourth vertex is re-determined based on the updated target bounding box.

[0045] In one embodiment of the present invention, the above-mentioned device further includes:

[0046] The view switching module is configured to display the re-determined fourth vertex in the second view in response to a user's correction operation on the third vertex.

[0047] In one embodiment of the present invention, the target bounding box includes a three-dimensional bounding box and a two-dimensional bounding box, and the apparatus further includes:

[0048] A first vertex determination module, configured to determine a first proposed third vertex according to the two-dimensional bounding box;

[0049] A second vertex determination module, configured to determine a second proposed third vertex according to the three-dimensional bounding box;

[0050] The third vertex determining module is configured to determine the third vertex according to the first proposed third vertex and the second proposed third vertex.

[0051] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0052] a memory for storing a computer program;

[0053] a processor for executing the program stored in the memory to implement the method steps of any of the above.

[0054] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method steps of any of the above.

[0055] The embodiment of the present application has the following beneficial effects:

[0056] The target labeling method provided by the embodiment of the present application first clusters the target point cloud determined by the first vertex and the second vertex in response to the first vertex and the second vertex determined by the user in the first view of the original point cloud data to obtain a target bounding box; determines the third vertex and the fourth vertex according to the target bounding box; and determines the three-dimensional bounding box of the target to be labeled according to the first vertex, the second vertex, the third vertex and the fourth vertex. According to the first vertex, the second vertex, the third vertex and the fourth vertex, the length, width, height, shape scale and other information of the target to be labeled can be obtained, based on which the three-dimensional bounding box of the target to be labeled can be labeled in the point cloud data. The three-dimensional bounding box obtained is determined through mutual comparison of the four vertices located on different planes, and can be as close as possible to the real contour of the target to be labeled. The target boundary does not need to be determined by the eyes of the labeling personnel, which simplifies the operation of the labeling personnel, improves the accuracy and the efficiency of labeling the target.

[0057] Of course, implementing any product or method of the present application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0059] Figure 1a A flowchart of the first target labeling method provided by the embodiment of the present application;

[0060] Figure 1b A free perspective view of the target to be labeled provided by the embodiment of the present application;

[0061] Figure 1c A top view of the target to be labeled provided by the embodiment of the present application;

[0062] Figure 1d A side view of a target to be labeled provided for an embodiment of the present application;

[0063] Figure 1e A front view of a target to be labeled provided for an embodiment of the present application;

[0064] Figure 1f An example diagram of labeling a first vertex and a second vertex in a first view of point cloud data provided for an embodiment of the present application;

[0065] Figure 1g An example diagram of a three-dimensional bounding box of a target to be labeled in a free view of point cloud data provided for an embodiment of the present application;

[0066] Figure 2 A flowchart of a second target labeling method provided for an embodiment of the present application;

[0067] Figure 3a A reference example diagram of obtaining a two-dimensional bounding box by clustering point clouds provided for an embodiment of the present application;

[0068] Figure 3b An example diagram of a target to be labeled in a second view of point cloud data provided for an embodiment of the present application;

[0069] Figure 3c A process example diagram of selecting a fourth vertex provided for an embodiment of the present application;

[0070] Figure 3d An example diagram of a target to be labeled in a second view of point cloud data provided for an embodiment of the present application;

[0071] Figure 4 A structural diagram of a first target labeling device provided for an embodiment of the present application;

[0072] Figure 5 A structural diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application are within the scope of protection of the present application.

[0074] In order to solve the problem of low efficiency of target labeling by labeling personnel based on point cloud, the embodiments of the present application provide a target labeling method, device, electronic device and storage medium, and the target labeling method provided by the embodiments of the present application will be described in detail below through specific embodiments.

[0075] The method of the embodiment of the present application is applied to a smart terminal, and can be implemented by the smart terminal. In actual use, the smart terminal can be a computer, a mobile terminal, or the like.

[0076] In one embodiment of the present application, as shown in FIG. 1, a flowchart of a first target labeling method is provided, and the method includes the following steps. Figure 1a

[0077] Step S11: In response to a first vertex and a second vertex determined by a user in a first view of original point cloud data, a target point cloud determined by the first vertex and the second vertex is clustered to obtain a target bounding box.

[0078] Step S12: A third vertex and a fourth vertex are determined according to the target bounding box.

[0079] Step S13: A three-dimensional bounding box of a target to be labeled is determined according to the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0080] The first vertex and the second vertex are determined by a user and are two vertices of an edge of the target to be labeled in the first view of the original point cloud data, representing a boundary point of a contour of an object currently performing a manual labeling task.

[0081] The target to be labeled is any three-dimensional object in a three-dimensional space, such as a vehicle, a pedestrian, an advertising sign, a tree, or the like, which needs to be labeled based on point cloud data to represent target information of the target, such as a size, a shape, or the like.

[0082] For example, as shown in FIG. 2, a point cloud data diagram of a target to be labeled is shown, where the target to be labeled is a vehicle. As can be seen from the diagram, the obtained point cloud data includes point cloud data of the vehicle and point cloud data of a scene where the vehicle is located. Figure 1b Figure 1b A point cloud data diagram of a target to be labeled is shown, where the target to be labeled is a vehicle. As can be seen from the diagram, the obtained point cloud data includes point cloud data of the vehicle and point cloud data of a scene where the vehicle is located.

[0083] Since the target to be labeled is a three-dimensional object, after obtaining the point cloud data of the target to be labeled, the target to be labeled needs to be displayed through multiple views to determine more complete target information of the target to be labeled. Specifically, the target to be labeled can be displayed through a front view, a side view, and a top view. For example, as shown in FIG. 3, a free-view diagram of a target to be labeled is shown, Figure 1b Figure 1c a top view of the target to be labeled is shown, Figure 1d a side view of the target to be labeled is shown, Figure 1e and a front view of the target to be labeled is shown.

[0084] ​​​In one example, the first view can be one of a side view and a top view, and specifically, the first view is a view including at least one clear edge of the target to be labeled. Based on this, the first view of the point cloud data is determined and displayed, so that the user selects one clear edge of the target to be labeled in the displayed point cloud data, and determines two vertices labeled by the user based on the edge as the first vertex and the second vertex.

[0085] Since the first vertex and the second vertex are two vertices of an edge, the target point cloud to be labeled can be determined according to the first vertex and the second vertex, and then clustered, so as to obtain the target bounding box including the point cloud of the target to be labeled in the first view.

[0086] In one example, the first view of the point cloud data is displayed, and the user is prompted to label the first vertex and the second vertex of an edge of the target to be labeled in sequence, so that the user labels the first vertex and the second vertex in the point cloud data, and determines that the direction of the target to be labeled is from the first vertex to the second vertex.

[0087] For example Figure 1f As shown, Figure 1f An example diagram of labeling the first vertex and the second vertex by the user in the first view of the point cloud data is shown, where the first view is a top view, the user labels two vertices of an edge of the target to be labeled as the first vertex p1 and the second vertex p2 in sequence, and determines that the arrow direction of p1 pointing to p2 represents the direction of the target to be labeled, and two side auxiliary lines are drawn along the edge of the target to be labeled through the first vertex and the second vertex respectively, the target point cloud is determined according to the side auxiliary lines and clustered, and then two other boundary points, the third vertex and the fourth vertex, are determined.

[0088] As known from the above, the first vertex and the second vertex are on the same edge, and the third vertex and the fourth vertex are also on the same edge, but the first vertex and the second vertex are not on the same plane of the three-dimensional bounding box of the target to be labeled as the third vertex and the fourth vertex. At this time, the length, width, height and other size information of the target to be labeled can be determined according to the first vertex, the second vertex, the third vertex and the fourth vertex, so as to obtain the three-dimensional bounding box of the target to be labeled, which can represent the target to be labeled, for example Figure 1g As shown, Figure 1g An example diagram of the three-dimensional bounding box of the target to be labeled in the free view of the point cloud data is shown.

[0089] As can be seen from the above, the target labeling method provided by the embodiment of the present application firstly clusters the target point cloud determined by the first vertex and the second vertex determined by the user in the first view of the original point cloud data to obtain a target bounding box, then determines the third vertex and the fourth vertex according to the target bounding box, and finally determines the three-dimensional bounding box of the target to be labeled according to the first vertex, the second vertex, the third vertex and the fourth vertex. According to the first vertex, the second vertex, the third vertex and the fourth vertex, the length, width, height, shape scale and other information of the target to be labeled can be obtained, and based on this, the three-dimensional bounding box of the target to be labeled can be labeled in the point cloud data. The three-dimensional bounding box obtained is determined through mutual comparison of the four vertices located on different planes, and can be as close as possible to the real contour of the target to be labeled. Moreover, the embodiment of the present application does not need the labeling personnel to manually switch the view to artificially determine the data, simplifies the operation of the labeling personnel, improves the efficiency of labeling the target while improving the accuracy.

[0090] In an embodiment of the present application, as shown in Figure 2 , Figure 2 a flowchart of a second target point cloud labeling method is shown, which includes:

[0091] Step S21: determining a first plane perpendicular to the first edge and passing through the first vertex, and a second plane perpendicular to the first edge and passing through the second vertex according to the first vertex and the second vertex, to obtain a point cloud coverage range.

[0092] The point cloud coverage range is the area range between the first plane and the second plane in the original point cloud data, and the first edge is the edge formed by the first vertex and the second vertex.

[0093] Step S22: determining the point cloud in the point cloud coverage range and having a distance less than a preset threshold from the first edge as the target point cloud.

[0094] In the embodiment of the present application, the first plane perpendicular to the first edge and passing through the first vertex is firstly determined, and the first plane can be a plane on which one face of the target to be labeled is located. Then the second plane perpendicular to the first edge and passing through the second vertex is determined, and the second plane can be a plane on which another face of the target to be labeled is located. The first plane and the second plane can be two opposite parallel planes. The area range between the first plane and the second plane in the point cloud data is taken as the point cloud coverage range.

[0095] Further, the length and size of the target such as people and vehicles on the road generally have little difference. In order to save clustering time and improve clustering efficiency, the point cloud in the point cloud coverage range and the distance from the first edge is less than the preset threshold is determined as the target point cloud. The preset threshold can be determined based on the maximum dimension of some vehicles. When the distance between the point cloud and the first edge is less than the distance, it can be considered that the point cloud is the point cloud of the target to be labeled. At this time, the target point cloud representing the target to be labeled is determined. Therefore, only the target point cloud representing the target to be labeled is clustered, and all point clouds in the point cloud coverage range do not need to be clustered.

[0096] As can be seen from the above, the target labeling method provided by the embodiment of the application determines the point cloud coverage range according to the first vertex and the second vertex, so that the point cloud coverage range as completely as possible includes the points of the target to be labeled in the first view, so that the points included in the point cloud coverage range are more complete. The point cloud in the point cloud coverage range and the distance from the first edge is less than the preset threshold is determined as the target point cloud, and then clustering is performed, which can more accurately determine the target to be labeled.

[0097] In an embodiment of the application, the target bounding box includes a three-dimensional bounding box or a two-dimensional bounding box, and the target point cloud determined by the first vertex and the second vertex is clustered to obtain a target bounding box, including:

[0098] The target point cloud is clustered by using a preset three-dimensional clustering algorithm to obtain the three-dimensional bounding box;

[0099] Or,

[0100] The target point cloud is projected onto a second view direction to obtain a first point cloud projection image; the second view does not include the first vertex;

[0101] A two-dimensional bounding box in the first point cloud projection image is determined by using a preset two-dimensional clustering algorithm.

[0102] In an embodiment of the application, the third vertex and the fourth vertex are determined according to the target bounding box, including:

[0103] In response to the user determining the second vertex, the display interface is switched from the first view to the second view, and the third vertex and the fourth vertex are displayed on the second view, wherein the second view does not include the first vertex;

[0104] The positions of the third vertex and the fourth vertex are updated according to the user's correction operation.

[0105] After obtaining the target point cloud, the target point cloud is projected onto a second view direction to obtain a first point cloud projection image. The second view is a view different from the first view, and the first vertex is not included in the second view. As mentioned above, the first view can be one of a side view and a top view. If the first view is a side view, the second view is one of a front view and a top view. If the first view is a top view, the second view is one of a side view and a front view.

[0106] In a possible implementation, the determination of the two-dimensional bounding box of the target to be labeled in the first point cloud projection image can be calculated by using a preset clustering algorithm. The preset clustering algorithm can be a point cloud clustering algorithm preset in advance, for example, Kmeans (prototype clustering), Euclidean clustering, etc., which is not limited here. For example Figure 3a As shown in the figure, Figure 3a A reference example of the obtained two-dimensional bounding box obtained by clustering the point cloud is shown in the figure. The positions of the third vertex and the fourth vertex can be determined according to the two-dimensional bounding box.

[0107] In order to avoid manual switching of the view by the user, the view can be automatically switched to the second view when the user determines the second vertex. The third vertex and the fourth vertex can be automatically displayed in the display interface. At this time, the user can accept the positions of the third vertex and the fourth vertex or manually correct them according to the requirements. The third vertex and the fourth vertex after the positions are updated according to the correction operation of the user are the final third vertex and fourth vertex.

[0108] For example Figure 3b As shown in the figure, Figure 3b An example of the target to be labeled in the second view of the point cloud data is shown, in which the arrow represents the direction of the target to be labeled, and p3 is the third vertex. For another example Figure 3c and Figure 3d As shown in the figure, in which Figure 3c is an example of the process of selecting the fourth vertex, Figure 3d An example of the target to be labeled in the second view of the point cloud data is shown, in which the arrow represents the direction of the target to be labeled, and p4 is the fourth vertex.

[0109] After obtaining the two-dimensional bounding box, the two-dimensional bounding box can reflect the target information of the target to be labeled in the first view and the second view. If the user thinks that the two-dimensional bounding box is not accurate enough, the user can manually correct it. For example, the user clicks the right mouse button to accept the position of the third vertex given by the clustering algorithm, and the left mouse button represents that the position is not accepted. After clicking the left mouse button, the position of the third vertex moves dynamically with the mouse cursor. When the user moves the cursor to a certain position and clicks the left mouse button again, the position is confirmed as the updated p3 of the user.

[0110] Therefore, the position of each vertex is estimated and displayed by using the clustering algorithm, so that the user does not need to determine the boundary size of the target again by relying on the eyes, and the labor cost is saved.

[0111] As can be seen from the above, the target labeling method provided by the embodiment of the application is used to cluster the point cloud in the determined region of interest in the first point cloud projection image to obtain the target bounding box of the target to be labeled, and the obtained target bounding box can reflect the target information displayed by the target to be labeled in the first view, so that the user no longer needs to repeatedly switch the first view and the second view for labeling, but only needs to operate in the second view, and the user can update the positions of the third vertex and the fourth vertex, so that the three-dimensional bounding box determined based on this is more accurate.

[0112] In an embodiment of the application, the updating of the positions of the third vertex and the fourth vertex according to the user's correction operation comprises:

[0113] The second target point cloud in the region of interest is clustered to determine the fourth vertex again based on the first vertex, the second vertex, and the third vertex corrected by the user.

[0114] The determination process of the region of interest comprises: taking the region range between the first plane and the second plane in the point cloud data as the point cloud coverage range of the target to be labeled, and the point cloud coverage range can be the region range between the first plane and the second plane in the point cloud data. Based on the similar manner of determining the first plane and the second plane, the edge formed by the second vertex and the third vertex is taken as the second edge, the third plane perpendicular to the second edge and passing through the second vertex is determined, and the third plane is the plane where the third face of the target to be labeled is located except the first plane and the second plane. The fourth plane perpendicular to the second edge and passing through the third vertex is determined, and the fourth plane is the plane where the fourth face of the target to be labeled is located except the first plane, the second plane, and the third plane. Since the second vertex and the third vertex are two vertices of the second edge, the third plane and the fourth plane can be two relatively parallel planes, and the first plane, the second plane, the third plane, and the fourth plane can form a surrounding four-face structure, i.e., the region of interest.

[0115] By analogy, the second target point cloud can be the point cloud in the region of interest, and the distance from the first edge and / or the second edge satisfies a preset condition.

[0116] When the target bounding box is a two-dimensional bounding box, the second target point cloud is projected into the second view, and the position of the updated fourth vertex is predicted according to the above method.

[0117] When the target bounding box is a three-dimensional bounding box, the second target point cloud is clustered to determine the fourth vertex again.

[0118] In one embodiment of the present application, the method further comprises:

[0119] When the user corrects the third vertex, the position of the fourth vertex can be updated synchronously in the second view, or the display interface can be switched from the second view to a third view in response to the correction operation of the user on the third vertex, and the re-determined fourth vertex is displayed in the third view. The third view is a third view different from the first view and the second view. As mentioned above, the first view can be one of a side view and a top view. When the first view is a side view, if the second view is a front view, the third view is a top view; if the second view is a top view, the third view is a front view; when the first view is a top view, if the second view is a side view, the third view is a front view; if the second view is a front view, the third view is a side view.

[0120] In one embodiment of the present application, the target bounding box includes a three-dimensional bounding box and a two-dimensional bounding box, and the method further comprises:

[0121] determining a first proposed third vertex according to the two-dimensional bounding box;

[0122] determining a second proposed third vertex according to the three-dimensional bounding box;

[0123] determining the third vertex according to the first proposed third vertex and the second proposed third vertex.

[0124] When the confirmation operation of the third vertex is performed, the generation of the two-dimensional bounding box and the three-dimensional bounding box can be performed simultaneously, and two proposed vertices are determined according to the two-dimensional bounding box and the three-dimensional bounding box respectively. The two proposed vertices can be directly displayed to the user, so that the user can directly select any proposed vertex as the vertex, or the two proposed vertices can be set with weight values, and when the distance between the two proposed vertices is greater than a certain preset value, the proposed vertex with a greater weight is output for display to the user for correction.

[0125] Referring to Figure 4 The embodiment of the present application also provides a structural schematic diagram of a target labeling device, which comprises:

[0126] a target bounding box determination module 401, configured to cluster target point clouds determined by first and second vertices determined by a user in a first view of original point cloud data, to obtain a target bounding box;

[0127] a vertex determination module 402, configured to determine third and fourth vertices according to the target bounding box;

[0128] The three-dimensional bounding box determination module 403 determines a three-dimensional bounding box of the target according to the first vertex, the second vertex, the third vertex and the fourth vertex.

[0129] As can be seen from the above, the target labeling device provided by the embodiment of the present application first clusters the target point cloud determined by the first vertex and the second vertex in response to the first vertex and the second vertex determined by the user in the first view of the original point cloud data to obtain a target bounding box, then determines the third vertex and the fourth vertex according to the target bounding box, and finally determines the three-dimensional bounding box of the target to be labeled according to the first vertex, the second vertex, the third vertex and the fourth vertex. According to the first vertex, the second vertex, the third vertex and the fourth vertex, the length, width, height, shape and scale of the target to be labeled and other types of information can be obtained, based on which the three-dimensional bounding box of the target to be labeled can be labeled in the point cloud data. The three-dimensional bounding box obtained is determined through the mutual comparison of the four vertices located on different planes, and can be as close as possible to the real contour of the target to be labeled. Moreover, the embodiment of the present application does not require the labeling personnel to manually switch the view to artificially determine the data, simplifies the operation of the labeling personnel, improves the efficiency of labeling the target while improving the accuracy.

[0130] In an embodiment of the present application, the device further comprises:

[0131] The point cloud coverage range determination module is configured to determine a first plane perpendicular to the first edge and passing through the first vertex and a second plane perpendicular to the first edge and passing through the second vertex according to the first vertex and the second vertex to obtain a point cloud coverage range, wherein the point cloud coverage range is a range of an area between the first plane and the second plane in the original point cloud data, and the first edge is an edge formed by the first vertex and the second vertex.

[0132] The target point cloud determination module is configured to determine point clouds in the point cloud coverage range and having a distance less than a preset threshold from the first edge as target point clouds.

[0133] As can be seen from the above, the target labeling device provided by the embodiment of the present application determines the point cloud coverage range according to the first vertex and the second vertex, so that the point cloud coverage range as completely as possible includes the points of the target to be labeled in the first view, so that the points included in the point cloud coverage range are more complete, and the point clouds in the point cloud coverage range and having a distance less than a preset threshold from the first edge are determined as target point clouds, and then clustering is performed, so that the target to be labeled can be more accurately determined.

[0134] In an embodiment of the present application, the target bounding box includes a three-dimensional bounding box or a two-dimensional bounding box, and the target bounding box determination module 401 is specifically configured to:

[0135] projecting the target point cloud onto a second view direction to obtain a first point cloud projection image; the first vertex is not included in the second view;

[0136] or,

[0137] projecting the target point cloud onto a second view direction to obtain a first point cloud projection image; the first vertex is not included in the second view;

[0138] determining a two-dimensional bounding box in the first point cloud projection image by using a preset two-dimensional clustering algorithm.

[0139] In an embodiment of the present application, the vertex determination module 402 comprises:

[0140] a view switching submodule, configured to switch the display interface from the first view to the second view and display the third vertex and the fourth vertex on the second view in response to the user determining the second vertex, wherein the second view does not include the first vertex;

[0141] a vertex position updating submodule, configured to update the positions of the third vertex and the fourth vertex according to the user's correction operation.

[0142] As can be seen from the above, the target labeling device provided by the embodiments of the present application can obtain the target bounding box of the target to be labeled by clustering the point cloud in the region of interest determined in the first point cloud projection image, and the target bounding box obtained can reflect the target information of the target to be labeled displayed in the first view, so that the user no longer needs to repeatedly switch the first view and the second view for labeling, but only needs to operate in the second view, and the user can update the positions of the third vertex and the fourth vertex, so that the three-dimensional bounding box determined based thereon is more accurate.

[0143] In an embodiment of the present application, the vertex position updating submodule is specifically configured to:

[0144] determine the region of interest based on the first vertex, the second vertex and the third vertex corrected by the user, cluster the second target point cloud in the region of interest to obtain an updated target bounding box, and determine the fourth vertex again based on the updated target bounding box.

[0145] In an embodiment of the present application, the device further comprises:

[0146] a view switching module, configured to display the fourth vertex determined again in the second view in response to the user's correction operation on the third vertex.

[0147] In an embodiment of the present application, the target bounding box comprises a three-dimensional bounding box and a two-dimensional bounding box, and the device further comprises:

[0148] The first vertex determining module is configured to determine a first tentative third vertex according to the two-dimensional bounding box.

[0149] The second vertex determining module is configured to determine a second tentative third vertex according to the three-dimensional bounding box.

[0150] The third vertex determining module is configured to determine the third vertex according to the first tentative third vertex and the second tentative third vertex.

[0151] The embodiment of the present application further provides an electronic device, as shown in the accompanying drawings, Figure 5 The processor 501, the communication interface 502 and the memory 503 can communicate with each other through the communication bus 504,

[0152] The memory 503 is configured to store a computer program.

[0153] The processor 501 is configured to execute the program stored in the memory 503, so as to realize the method steps of any target labeling.

[0154] The communication bus mentioned in the above electronic device can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is configured to realize the communication between the above electronic device and other devices.

[0156] The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0157] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0158] In yet another embodiment provided by the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the method for target labeling according to any of the above embodiments.

[0159] In yet another embodiment provided by the present application, a computer program product is provided, and the computer program product includes instructions. When the computer program product is executed on a computer, the computer is caused to perform the method for target labeling according to any of the above embodiments.

[0160] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is entirely or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0161] It is to be noted that, in the present text, the terms such as first and second, and the like, are used merely to differentiate one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "containing", or any other variant thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements in the list, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the stated element.

[0162] The various embodiments in the present specification are described in a related manner, and the same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device, electronic device, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0163] The above only describes the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A target labeling method, characterized in that: The method comprises: In response to a first vertex and a second vertex determined by a user in a first view of the original point cloud data, a first plane perpendicular to a first edge and passing through the first vertex, and a second plane perpendicular to the first edge and passing through the second vertex are determined based on the first vertex and the second vertex, to obtain a point cloud coverage range, wherein the point cloud coverage range is an area between the first plane and the second plane in the original point cloud data, and the first edge is an edge formed by the first vertex and the second vertex; Determine a point cloud within the coverage range of the point cloud and having a distance from the first edge less than a preset threshold as a target point cloud; Clustering the target point cloud determined by the first vertex and the second vertex to obtain a target bounding box; Automatically displaying a third vertex and a fourth vertex according to the target bounding box, wherein the third vertex and the fourth vertex are boundary points of the target bounding box; A three-dimensional bounding box of the object to be labeled is determined according to the first vertex, the second vertex, the third vertex, and the fourth vertex.

2. The method according to claim 1, characterized in that The target bounding box includes a three-dimensional bounding box or a two-dimensional bounding box, and clustering the target point cloud determined by the first vertex and the second vertex to obtain the target bounding box includes: Using a preset three-dimensional clustering algorithm, the target point cloud is clustered to obtain the three-dimensional bounding box; or, Projecting the target point cloud onto a second view direction to obtain a first point cloud projection image; the second view does not include the first vertex; A preset two-dimensional clustering algorithm is used to determine a two-dimensional bounding box in the first point cloud projection image.

3. The method according to claim 2, characterized in that The automatically displaying the third vertex and the fourth vertex according to the target bounding box includes: In response to the user determining the second vertex, switching the display interface from the first view to the second view, and displaying the third vertex and the fourth vertex on the second view, wherein the second view does not include the first vertex; The positions of the third vertex and the fourth vertex are updated according to the correction operation of the user.

4. The method according to claim 3, characterized in that The updating of the positions of the third vertex and the fourth vertex according to the correction operation of the user includes: An area of ​​interest is determined based on the first vertex, the second vertex, and the third vertex corrected by the user, and a second target point cloud in the area of ​​interest is clustered to obtain a fourth vertex.

5. The method according to claim 4, characterized in that include: In response to the user's correction operation on the third vertex, a re-determined fourth vertex is displayed in the second view.

6. The method according to claim 2, characterized in that The target bounding box includes a three-dimensional bounding box and a two-dimensional bounding box, and the method further includes: Determining a first proposed third vertex according to the two-dimensional bounding box; determining a second proposed third vertex according to the three-dimensional bounding box; The third vertex is determined according to the first proposed third vertex and the second proposed third vertex.

7. A target marking device, characterized in that: The device comprises: a target bounding box determination module, configured to respond to a first vertex and a second vertex determined by a user in a first view of the original point cloud data; a point cloud coverage range determination module, configured to determine, based on the first vertex and the second vertex, a first plane perpendicular to the first edge and passing through the first vertex, and a second plane perpendicular to the first edge and passing through the second vertex, to obtain a point cloud coverage range, wherein the point cloud coverage range is an area between the first plane and the second plane in the original point cloud data, and the first edge is an edge formed by the first vertex and the second vertex; a target point cloud determination module, configured to determine a point cloud within the coverage range of the point cloud and having a distance from the first edge less than a preset threshold as a target point cloud; The target bounding box determination module is further configured to cluster the target point cloud determined by the first vertex and the second vertex to obtain a target bounding box; a vertex determination module, configured to determine a third vertex and a fourth vertex according to the target bounding box; The three-dimensional bounding box determination module determines a three-dimensional bounding box of the target to be labeled according to the first vertex, the second vertex, the third vertex, and the fourth vertex.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 6 are implemented.

Images