Object labeling method and device, electronic equipment and storage medium

Abstract

The invention discloses an object labeling method and device, electronic equipment and a storage medium. The method comprises the steps that point cloud data corresponding to a to-be-labeled area and multiple voxel grids corresponding to the point cloud data are acquired; the point cloud data comprises a plurality of sub-point cloud data, each voxel grid corresponds to one sub-point cloud data, and the to-be-labeled area comprises a to-be-labeled object; determining first category information corresponding to each voxel grid based on the intensity information and the height information corresponding to each piece of sub-point cloud data; based on the position information of each voxel grid in the plurality of voxel grids, determining at least one first voxel grid corresponding to the preset labeling box; on the basis of the first category information corresponding to each first voxel grid, the preset annotation box is updated, a first target annotation box of the to-be-annotated object is obtained, and the range of the first target annotation box is smaller than that of the preset annotation box. According to the embodiment of the invention, the accuracy and efficiency of object labeling can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an object labeling method, device, electronic device and storage medium. Background Art

[0002] In the field of autonomous driving related technologies, the annotation of point cloud data is crucial. Point cloud data annotation is to mark the target objects (such as vehicles, pedestrians, advertising signs and trees, etc.) in the 3D images collected by the lidar through 3D annotation boxes. The accuracy and efficiency of point cloud annotation in existing technologies are low, especially when annotating large data point clouds, which takes a long time. Summary of the invention

[0003] In view of the above problems in the prior art, the present invention discloses an object annotation method, device, electronic device and storage medium, which can accurately and quickly generate an annotation box for an object to be annotated, realize automatic and rapid annotation, and thus improve the accuracy and efficiency of object annotation. The technical solution disclosed in the present invention is as follows:

[0004] According to one aspect of the disclosed embodiment of the present invention, there is provided an object labeling method, comprising:

[0005] Acquire point cloud data corresponding to the area to be labeled, and a plurality of voxel grids corresponding to the point cloud data; the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the area to be labeled includes the object to be labeled;

[0006] Based on the intensity information and height information corresponding to each sub-point cloud data, determining the first category information corresponding to each voxel grid; the first category information is used to characterize the category information of the object corresponding to each voxel grid;

[0007] Determine at least one first voxel grid corresponding to a preset annotation box based on the position information of each voxel grid in the plurality of voxel grids;

[0008] Based on the first category information corresponding to each first voxel grid, the preset annotation box is updated to obtain a first target annotation box of the object to be annotated; the range of the first target annotation box is smaller than the range of the preset annotation box.

[0009] Optionally, the determining, based on the intensity information and the height information corresponding to each sub-point cloud data, the first category information corresponding to each voxel grid includes:

[0010] Based on the intensity information of each point in each sub-point cloud data, determining the intensity category information corresponding to each point;

[0011] Based on the height information of each point, determine the height category information corresponding to each point;

[0012] Performing weighted summation on the intensity category information and the height category information to obtain second category information corresponding to each point; the second category information is category information of the object corresponding to each point;

[0013] Based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid is determined.

[0014] Optionally, the determining, based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid comprises:

[0015] Determine the second category information of each point corresponding to each voxel grid, and the first category proportion information among the multiple points corresponding to each voxel grid;

[0016] When the first category proportion information is greater than a first preset proportion threshold, the corresponding second category information is determined as the first category information.

[0017] Optionally, the determining, based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid comprises:

[0018] Determine weight information corresponding to each second category information based on the number of points corresponding to each second category information in each voxel grid;

[0019] Based on the weight information, weighted sum is performed on each second category information to obtain the first category information.

[0020] Optionally, the updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated includes:

[0021] Determine first category information corresponding to each of the first voxel grids and second category proportion information in the plurality of first voxel grids;

[0022] When the second category proportion information is greater than a second preset proportion threshold, determining the corresponding first voxel grid as a first target voxel grid;

[0023] The preset annotation box is updated based on the first target voxel grid to obtain the first target annotation box; the first target voxel grid is located within the range of the first target annotation box, and the non-first target voxel grid is not located within the range of the first target annotation box, and the non-first target voxel grid is a voxel grid other than the first target voxel grid in the multiple first voxel grids.

[0024] Optionally, the position information includes horizontal position information, the first voxel grid includes a second voxel grid, and determining at least one first voxel grid corresponding to a preset annotation box based on the position information of each voxel grid in the plurality of voxel grids includes:

[0025] Based on the horizontal position information of each voxel grid, determining at least one second voxel grid corresponding to the preset annotation box in the horizontal direction;

[0026] Correspondingly, the updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated includes:

[0027] In the horizontal direction, based on the first category information corresponding to each of the second voxel grids, the preset annotation box is updated to obtain an updated annotation box of the object to be annotated;

[0028] In the vertical direction, a second target voxel grid is determined from the first voxel grid corresponding to the updated annotation box; the second target voxel grid has the same first category information as the third target voxel grid, and the third target voxel grid is the second voxel grid corresponding to the updated annotation box;

[0029] The updated annotation box is updated based on the second target voxel grid to obtain the first target annotation box; the second target voxel grid and the third target voxel grid are both located within the scope of the first target annotation box, and other voxel grids are not located within the scope of the first target annotation box, and the other voxel grids are voxel grids among the multiple first voxel grids, except for the second target voxel grid and the third target voxel grid.

[0030] Optionally, after the preset annotation box is updated based on the first category information corresponding to each first voxel grid to obtain a first target annotation box of the object to be annotated, the method further includes:

[0031] When the intensity information of the point corresponding to the boundary voxel grid satisfies the intensity range, the corresponding point is determined as the target point; the boundary voxel grid is the voxel grid corresponding to the boundary of the first target annotation box, and the intensity range is the intensity range corresponding to the first category information of the boundary voxel grid;

[0032] The first target annotation box is updated based on the target point to obtain a second target annotation box; the target point is within the range of the second target annotation box, and the non-target point is not within the range of the second target annotation box, and the non-target point is a point other than the target point among the points corresponding to the boundary voxel grid.

[0033] According to another aspect of the disclosed embodiment of the present invention, there is provided an object labeling device, comprising:

[0034] An acquisition module is used to acquire point cloud data corresponding to the area to be labeled and a plurality of voxel grids corresponding to the point cloud data; the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the area to be labeled includes an object to be labeled;

[0035] A first category information determination module, used to determine first category information corresponding to each voxel grid based on intensity information and height information corresponding to each sub-point cloud data; the first category information is used to characterize category information of an object corresponding to each voxel grid;

[0036] A first voxel grid determination module, configured to determine at least one first voxel grid corresponding to a preset annotation box based on position information of each voxel grid in the plurality of voxel grids;

[0037] The updating module is used to update the preset annotation box based on the first category information corresponding to each first voxel grid to obtain a first target annotation box of the object to be annotated.

[0038] According to another aspect of the disclosed embodiment of the present invention, there is provided an electronic device for object labeling, including a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement any of the above-mentioned object labeling methods.

[0039] According to another aspect of the disclosed embodiment of the present invention, a computer-readable storage medium is provided, wherein at least one instruction is stored in the computer storage medium, and the at least one instruction is loaded and executed by a processor to implement any of the above-mentioned object labeling methods.

[0040] According to another aspect of the embodiments disclosed in the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any one of the object labeling methods described above in the embodiments disclosed in the present invention.

[0041] The object labeling method provided by the present invention has the following technical effects:

[0042] The object labeling method provided by the present invention first obtains point cloud data corresponding to a to-be-labeled area and a plurality of voxel grids corresponding to the point cloud data, wherein the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the to-be-labeled area includes the to-be-labeled object; then, based on the intensity information and height information corresponding to each sub-point cloud data, first category information corresponding to each voxel grid is determined, wherein the first category information is used to characterize the category information of the object corresponding to each voxel grid; then, based on the position information of each voxel grid in the plurality of voxel grids, at least one first voxel grid corresponding to a preset labeling box is determined, and then based on the first category information corresponding to each first voxel grid, the preset labeling box is updated to obtain a first target labeling box of the to-be-labeled object, thereby accurately and quickly generating a labeling box of the to-be-labeled object, realizing automatic and fast labeling, thereby improving the accuracy and efficiency of object labeling.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a flowchart of an object labeling method according to an exemplary embodiment;

[0046] Figure 2 is a schematic diagram of a process for determining first category information according to an exemplary embodiment;

[0047] Figure 3 is a schematic diagram of another process of determining first category information according to an exemplary embodiment;

[0048] Figure 4 is a schematic diagram of another process of determining first category information according to an exemplary embodiment;

[0049] Figure 5 is a schematic diagram of a process of determining a first target annotation frame according to an exemplary embodiment;

[0050] Figure 6 is a schematic diagram of another process of determining a first target annotation frame according to an exemplary embodiment;

[0051] Figure 7is a block diagram of an object labeling device according to an exemplary embodiment;

[0052] Figure 8 is a block diagram of a terminal electronic device for object annotation according to an exemplary embodiment;

[0053] Fig. 9 The invention is a block diagram of a server electronic device for object annotation according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] In order to enable ordinary persons in the art to better understand the technical solutions disclosed in the present invention, the technical solutions in the embodiments disclosed in the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary persons in the art without making creative work are within the scope of protection of the present invention.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims disclosed in the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments disclosed in the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0056] See also Figure 1 , Figure 1 It is a flowchart of an object labeling method shown according to an exemplary embodiment. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, as Figure 1 As shown, the above method may include:

[0057] S101: Obtain point cloud data corresponding to the area to be labeled, and a plurality of voxel grids corresponding to the point cloud data.

[0058] In a specific embodiment, the area to be annotated may be a three-dimensional area that needs to be annotated, such as the surrounding area of ​​a vehicle. The area to be annotated may include objects to be annotated. The objects to be annotated may be any object in the environment. Each object may correspond to a category information. Specifically, the category information may include vehicles, pedestrians, trees, and advertising signs. The point cloud data of the area to be annotated may be three-dimensional point cloud data, which may be collected by radar equipment. The point cloud data may include multiple sub-point cloud data. Specifically, the area to be annotated may be divided into several sub-areas, and each sub-point cloud data may correspond to a sub-area.

[0059] Optionally, the above point cloud data can be voxelized to obtain multiple voxel grids corresponding to the point cloud data. Specifically, each voxel grid can correspond to a sub-point cloud data. In practical applications, the three-dimensional area to be labeled can be evenly divided into a number of cubes. The three-dimensional area can correspond to the point cloud data. Each cube area can contain a number of points, and each cube containing a number of points is a voxel grid. The size of each voxel grid can be set according to the actual application requirements. For example, the grid side length can be set to any value between 0.5 meters and 1 meter.

[0060] S103: Determine first category information corresponding to each voxel grid based on the intensity information and height information corresponding to each sub-point cloud data.

[0061] In a specific embodiment, the point cloud data may include three-dimensional coordinate information and intensity information of each point, the three-dimensional coordinate information may include coordinate information in the horizontal direction (X-axis and Y-axis direction) and coordinate information in the vertical direction (Z-axis direction) (i.e., height information), and the intensity information of each point may be used to characterize the reflection intensity of the laser at the point. The first category information may be used to characterize the category information of the object corresponding to each voxel grid, i.e., the category information of the object in the three-dimensional region corresponding to the voxel grid.

[0062] In an alternative embodiment, Figure 2 is a schematic diagram of a process for determining first category information according to an exemplary embodiment. Figure 2 As shown, the determination of the first category information corresponding to each voxel grid based on the intensity information and height information corresponding to each sub-point cloud data may include:

[0063] S201: Based on the intensity information of each point in each sub-point cloud data, determine the intensity category information corresponding to each point.

[0064] In a specific embodiment, the intensity category information may be category information of an object corresponding to the point determined according to the intensity information of the point, and the intensity information of each point may correspond to one intensity category information. Specifically, the intensity category information corresponding to the intensity information of each point may be determined according to the intensity information of each point and a first preset correspondence relationship between the intensity information and the intensity category information, and the first preset correspondence relationship may be set according to actual application requirements.

[0065] S203: Based on the height information of each point, determine the height category information corresponding to each point.

[0066] In a specific embodiment, the height information of a point may be the coordinate information of the point in the vertical direction (Z-axis direction). The height category information may be the category information of the object corresponding to the point determined according to the height information of the point, and the height information of each point may correspond to one type of height category information. Specifically, the height category information corresponding to the height information of each point may be determined according to the height information of each point and the second preset correspondence between the height information and the height category information, and the second preset correspondence may be set according to actual application requirements.

[0067] S205: Perform weighted summation on the intensity category information and the height category information to obtain second category information corresponding to each point.

[0068] In a specific embodiment, the second category information may be category information of an object corresponding to each point, and each point may correspond to a second category information. Based on the weights corresponding to the intensity category information and the height category information, the intensity category information and the height category information may be weighted and summed to obtain the second category information corresponding to each point. Specifically, the weights corresponding to the intensity category information and the height category information may be set according to actual application requirements, and the weight corresponding to the intensity category information may be greater than the weight corresponding to the height category information. For example, the weight corresponding to the intensity category information may be 0.8, and the weight corresponding to the height category information may be 0.2.

[0069] S207: Determine the first category information corresponding to each voxel grid based on the second category information of the plurality of points corresponding to each voxel grid.

[0070] In an alternative embodiment, Figure 3 is a schematic diagram of another process of determining first category information according to an exemplary embodiment. Figure 3 As shown, the determination of the first category information corresponding to each voxel grid based on the second category information of the plurality of points corresponding to each voxel grid may include:

[0071] S301: Determine second category information of each point corresponding to each voxel grid, and first category proportion information of a plurality of points corresponding to each voxel grid.

[0072] In a specific embodiment, the multiple points corresponding to each voxel grid may correspond to one or more second category information. Specifically, the first category proportion information may be the proportion of each second category information in the second category information of all points corresponding to the voxel grid.

[0073] S303: When the first category proportion information is greater than a first preset proportion threshold, the corresponding second category information is determined as the first category information.

[0074] In a specific embodiment, the category information with a larger proportion among the second category information corresponding to all points in the voxel grid can be determined as the category information of the voxel grid. The first preset proportion threshold can be set according to actual application requirements.

[0075] In an alternative embodiment, Figure 4 is a schematic diagram of another process of determining first category information according to an exemplary embodiment. Figure 4 As shown, the determination of the first category information corresponding to each voxel grid based on the second category information of the plurality of points corresponding to each voxel grid may also include:

[0076] S401: Determine weight information corresponding to each second category information based on the number of points corresponding to each second category information in each voxel grid.

[0077] In a specific embodiment, the weight information corresponding to each second category information can be determined based on the number of points corresponding to each second category information in each voxel grid, and the third preset correspondence between the number of points and the weight information. Specifically, in a voxel grid, the more points whose category information is the second category information, the greater the weight information corresponding to the second category information. The third preset correspondence can be set according to actual application requirements.

[0078] S403: Based on the weight information, perform weighted summation on each second category information to obtain the first category information.

[0079] In a specific embodiment, the second category information corresponding to a voxel grid may be weighted and summed based on the weight information corresponding to each second category information, so as to obtain the first category information corresponding to the voxel grid.

[0080] S105: Determine at least one first voxel grid corresponding to a preset annotation box based on the position information of each voxel grid in the plurality of voxel grids.

[0081] In a specific embodiment, the position information of each voxel grid can be used to characterize the range of each voxel grid, and the position information can be determined based on the coordinate information of the grid points of each voxel grid in three-dimensional space. Specifically, the range of each voxel grid can be determined based on the lower limit coordinate information and upper limit coordinate information of the grid points of each voxel grid. The preset annotation box can be a box box with a larger range that includes the object to be annotated. The environment area corresponding to the preset annotation box can include the object to be annotated and other objects. Specifically, each boundary point of the preset annotation box can correspond to a three-dimensional coordinate information. The voxel grid corresponding to the preset annotation box can be the voxel grid corresponding to the area covered by the preset annotation box. Specifically, it can be a voxel grid located at the boundary and included range of the preset annotation box. Among the voxel grids corresponding to the area to be annotated, the voxel grid corresponding to the preset annotation box is screened out.

[0082] In practical applications, the above-mentioned preset annotation box can be drawn by the user or AI in a three-dimensional scene.

[0083] S107: Based on the first category information corresponding to each first voxel grid, the preset annotation box is updated to obtain a first target annotation box of the object to be annotated.

[0084] In a specific embodiment, the range of the first target annotation box may be smaller than the range of the preset annotation box. Specifically, the preset annotation box may be reduced based on the first category information corresponding to each first voxel grid, thereby obtaining the first target annotation box of the object to be annotated.

[0085] In an alternative embodiment, Figure 5 is a schematic diagram of a process of determining a first target annotation frame according to an exemplary embodiment. Figure 5 As shown, the above-mentioned updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated may include:

[0086] S501: Determine first category information corresponding to each first voxel grid, and second category proportion information in a plurality of first voxel grids.

[0087] In a specific embodiment, each first voxel grid may correspond to one type of first category information, multiple first voxel grids corresponding to the preset annotation box may correspond to one or more types of first category information, and the second category proportion information may be the proportion information of each type of first category information in the first category information corresponding to the multiple first voxel grids.

[0088] S503: When the second category proportion information is greater than a second preset proportion threshold, determine the corresponding first voxel grid as a first target voxel grid.

[0089] In a specific embodiment, a voxel grid with a larger proportion of first category information among the plurality of first voxel grids corresponding to the preset annotation box can be determined as the first target voxel grid. The second preset proportion threshold can be set according to actual application requirements.

[0090] S505: updating the preset annotation box based on the first target voxel grid to obtain a first target annotation box.

[0091] In a specific embodiment, the preset annotation box can be reduced based on the first target voxel grid to obtain the first target annotation box. Specifically, the first target voxel grid can be located within the range of the first target annotation box, the non-first target voxel grid can be located outside the range of the first target annotation box, and the non-first target voxel grid can be a voxel grid other than the first target voxel grid among multiple first voxel grids.

[0092] In practical applications, the voxel grids with the first category information that accounts for a relatively large proportion of the category information can be screened out from the multiple first voxel grids corresponding to the preset annotation box to obtain a box that can surround these voxel grids, that is, the above-mentioned first target annotation box, so that the generated annotation box of the object to be annotated is more accurate.

[0093] In an optional embodiment, the determining, based on the position information of each voxel grid in the multiple voxel grids, at least one first voxel grid corresponding to the preset annotation box may include:

[0094] At least one second voxel grid corresponding to the preset annotation box in the horizontal direction is determined based on the horizontal position information of each voxel grid.

[0095] In a specific embodiment, the position information of each voxel grid may include horizontal position information and vertical position information, the horizontal position information may be the coordinate information of the grid point in the horizontal direction (X-axis and Y-axis direction), and the vertical position information may be the coordinate information of the grid point in the vertical direction (Z-axis direction). Specifically, the voxel grids covered by the preset annotation box range may be firstly screened from the voxel grids corresponding to the to-be-annotated area in the horizontal direction according to the coordinate information of the grid points and the range of the preset annotation box.

[0096] Accordingly, Figure 6 is a schematic diagram of another process of determining a first target annotation frame according to an exemplary embodiment. Figure 6 As shown, the above-mentioned updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated may include:

[0097] S601: In the horizontal direction, based on the first category information corresponding to each second voxel grid, a preset annotation box is updated to obtain an updated annotation box of the object to be annotated.

[0098] In a specific embodiment, the range of the updated annotation box is smaller than the preset annotation box. In the horizontal direction, the voxel grid whose category information is the first category information with a relatively large proportion in the second voxel grid can be determined as the third target voxel grid, and the preset annotation box is updated based on the third target voxel grid, thereby obtaining an updated annotation box of the object to be annotated. The specific process can refer to the above-mentioned refinement step of updating the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated, which will not be repeated here.

[0099] S603: In the vertical direction, determine a second target voxel grid from the first voxel grid corresponding to the updated annotation box.

[0100] In a specific embodiment, the third target voxel grid may be the second voxel grid corresponding to the update annotation box, the second target voxel grid may have the same first category information as the third target voxel grid, and the first category information of the third target voxel grid may be the same. Specifically, in the vertical direction, the first category information of the third target voxel grid screened in the horizontal direction is used as a reference to perform voxel screening in the vertical direction. Specifically, the voxel grid whose category information is the same as the first category information of the third target voxel grid in the first voxel grid corresponding to the update annotation box in the vertical direction is determined as the second target voxel grid.

[0101] S605: Update the updated annotation box based on the second target voxel grid to obtain the first target annotation box.

[0102] In a specific embodiment, the preset annotation box can be reduced based on the second target voxel grid to obtain the first target annotation box, the second target voxel grid and the third target voxel grid are both located within the scope of the first target annotation box, and other voxel grids are not located within the scope of the first target annotation box. The other voxel grids can be voxel grids among multiple first voxel grids, except for the second target voxel grid and the third target voxel grid.

[0103] In practical applications, we can first filter out the voxel grids whose category information is the first category information that accounts for a relatively large proportion among the multiple voxel grids corresponding to the preset annotation box in the horizontal direction, and then further filter out the voxel grids whose category information is the same as the first category information of the above-filtered voxel grids in the vertical direction. Based on the voxel grids filtered out twice, we obtain a box surrounding these voxel grids, that is, the above-mentioned first target annotation box, so that the generated annotation box of the object to be labeled is more accurate.

[0104] In an optional embodiment, after the preset annotation box is updated based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated, the method may further include:

[0105] When the intensity information of the point corresponding to the boundary voxel grid satisfies the intensity range, the corresponding point is determined as the target point;

[0106] The first target annotation box is updated based on the target point to obtain a second target annotation box.

[0107] In a specific embodiment, the boundary voxel grid may be a voxel grid corresponding to the boundary of the first target annotation box, and the category information of the boundary voxel grid is the same as that of the second target voxel grid. The intensity range may be an intensity range corresponding to the first category information of the boundary voxel grid. Specifically, the intensity range may be determined according to a fourth preset correspondence between the first category information of the boundary voxel grid and the intensity range. The fourth preset correspondence may be set according to actual application requirements.

[0108] In a specific embodiment, the range of the second target annotation box is smaller than the range of the first target annotation box, the target point may be within the range of the second target annotation box, the non-target point may not be within the range of the second target annotation box, and the non-target point may be a point other than the target point among the points corresponding to the boundary voxel grid. Each first category information may correspond to an intensity range, and the point corresponding to the boundary voxel grid and the point with the same first category information as the boundary voxel grid is determined as the above target point, and the first target annotation box is reduced based on the target point to obtain the second target annotation box, further reducing the error of the annotation box and improving the accuracy of the annotation.

[0109] It can be seen from the technical solutions provided in the above embodiments of this specification that, in this specification, point cloud data corresponding to the area to be labeled and multiple voxel grids corresponding to the point cloud data are first obtained, wherein the point cloud data includes multiple sub-point cloud data, each voxel grid corresponds to one sub-point cloud data, and the area to be labeled includes the object to be labeled; then, based on the intensity information and height information corresponding to each sub-point cloud data, the first category information corresponding to each voxel grid is determined, wherein the first category information is used to characterize the category information of the object corresponding to each voxel grid; then, based on the position information of each voxel grid in the multiple voxel grids, at least one first voxel grid corresponding to the preset annotation box is determined, and then based on the first category information corresponding to each first voxel grid, the preset annotation box is updated to obtain the first target annotation box of the object to be labeled, thereby, the annotation box of the object to be labeled can be accurately and quickly generated, automatic and fast annotation can be achieved, thereby improving the accuracy and efficiency of object annotation.

[0110] The embodiment of the present invention also provides an object labeling device, such as Figure 7 As shown, the device comprises:

[0111] The acquisition module 710 is used to acquire point cloud data corresponding to the area to be labeled and a plurality of voxel grids corresponding to the point cloud data; the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the area to be labeled includes the object to be labeled;

[0112] A first category information determination module 720 is used to determine the first category information corresponding to each voxel grid based on the intensity information and height information corresponding to each sub-point cloud data; the first category information is used to characterize the category information of the object corresponding to each voxel grid;

[0113] A first voxel grid determination module 730 is used to determine at least one first voxel grid corresponding to a preset annotation box based on the position information of each voxel grid in the multiple voxel grids;

[0114] The updating module 740 is configured to update the preset annotation box based on the first category information corresponding to each first voxel grid to obtain a first target annotation box of the object to be annotated.

[0115] Optionally, the first category information determining module 720 includes:

[0116] an intensity category information determining unit, configured to determine intensity category information corresponding to each point based on the intensity information of each point in each sub-point cloud data;

[0117] A height category information determining unit, configured to determine the height category information corresponding to each point based on the height information of each point;

[0118] A second category information determination unit is configured to perform a weighted summation on the intensity category information and the height category information to obtain second category information corresponding to each point; the second category information is category information of the object corresponding to each point;

[0119] The first category information determining unit is used to determine the first category information corresponding to each voxel grid based on the second category information of the plurality of points corresponding to each voxel grid.

[0120] Optionally, the first category information determining unit includes:

[0121] A first category proportion information determining unit, used to determine the second category information of each point corresponding to each voxel grid, and the first category proportion information of the plurality of points corresponding to each voxel grid;

[0122] The third category information determining unit is configured to determine the corresponding second category information as the first category information when the first category proportion information is greater than a first preset proportion threshold.

[0123] Optionally, the first category information determining unit includes:

[0124] A weight information determining unit, configured to determine the weight information corresponding to each second category information based on the number of points corresponding to each second category information in each voxel grid;

[0125] The fourth category information determination unit is configured to perform weighted summation on each piece of second category information based on the weight information to obtain the first category information.

[0126] Optionally, the updating module 740 includes:

[0127] A second category proportion information determining unit, configured to determine the first category information corresponding to each of the first voxel grids, and the second category proportion information in the plurality of first voxel grids;

[0128] A first target voxel grid determining unit, configured to determine the corresponding first voxel grid as a first target voxel grid when the second category proportion information is greater than a second preset proportion threshold;

[0129] A first updating unit is used to update the preset annotation box based on the first target voxel grid to obtain the first target annotation box; the first target voxel grid is located within the range of the first target annotation box, and the non-first target voxel grid is not located within the range of the first target annotation box, and the non-first target voxel grid is a voxel grid other than the first target voxel grid in the multiple first voxel grids.

[0130] Optionally, the position information includes horizontal position information, the first voxel grid includes a second voxel grid, and the first voxel grid determination module 730 includes:

[0131] A second voxel grid determining unit, configured to determine at least one second voxel grid corresponding to the preset annotation frame in the horizontal direction based on the horizontal position information of each voxel grid;

[0132] Accordingly, the updating module 740 includes:

[0133] an updated annotation box determining unit, configured to update the preset annotation box in a horizontal direction based on the first category information corresponding to each of the second voxel grids to obtain an updated annotation box of the object to be annotated;

[0134] A second target voxel grid determining unit is used to determine a second target voxel grid from the first voxel grid corresponding to the updated annotation box in a vertical direction; the second target voxel grid has the same first category information as a third target voxel grid, and the third target voxel grid is the second voxel grid corresponding to the updated annotation box;

[0135] A second updating unit is used to update the updated annotation box based on the second target voxel grid to obtain the first target annotation box; the second target voxel grid and the third target voxel grid are both located within the scope of the first target annotation box, and other voxel grids are not located within the scope of the first target annotation box, and the other voxel grids are voxel grids among the multiple first voxel grids, except for the second target voxel grid and the third target voxel grid.

[0136] Optionally, the device further comprises:

[0137] a target point determination unit, configured to determine the corresponding point as a target point when the intensity information of the point corresponding to the boundary voxel grid satisfies an intensity range; the boundary voxel grid is a voxel grid corresponding to the boundary of the first target annotation box, and the intensity range is an intensity range corresponding to the first category information of the boundary voxel grid;

[0138] A third updating unit is used to update the first target annotation box based on the target point to obtain a second target annotation box; the target point is located within the range of the second target annotation box, and the non-target point is not located within the range of the second target annotation box, and the non-target point is a point other than the target point among the points corresponding to the boundary voxel grid.

[0139] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] Figure 8 is a block diagram of an electronic device for object annotation according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an object labeling method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0141] Fig. 9 is a block diagram of an electronic device for object annotation according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an object labeling method is implemented.

[0142] Those skilled in the art will understand that Figure 8 or Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the disclosed solution of the present invention, and does not constitute a limitation on the electronic device to which the disclosed solution of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] In an exemplary embodiment, an electronic device for object labeling is also provided, including a processor and a memory, wherein at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the object labeling method in the disclosed embodiment of the present invention.

[0144] In an exemplary embodiment, a computer-readable storage medium is further provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the object labeling method in the disclosed embodiment of the present invention.

[0145] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to perform the object labeling method in the disclosed embodiment of the present invention.

[0146] Those of ordinary skill in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate (DDR), and the like.

[0147] SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM), etc.

[0148] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0149] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An object labeling method, characterized in that: include: Acquire point cloud data corresponding to the area to be labeled, and a plurality of voxel grids corresponding to the point cloud data; the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the area to be labeled includes the object to be labeled; Determine the first category information corresponding to each voxel grid based on the intensity information and the height information corresponding to each sub-point cloud data; The first category information is used to represent category information of the object corresponding to each voxel grid; Based on the position information of each voxel grid in the plurality of voxel grids, determining at least one first voxel grid corresponding to a preset annotation box; Based on the first category information corresponding to each first voxel grid, the preset annotation box is updated to obtain a first target annotation box of the object to be annotated; The range of the first target annotation box is smaller than the range of the preset annotation box.

2. The method according to claim 1, characterized in that The determining, based on the intensity information and the height information corresponding to each sub-point cloud data, the first category information corresponding to each voxel grid comprises: Based on the intensity information of each point in each sub-point cloud data, determining the intensity category information corresponding to each point; Based on the height information of each point, determine the height category information corresponding to each point; Performing weighted summation on the intensity category information and the height category information to obtain second category information corresponding to each point; the second category information is category information of the object corresponding to each point; Based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid is determined.

3. The method according to claim 2, characterized in that The determining, based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid comprises: Determine the second category information of each point corresponding to each voxel grid, and the first category proportion information among the multiple points corresponding to each voxel grid; When the first category proportion information is greater than a first preset proportion threshold, the corresponding second category information is determined as the first category information.

4. The method according to claim 2, characterized in that: The determining, based on the second category information of the plurality of points corresponding to each voxel grid, the first category information corresponding to each voxel grid comprises: Determine weight information corresponding to each second category information based on the number of points corresponding to each second category information in each voxel grid; Based on the weight information, weighted sum is performed on each second category information to obtain the first category information.

5. The method according to claim 1, characterized in that The updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated includes: Determine first category information corresponding to each of the first voxel grids and second category proportion information in the plurality of first voxel grids; When the second category proportion information is greater than a second preset proportion threshold, determining the corresponding first voxel grid as a first target voxel grid; The preset annotation box is updated based on the first target voxel grid to obtain the first target annotation box; the first target voxel grid is located within the range of the first target annotation box, and the non-first target voxel grid is not located within the range of the first target annotation box, and the non-first target voxel grid is a voxel grid other than the first target voxel grid in the multiple first voxel grids.

6. The method according to claim 1, characterized in that The position information includes horizontal position information, the first voxel grid includes a second voxel grid, and determining at least one first voxel grid corresponding to a preset annotation box based on the position information of each voxel grid in the plurality of voxel grids includes: Based on the horizontal position information of each voxel grid, determining at least one second voxel grid corresponding to the preset annotation box in the horizontal direction; Correspondingly, the updating of the preset annotation box based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated includes: In the horizontal direction, based on the first category information corresponding to each of the second voxel grids, the preset annotation box is updated to obtain an updated annotation box of the object to be annotated; In the vertical direction, a second target voxel grid is determined from the first voxel grid corresponding to the updated annotation box; the second target voxel grid has the same first category information as the third target voxel grid, and the third target voxel grid is the second voxel grid corresponding to the updated annotation box; The updated annotation box is updated based on the second target voxel grid to obtain the first target annotation box; the second target voxel grid and the third target voxel grid are both located within the scope of the first target annotation box, and other voxel grids are not located within the scope of the first target annotation box, and the other voxel grids are voxel grids among the multiple first voxel grids, except for the second target voxel grid and the third target voxel grid.

7. The method according to claim 1, characterized in that After the preset annotation box is updated based on the first category information corresponding to each first voxel grid to obtain the first target annotation box of the object to be annotated, the method further includes: When the intensity information of the point corresponding to the boundary voxel grid satisfies the intensity range, the corresponding point is determined as the target point; the boundary voxel grid is the voxel grid corresponding to the boundary of the first target annotation box, and the intensity range is the intensity range corresponding to the first category information of the boundary voxel grid; The first target annotation box is updated based on the target point to obtain a second target annotation box; the target point is within the range of the second target annotation box, and the non-target point is not within the range of the second target annotation box, and the non-target point is a point other than the target point among the points corresponding to the boundary voxel grid.

8. An object labeling device, characterized in that: The device comprises: An acquisition module is used to acquire point cloud data corresponding to the area to be labeled and a plurality of voxel grids corresponding to the point cloud data; the point cloud data includes a plurality of sub-point cloud data, each voxel grid corresponds to a sub-point cloud data, and the area to be labeled includes the object to be labeled; A first category information determination module, used to determine first category information corresponding to each voxel grid based on intensity information and height information corresponding to each sub-point cloud data; the first category information is used to characterize category information of an object corresponding to each voxel grid; A first voxel grid determination module, configured to determine at least one first voxel grid corresponding to a preset annotation box based on position information of each voxel grid in the plurality of voxel grids; The updating module is used to update the preset annotation box based on the first category information corresponding to each first voxel grid to obtain a first target annotation box of the object to be annotated.

9. An electronic device for object annotation, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the object labeling method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the object labeling method according to any one of claims 1 to 7.

Images