A data processing method and device, electronic device and computer readable storage medium

By filtering and deleting point cloud files and annotation files, combined with the threshold for the number of points in the 3D annotation box and manual inspection, the quality problem of point cloud data during the copying process was solved, and the data processing efficiency and the accuracy of algorithm development were improved.

CN114926662BActive Publication Date: 2024-11-22CHONGQING CHANGAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210601774.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-30
Publication Date
2024-11-22
Estimated Expiration
2042-05-30

AI Technical Summary

Technical Problem

In existing technologies, point cloud data is prone to quality problems during the copying process, resulting in unqualified data during algorithm development, affecting the accuracy of neural networks, and lacking efficient data quality verification methods.

Method used

By acquiring a set of point cloud files and annotation files, filtering and deletion operations are performed to ensure the correlation between the point cloud files and annotation files. Unqualified data is removed based on the point count threshold of the 3D annotation frame, and finally, a visual manual quality inspection is carried out.

Benefits of technology

It improved the efficiency of data processing, ensured the quality of point cloud data, reduced errors in the data verification process, and improved the accuracy of algorithm development.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114926662B_ABST
    Figure CN114926662B_ABST
Patent Text Reader

Abstract

The present application provides a data processing method and device, an electronic device and a computer-readable storage medium, comprising: obtaining a first point cloud file set and a first annotation file set; deleting the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file; deleting the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file; deleting the second annotation file from the second annotation file set to obtain a third annotation file set, and deleting the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box. By using this method, the efficiency of data processing can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning, and specifically to a data processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] The current laser point cloud target detection algorithm based on deep learning mainly inputs the labeled point cloud data into the deep neural network, and the deep neural network optimizes the weight of the neural network through the label and loss function. Therefore, the quality of the point cloud data annotation has a significant impact on the parameters of the neural network and the accuracy of the algorithm.

[0003] Due to system design reasons of different companies, these large amounts of labeled data that have passed quality inspection are often stored in the same database. When some of the data is needed for development, it is necessary to copy part of the data to another database for algorithm development. However, once a problem occurs in the copying process, the quality of the labeled data will be greatly reduced, which is undoubtedly a catastrophic error for algorithm development engineers. Since data processing in the labeling stage is relatively delicate and complex and is not suitable for algorithm development, it is very necessary to efficiently verify the quality of the data before using it. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a method to solve the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a data processing method, the method comprising:

[0006] Acquire a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated;

[0007] Deleting the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file;

[0008] The first annotation file is deleted from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file;

[0009] The second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box.

[0010] In one embodiment, the annotation file associated with the first point cloud file includes an identifier of the first point cloud file, and the point cloud file associated with the first annotation file includes the identifier of the first annotation file.

[0011] In one embodiment, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set. Before obtaining the third point cloud file set, it also includes: determining whether the number of points inside the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box, and the first three-dimensional annotation box is any three-dimensional annotation box in the second annotation file; if so, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set.

[0012] In one implementation, the first three-dimensional annotation frame is in the shape of a cuboid.

[0013] In one embodiment, the plane equation corresponding to each of the six faces of the first three-dimensional annotation box is determined based on the second annotation file; based on the plane equation corresponding to each face and the coordinate value of the first annotation point, it is determined whether the first annotation point is located in the first three-dimensional annotation box, and the first annotation point is any annotation point in the second annotation file.

[0014] In one embodiment, the eight vertex coordinates of the three-dimensional annotation box and the normal vector of each of the six faces of the first three-dimensional annotation box are calculated based on the center point of the first three-dimensional annotation box included in the second annotation file, the length, width and height of the first three-dimensional annotation box, and the heading angle of the first three-dimensional annotation box; the plane equation corresponding to each face is calculated based on the eight vertices of the first three-dimensional annotation box and the normal vector of each of the six faces.

[0015] In one embodiment, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain the third point cloud file set, and it also includes: extracting a preset number of point cloud files from the third point cloud file set, and extracting a preset number of annotation files respectively associated with a preset number of point cloud files from the third annotation file set, and the preset number of point cloud files and the preset number of annotation files are used for visual manual quality inspection.

[0016] In one embodiment, if the corresponding target object is annotated by the size and orientation of the second three-dimensional annotation box, the annotation data quality in the third point cloud file set and the third annotation file set is determined to be qualified, the second three-dimensional annotation box is included in the third annotation file, and the third annotation file is any one of a preset number of annotation files.

[0017] In a second aspect, an embodiment of the present application further provides a data processing device, including:

[0018] An acquiring unit, configured to acquire a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated;

[0019] A processing unit, configured to delete the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file;

[0020] The processing unit is further configured to delete the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file;

[0021] The above-mentioned processing unit is also used to delete the second annotation file from the second annotation file set to obtain a third annotation file set, and delete the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box.

[0022] In a third aspect, the present application provides a data processing device, comprising:

[0023] one or more processors;

[0024] The storage device is used to store one or more programs. When the one or more programs are executed by one or more processors, the electronic device implements the data processing method described in the first aspect.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium having computer-readable instructions stored thereon. When the computer-readable instructions are executed by a processor of a computer, the computer executes the data processing method described in the first aspect.

[0026] Beneficial effects of the present application: The terminal in the present application can obtain a first point cloud file set and a first annotation file set, and the first point cloud file set and the first annotation file set are at least partially associated; the first point cloud file is deleted from the first point cloud file set to obtain a second point cloud file set, and the first annotation file set does not include the annotation file associated with the first point cloud file; the first annotation file is deleted from the first annotation file set to obtain a second annotation file set, and the first point cloud file set does not include the point cloud file associated with the first annotation file; the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set, and the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than the threshold corresponding to the first three-dimensional annotation box. Through this method, the efficiency of data processing can be improved.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0029] Figure 1 is a schematic diagram of a data processing framework shown in an exemplary embodiment of the present application;

[0030] Figure 2 is a flow chart of a data processing method shown in an exemplary embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a point cloud file shown in an exemplary embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a marking file shown in an exemplary embodiment of the present application;

[0033] Figure 5 is a flowchart of an exemplary embodiment of the present application showing whether the annotation quality of a point cloud file and an annotation file is qualified;

[0034] Figure 6 is an algorithm flow chart showing an exemplary embodiment of the present application for determining whether a marked point is within a first three-dimensional marking frame;

[0035] Figure 7 is a block diagram of a data processing device shown in an exemplary embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0038] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0039] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0040] First of all, it should be noted that point cloud data refers to a set of vectors in a three-dimensional coordinate system. In addition to geometric positions, some point cloud data also have color information. Color information is usually obtained by a camera to obtain a color image, and then the color information (RGB) of the pixel at the corresponding position is assigned to the corresponding point in the point cloud. The acquisition of intensity information is the echo intensity collected by the laser scanner receiving device. This intensity information is related to the surface material, roughness, incident angle direction of the target, and the emission energy of the instrument and the laser wavelength. In the embodiment of the present application, the device can obtain a point cloud file, which can be a frame of image presented in the form of a point cloud. The point cloud file can include point cloud data. The format of the point cloud file (i.e., the suffix name) can be OFF (Object File Format), PLY (Polygon File Format), PCD (Point Cloud Data), static polygon model (OBJ), etc. The point cloud file can be in any format, which is not limited by the embodiment of the present application.

[0041] Label data is for point cloud data, and one point cloud data can be associated with one label data. Label data can be used to mark the information corresponding to a point cloud data, for example, the size, position, direction, etc. of a point cloud data can be marked. In the embodiment of the present application, one point cloud file can be associated with one label file.

[0042] The heading angle (yaw) usually refers to the angle between the vehicle's center of mass velocity and the horizontal axis in the ground coordinate system.

[0043] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a data processing framework shown in an exemplary embodiment of the present application. Figure 1As shown, the framework may include a database, a first filtering module, a second filtering module and a random sampling module. Among them, the first point cloud file set may be data collected by a laser device and stored in a database, the first annotation file is associated with the first point cloud file, and the first point cloud file set and the first annotation file set are at least partially associated. The first filtering module may obtain the first point cloud file set and the first annotation file set from the database, perform a first filtering on both, obtain a second point cloud file set and a second annotation file set, and then input them into the second filtering module to obtain a third point cloud file set and a third annotation file set. Finally, the random sampling module extracts some files from the third point cloud file set and the third annotation file set for visual manual inspection. It should be noted that the first filtering module, the second filtering module and the random sampling module in the framework may belong to the same terminal, and the database may also belong to the terminal, or may also be a database outside the terminal, which is not limited in the embodiments of the present application. The terminal may be a PC (Personal Computer, PC).

[0044] In order to realize intelligent control of independent and combined vehicle lights, the embodiments of the present application respectively propose a data processing method, a data processing device, an electronic device, a computer-readable storage medium and a computer program product, which will be described in detail below.

[0045] See also Figure 2 , Figure 2 is a flowchart of a data processing method shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0046] like Figure 2 As shown, in an exemplary embodiment, the data processing method at least includes steps S210 to S240, which are described in detail as follows:

[0047] Step S210: acquiring a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated.

[0048] Specifically, the terminal may obtain the first point cloud file set and the first annotation file set from a database, wherein the first point cloud file set is associated with at least some of the point cloud files and annotation files in the first annotation file set.

[0049] It should be noted that, ideally, the number of point cloud files in a point cloud file set should be the same as the number of annotation files in a annotation file combination. Each point cloud file has an associated annotation file to annotate the point cloud data in the point cloud file. However, due to data duplication and migration, some annotation files and point cloud files may lose or become confused, or even some annotation files and point cloud files may be lost. Therefore, it is necessary to filter the changed point cloud file set and annotation file set. For example, ideally, the first point cloud file set includes 100 point cloud files, the first annotation file set includes 100 annotation files, and any point cloud file in the first point cloud file set can find an associated annotation file in the first annotation file set. However, due to data duplication and migration, two files are lost in the first annotation file set, so it is necessary to delete the two point cloud files associated with the two lost annotation files from the first point cloud file. Correspondingly, when two files are lost in the first point cloud file, the same operation needs to be performed on the first annotation file set, which will not be repeated here.

[0050] Step S220: deleting the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file.

[0051] Most point cloud annotation systems annotate point cloud files that meet this rule: for a point cloud file, the associated annotation files have the same name except for the suffix. Therefore, the algorithm can associate the annotation file name with the original point cloud file name without the suffix, and retain the annotation files and original point cloud files that can be associated, and delete those that cannot be associated.

[0052] Therefore, the terminal can first determine the identifier of the first point cloud file from the first point cloud file set, and the identifier can be the name of the first point cloud file. Then determine from the first annotation file set whether an annotation file with the name of the first point cloud file is included. For example, if the name of the first point cloud file is "123.off", then the terminal can search from the first annotation file set whether there is an annotation file named "123". If so, it can be determined that the two match, and then both can be retained in their respective sets; if not, the first point cloud file needs to be deleted from the first point cloud file set. When the terminal performs this operation on all point cloud files in the first point cloud file set, the second point cloud file set can be obtained.

[0053] Optionally, if the terminal detects that the number of point cloud files in the first point cloud file set is inconsistent with the number of annotated files in the first annotated file set, it can be determined that there must be files in the two sets that cannot be associated with each other, and therefore data filtering is required.

[0054] Optionally, even if the terminal detects that the number of point cloud files in the first point cloud file set is consistent with the number of annotated files in the first annotated file set, data filtering is still required because there may be files whose names cannot be associated with each other. For example, the number of files in the first point cloud file set and the first annotated file set is 100, but the point cloud file named "123.off" in the first point cloud file set cannot find the annotated file named "123" in the first annotated file set.

[0055] Step S230: deleting the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include the point cloud file associated with the first annotation file.

[0056] Specifically, the terminal can first determine the identifier of the first annotation file from the first annotation file set, and the identifier can be the name of the first annotation file. Then determine from the first point cloud file set whether a point cloud file with the name of the first annotation file is included. If so, it can be determined that the two match, and then the two can be retained in their respective sets; if not, the first annotation file needs to be deleted from the first point cloud file set. After the terminal performs this operation on all annotation files in the first annotation file set, the second annotation file set can be obtained.

[0057] The second point cloud file set and the second annotation file set have the same number of files, and each file in one file set can find a related file in the other file set.

[0058] Step S240, delete the second annotation file from the second annotation file set to obtain a third annotation file set, and delete the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than the threshold corresponding to the first three-dimensional annotation box.

[0059] Specifically, the terminal can determine whether the first annotation point is located in the first three-dimensional annotation box in the second annotation file set, where the first annotation point is any annotation point in the second annotation file, and the first three-dimensional annotation box is any three-dimensional annotation box in the second annotation file. If so, the number of points included in the first three-dimensional annotation box is added by one; after the terminal traverses all points in the second annotation file, if it is finally determined that the number of points in the first three-dimensional annotation box is less than the threshold corresponding to the first three-dimensional annotation box, the second annotation file is deleted from the second annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set. When the terminal performs the above operations on all annotation files in the second annotation file set, a third annotation file set and a third point cloud file set associated with the third annotation file set can be obtained.

[0060] Optionally, if the number of points in the first three-dimensional annotation box is greater than or equal to a threshold, the terminal can mark the second annotation file and the associated point cloud file as normal data; if the number of points in the first three-dimensional annotation box is less than the threshold, the second annotation file and the corresponding point cloud file can be marked as abnormal data.

[0061] Among them, a labeling file may include multiple three-dimensional labeling frames, each of which may correspond to an object in the point cloud file associated with the labeling file, and the shape of the three-dimensional labeling frame may be a cuboid. The terminal may count the number of points mi (i=1, 2, 3, 4...) included in the i-th three-dimensional labeling frame in a labeling file, and compare mi with the threshold value corresponding to the i-th three-dimensional labeling frame provided by the labeling document. When mi is greater than the corresponding threshold value, it can be determined that the labeling quality of the i-th three-dimensional labeling frame is qualified. When the number of points in all three-dimensional labeling frames in the labeling file is greater than the corresponding threshold value, it can be determined that the quality of the labeling file and the associated first point cloud file is qualified, and then the labeling file can be retained in the second labeling file set; if the number of points in the three-dimensional labeling frame in the labeling file is less than the threshold value corresponding to the three-dimensional labeling frame, the labeling file needs to be deleted from the second labeling file set. Finally, after the above operations, the third labeling file set and the third point cloud file set can be obtained.

[0062] It should be noted that the annotation document may specify different thresholds corresponding to different 3D annotation boxes, or may specify them to be the same, which is not limited in the embodiment of the present application. Among them, the threshold may be the minimum point value for determining that the annotation quality of the 3D annotation box is qualified.

[0063] For example, Figure 3 The figure shows a schematic diagram of a point cloud file provided by an embodiment of the present application. The point cloud file includes two identified target objects, namely target object 1 and target object 2, and point cloud data corresponding to the two target objects are obtained respectively. Figure 3 It can be seen that the number of points corresponding to target object 1 is more than that of target object 2, and there are points that do not belong to target object 1 or target object 2, namely point 5 and the three points around it. Figure 4 FIG. 1 is a schematic diagram of a labeling file provided in an embodiment of the present application, wherein the labeling file is associated with the point cloud file. Figure 4 As shown, the annotation file includes a three-dimensional annotation frame 1 and a three-dimensional annotation frame 2, wherein the three-dimensional annotation frame 1 corresponds to Figure 3 The target object 1 shown in FIG. 1 is a three-dimensional annotation frame 2 corresponding to the target object 1 shown in FIG. Figure 3 The terminal can perform statistics on the corresponding point cloud data according to the three-dimensional annotation frame of the annotation file.

[0064] Figure 4 , the number of points of the area in the 3D annotation box 1 corresponding to the first point cloud file is 47, while the number of points of the area in the 3D annotation box 2 corresponding to the first point cloud file is 24. If the annotation document specifies that the thresholds corresponding to the 3D annotation box 1 and the 3D annotation box 2 are both 20, then after comparison, it can be determined that the annotation quality of the 3D annotation box 1 and the 3D annotation box 2 are both qualified, and then it can be determined that the annotation quality of the annotation file and the first point cloud file is qualified; if the annotation document specifies that the thresholds corresponding to the 3D annotation box 1 and the 3D annotation box 2 are both 30, then it can be determined that the annotation quality of the 3D annotation box 1 is qualified, and the annotation quality of the 3D annotation box 2 is unqualified, and it is further determined that the annotation quality of the annotation file and the first point cloud file is unqualified.

[0065] In one possible implementation, the terminal can determine the plane equation corresponding to each of the six faces of the first three-dimensional annotation box based on the second annotation file, and then determine whether the first annotation point is located within the first three-dimensional annotation box based on the plane equation corresponding to each face and the coordinate value of the first annotation point.

[0066] Optionally, the terminal determines the plane equation corresponding to each of the six faces of the first three-dimensional annotation frame according to the second annotation file, which may include the following method: according to the center point (x, y, z) of the first three-dimensional annotation frame included in the second annotation file, the length, width and height (l, w, h) of the first three-dimensional annotation frame, and the heading angle of the first three-dimensional annotation frame, such as Figure 4As shown; then, according to the center point of the first three-dimensional annotation frame, the length, width and height of the first three-dimensional annotation frame, and the heading angle of the first three-dimensional annotation frame, the coordinates of the eight vertices (A, B, C, D, A1, B1, C1, D1) of the three-dimensional annotation frame and the normal vectors (n1, n2, n3, n4, n5, n6) of each of the six faces (a1, a2, a3, a4, a5, a6) of the first three-dimensional annotation frame are calculated; according to the coordinates of the eight vertices of the first three-dimensional annotation frame and the normal vectors of each of the six faces of the first three-dimensional annotation frame, the plane equation corresponding to each face is determined. Among them, the plane equation can be in the form of ax+by+cz+n=0, and a, b, c, and n are all real numbers.

[0067] Optionally, the terminal determines whether the first annotation point is located within the first three-dimensional annotation box based on the plane equation corresponding to each face of the first three-dimensional annotation box and the coordinate value of the first annotation point, which may include the following method: substituting the coordinate value of the first annotation point into the function of the plane equation corresponding to each face of the first three-dimensional annotation box, and if the results are all less than 0, it means that the first annotation point is within the first three-dimensional annotation box.

[0068] For example, Figure 4As shown, assuming that the coordinates of point 1 are (x1, y1, z1), substitute the function f1(x, y, z) = a1x + b1y + c1z + n1 corresponding to the plane equation a1x + b1y + c1z + n1 = 0 of plane a1, and we get f1(χ1, y1, z1) = a1χ1 + b1y1 + c1z1 + n1. If f1(χ1, y1, z1) is less than 0, it means that point 1 is in the area in the direction of the negative normal vector (-n1) corresponding to plane a1; if f1(χ1, y1, z1) is equal to 0, it means that point 1 is on plane a1; if f1(χ1, y1, z1) is greater than 0, it means that point 1 is in the area in the direction of the normal vector (n1) corresponding to plane a1. Furthermore, the coordinate value of point 1 is substituted into the function f2(χ,y,z)=a2χ+b2y+c2z+n2 corresponding to the plane equation of plane a2, and f2(χ1,y1,z1)=a2χ1+b2y1+c2z1+n2 is obtained. If f2(χ1,y1,z1) is less than 0, it means that point 1 is in the area in the direction of the negative normal vector (-n2) corresponding to plane a2; if f2(χ1,y1,z1) is equal to 0, it means that point 1 is on plane a2; if f2(χ1,y1,z1) is greater than 0, it means that point 1 is in the area in the direction of the normal vector (n2) corresponding to plane a2. Similarly, when it is determined that point 1 is located in the area corresponding to the negative normal vector of each of the six faces of the first three-dimensional annotation frame, it can be determined that point 1 is located inside the first three-dimensional annotation frame. Correspondingly, the same method can be used to determine whether all points in the annotation file are located in the three-dimensional annotation frame and in which three-dimensional annotation frame they are located. Exemplarily, the terminal may determine that point 2 is located in 3D annotation frame 1 , point 3 and point 4 are located in 3D annotation frame 2 , and point 5 is located neither in 3D annotation frame 1 nor in 3D annotation frame 2 .

[0069] It should be noted that, in the embodiment of the present application, after the first annotation file set and the first point cloud file set are processed as in step S220 and step S230, the two can be recorded as the second annotation file set and the second point cloud file set respectively. If all point cloud files in the first point cloud file set can find associated annotation files in the first annotation file set, and all annotation files in the first annotation file set can find associated point cloud files in the first point cloud file set, then it means that the first annotation file set and the second annotation file set are the same, and the first point cloud file set and the second point cloud file set are the same. Similarly, after the second annotation file set and the second point cloud file set are processed in step S240, if no files to be deleted are found, the third annotation file set can be the same as the second annotation file set, and the third point cloud file set can also be the same as the second point cloud file set. That is to say, ideally, the first point cloud file set, the second point cloud file set and the third point cloud file set are the same, and the first annotation file set, the second annotation file set and the third annotation file set are the same.

[0070] In a possible implementation, after determining the third annotation file set and the third point cloud file set associated with the third annotation file set, the terminal can extract a preset number of point cloud files from the third point cloud file set, and extract a preset number of annotation files associated with the preset number of point cloud files from the third annotation file set, and the preset number of point cloud files and the preset number of annotation files are used for visual manual quality inspection. The preset number can be 5% of all point cloud files in the third point cloud file set, which is not limited in the embodiment of the present application.

[0071] In a possible implementation, if it is determined through visual manual quality inspection that the corresponding target object can be annotated according to the size and orientation of the second three-dimensional annotation box, the terminal can determine that the quality of the annotation data in the third point cloud file set and the third annotation file set is qualified. The second three-dimensional annotation box is included in the third annotation file, and the third annotation file is any one of the third number of annotation files, that is, randomly selected.

[0072] Through the embodiment of the present application, the terminal can obtain a first point cloud file set and a first annotation file set, which are at least partially associated with the first annotation file set; delete the first point cloud file from the first point cloud file set to obtain a second point cloud file set, and the first annotation file set does not include the annotation file associated with the first point cloud file; delete the first annotation file from the first annotation file set to obtain a second annotation file set, and the first point cloud file set does not include the point cloud file associated with the first annotation file; delete the second annotation file from the second annotation file set to obtain a third annotation file set, and delete the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, and the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than the threshold value corresponding to the first three-dimensional annotation box. Through this method, the efficiency of data processing can be improved.

[0073] See also Figure 5 , Figure 5 is a flowchart of an exemplary embodiment of the present application showing whether the point cloud file and the annotation file have qualified annotation quality. The method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0074] in, Figure 5 The k-th point cloud file (pointcloud_2[k]) in the second point cloud file set (pointcloud_2) is associated with the k-th label file (label_2[k]) in the second label file set (label_2).

[0075] like Figure 5 As shown, for the detailed implementation process, please refer to the records in the aforementioned embodiments, which will not be described in detail here.

[0076] Through the embodiments of the present application, it is possible to quickly perform quality inspection on the files in the second point cloud file set and the second annotation file set.

[0077] See also Figure 6 , Figure 6 is an algorithm flow chart of an exemplary embodiment of the present application for determining whether a marked point is within the first three-dimensional marked frame. The method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0078] like Figure 6 As shown, for the detailed implementation process, please refer to the records in the aforementioned embodiments, which will not be described in detail here.

[0079] Preferably, if Figure 6 If the function value of a plane corresponding to any point (xi, yi, zi) in the equation is greater than 0, then there is no need to calculate the function value of the point in the function corresponding to the remaining planes. i ,y i ,z i ) is greater than 0, then f2(x i ,y i ,z i ) to f6(χ i ,y i ,z i ), which can save processor computing resources.

[0080] Through the embodiments of the present application, it can be determined whether the annotation point is within the first three-dimensional annotation frame.

[0081] See also Figure 7 , Figure 7 is a block diagram of a data processing device shown in an exemplary embodiment of the present application. The device can be applied to Figure 2 , Figure 5 or Figure 6 The device may also be applicable to other exemplary implementation environments and specifically configured in other devices, and this embodiment does not limit the implementation environment to which the device is applicable.

[0082] like Figure 7 As shown, the exemplary data processing device includes:

[0083] An acquiring unit 710 is configured to acquire a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated;

[0084] The processing unit 720 is configured to delete the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file;

[0085] The processing unit 720 is further configured to delete the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file;

[0086] The above-mentioned processing unit 720 is also used to delete the second annotation file from the second annotation file set to obtain a third annotation file set, and delete the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box.

[0087] In one embodiment of the present application, the annotation file associated with the first point cloud file includes an identifier of the first point cloud file, and the point cloud file associated with the first annotation file includes an identifier of the first annotation file.

[0088] In one embodiment of the present application, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain the third point cloud file set. The processing unit 720 is also used to determine whether the number of points inside the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box, and the first three-dimensional annotation box is any three-dimensional annotation box in the second annotation file; if so, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set.

[0089] In one embodiment of the present application, the shape of the first three-dimensional annotation frame is a cuboid.

[0090] In one embodiment of the present application, the above-mentioned processing unit 720 is also used to determine the plane equation corresponding to each of the six faces of the first three-dimensional annotation box according to the second annotation file; determine whether the first annotation point is located in the first three-dimensional annotation box according to the plane equation corresponding to each face and the coordinate value of the first annotation point, and the first annotation point is any annotation point in the second annotation file.

[0091] In one embodiment of the present application, the above-mentioned processing unit 720 is also used to calculate the eight vertex coordinates of the three-dimensional annotation box and the normal vector of each of the six faces of the first three-dimensional annotation box based on the center point of the first three-dimensional annotation box included in the second annotation file, the length, width and height of the first three-dimensional annotation box, and the heading angle of the first three-dimensional annotation box; calculate the plane equation corresponding to each face based on the eight vertices of the first three-dimensional annotation box and the normal vector of each of the six faces.

[0092] In one embodiment of the present application, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain the third point cloud file set. The above-mentioned processing unit 720 is also used to extract a preset number of point cloud files from the third point cloud file set, and extract a preset number of annotation files respectively associated with a preset number of point cloud files from the third annotation file set. The preset number of point cloud files and the preset number of annotation files are used for visual manual quality inspection.

[0093] In one embodiment of the present application, the above-mentioned processing unit 720 is also used to determine that the quality of the annotation data in the third point cloud file set and the third annotation file set is qualified if the corresponding target object is annotated by the size and orientation of the second three-dimensional annotation box, and the second three-dimensional annotation box is included in the third annotation file, and the third annotation file is any one of a preset number of annotation files.

[0094] It should be noted that the data processing device provided in the above embodiment and the data processing method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the data processing device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0095] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the data processing method provided in the above-mentioned embodiments.

[0096] See also Figure 8 , Figure 8 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0097] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803, such as executing the method described in the above embodiment. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, ROM 802 and RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0098] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0099] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 809, and / or installed from a removable medium 811. When the computer program is executed by a central processing unit (CPU) 801, various functions defined in the system of the present application are executed.

[0100] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0101] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0102] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0103] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the data processing method as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0104] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method provided in each of the above embodiments.

[0105] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated; Deleting the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file; Deleting the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file; The second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box.

2. The method according to claim 1, characterized in that The annotation file associated with the first point cloud file includes an identifier of the first point cloud file, and the point cloud file associated with the first annotation file includes an identifier of the first annotation file.

3. The method according to claim 1, characterized in that Before deleting the second annotation file from the second annotation file set to obtain the third annotation file set, and deleting the point cloud file associated with the second annotation file from the second point cloud file set to obtain the third point cloud file set, the method further includes: Determining whether the number of points inside the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box, where the first three-dimensional annotation box is any three-dimensional annotation box in the second annotation file; If so, the second annotation file is deleted from the second annotation file set to obtain a third annotation file set, and the point cloud file associated with the second annotation file is deleted from the second point cloud file set to obtain a third point cloud file set.

4. The method according to claim 3, characterized in that The shape of the first three-dimensional annotation frame is a cuboid; The determining whether the number of points inside the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box includes: Determine a plane equation corresponding to each of the six faces of the first three-dimensional annotation frame according to the second annotation file; Determine whether the first annotation point is located in the first three-dimensional annotation frame according to the plane equation corresponding to each face and the coordinate value of the first annotation point, wherein the first annotation point is any annotation point in the second annotation file.

5. The method according to claim 4, characterized in that The determining, according to the second annotation file, a plane equation corresponding to each of the six faces of the three-dimensional annotation frame includes: Calculate eight vertex coordinates of the three-dimensional annotation box and a normal vector of each of six faces of the first three-dimensional annotation box according to the center point of the first three-dimensional annotation box, the length, width and height of the first three-dimensional annotation box, and the heading angle of the first three-dimensional annotation box included in the second annotation file; The plane equation corresponding to each face is calculated according to the eight vertices of the first three-dimensional annotation box and the normal vector of each face of the six faces.

6. The method according to claim 1, characterized in that After deleting the second annotation file from the second annotation file set to obtain a third annotation file set, and deleting the point cloud file associated with the second annotation file from the second point cloud file set to obtain the third point cloud file set, the method further includes: A preset number of point cloud files are extracted from the third point cloud file set, and a preset number of annotation files respectively associated with the preset number of point cloud files are extracted from the third annotation file set, and the preset number of point cloud files and the preset number of annotation files are used for visual manual quality inspection.

7. The method according to claim 6, characterized in that The method further comprises: If the corresponding target object is annotated by the size and orientation of the second three-dimensional annotation box, it is determined that the quality of the annotation data in the third point cloud file set and the third annotation file set is qualified, the second three-dimensional annotation box is included in the third annotation file, and the third annotation file is any one of the preset number of annotation files.

8. A data processing device, characterized in that: include: An acquiring unit, configured to acquire a first point cloud file set and a first annotation file set, wherein the first point cloud file set and the first annotation file set are at least partially associated; a processing unit, configured to delete the first point cloud file from the first point cloud file set to obtain a second point cloud file set, wherein the first annotation file set does not include an annotation file associated with the first point cloud file; The processing unit is further configured to delete the first annotation file from the first annotation file set to obtain a second annotation file set, wherein the first point cloud file set does not include a point cloud file associated with the first annotation file; The processing unit is further used to delete the second annotation file from the second annotation file set to obtain a third annotation file set, and delete the point cloud file associated with the second annotation file from the second point cloud file set to obtain a third point cloud file set, wherein the second annotation file includes a first three-dimensional annotation box, and the number of points in the first three-dimensional annotation box is less than a threshold corresponding to the first three-dimensional annotation box.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the data processing method according to any one of claims 1 to 7.