Point cloud data labeling method and device, electronic equipment, chip and medium

Abstract

The invention provides a point cloud data labeling method and device, electronic equipment, a chip and a medium, and relates to the field of data labeling. The point cloud data labeling method comprises the following steps: receiving first part labeling data and second part labeling data of a first point cloud, wherein the first part labeling data and the second part labeling data have no intersection; based on first annotation data formed by the first part of annotation data and the second part of annotation data, obtaining different annotation data according to different point cloud tags in the first annotation data; taking different annotation data as a second annotation data sequence; and obtaining a second point cloud through data integration according to the first point cloud and the second annotation data sequence. Through the technical scheme provided by the invention, the problems that point cloud data labeling cannot be coordinated and the efficiency is low are solved, the point cloud labeling efficiency is improved, and the control on the labeling quality is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data annotation, and in particular to a point cloud data annotation method, device, electronic device, chip and medium. Background Art

[0002] In the field of artificial intelligence, the quality of sample data annotation plays a particularly important role in the training results of the model. In the annotation of three-dimensional data, especially the annotation of common point cloud data (pcd) files, pcd files are usually loaded locally and then edited. This annotation method has problems with data management and maintenance when the amount of sample data is large and many annotation workers are required, making it difficult to collaboratively and efficiently complete the annotation of point cloud data. Summary of the invention

[0003] The present invention provides a point cloud annotation method, device, electronic device, chip and medium to solve the problems of low point cloud annotation efficiency and inability to collaborate in annotation in related technologies, improve the point cloud annotation efficiency, and enhance the control of annotation quality.

[0004] A first aspect of the present disclosure provides a point cloud data annotation method, which is applied to a cloud server. The method includes:

[0005] Receive a first portion of annotation data and a second portion of annotation data of a first point cloud, wherein the first portion of annotation data and the second portion of annotation data do not have any intersection;

[0006] Based on the first annotation data composed of the first part of the annotation data and the second part of the annotation data, different annotation data are obtained according to different point cloud labels in the first annotation data;

[0007] Using different labeled data as a second labeled data sequence;

[0008] A second point cloud is obtained by data integration according to the first point cloud and the second labeled data sequence.

[0009] In an embodiment of the present disclosure, the first annotation data includes a point cloud label and a first point cloud attribute, and the first point cloud attribute includes at least one of a point cloud color and a point cloud sequence number.

[0010] In one embodiment of the present disclosure, the method further includes:

[0011] The second point cloud and the third point cloud are compared to determine the labeling accuracy of the second point cloud, wherein each point of the third point cloud has a correct label.

[0012] In one embodiment of the present disclosure, the cloud server includes at least one of a backend and a cloud.

[0013] A second aspect of the present disclosure provides a point cloud annotation method, which is applied to a front end. The method includes:

[0014] Obtaining a first part or a second part of a first point cloud and rendering it;

[0015] In response to a triggering annotation instruction, determining first portion of annotation data of the first portion, or second portion of annotation data of the second portion;

[0016] The first part of the labeled data or the second part of the labeled data is sent, and there is no intersection between the first part of the labeled data and the second part of the labeled data.

[0017] In one embodiment of the present disclosure, the method further includes:

[0018] Rendering the first part of the labeled data or the second part of the labeled data;

[0019] In response to triggering the marking instruction, modifying the first portion of the marking data or the second portion of the marking data;

[0020] The modified first part of the annotation data or the modified second part of the annotation data is sent, and there is no intersection between the modified first part of the annotation data and the modified second part of the annotation data.

[0021] In one embodiment of the present disclosure, the front end includes at least one of a browser end and a client end.

[0022] The third aspect of the present disclosure provides a point cloud data annotation device, which is applied to a cloud server. The device includes:

[0023] A receiving module, configured to receive a first portion of annotated data and a second portion of annotated data of a first point cloud, wherein the first portion of annotated data and the second portion of annotated data do not have any intersection;

[0024] An analysis module, configured to obtain different annotation data based on first annotation data composed of the first part of annotation data and the second part of annotation data according to different point cloud labels in the first annotation data;

[0025] A determination module is configured to use different labeled data as a second labeled data sequence;

[0026] The generating module is used to obtain the second point cloud by integrating the first point cloud and the second annotation data sequence.

[0027] A fourth aspect of the present disclosure provides a point cloud data annotation device, which is applied to a front end and includes:

[0028] A rendering module, used for obtaining and rendering the first part or the second part of the first point cloud;

[0029] a marking module, configured to determine, in response to a triggering marking instruction, a first portion of marking data of the first portion, or a second portion of marking data of the second portion;

[0030] The sending module is used to send the first part of the labeled data or the second part of the labeled data.

[0031] The fifth aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the methods in the first aspect and / or second aspect embodiments of the present disclosure.

[0032] The sixth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute any method in the first aspect and / or second aspect embodiments of the present disclosure.

[0033] The seventh aspect embodiment of the present disclosure proposes a computer program product, characterized in that it includes a computer program, and when the computer program is executed by a processor, it implements the method of any one of the embodiments of the first aspect and / or the second aspect of the present disclosure.

[0034] The eighth aspect embodiment of the present disclosure proposes a chip, characterized in that it includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals include computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes any one of the methods in the first aspect and / or the second aspect embodiments.

[0035] In summary, according to the point cloud annotation method proposed in the present disclosure, the first part of the annotation data and the second part of the annotation data of the first point cloud are received, and there is no intersection between the first part of the annotation data and the second part of the annotation data; based on the first annotation data composed of the first part of the annotation data and the second part of the annotation data, different annotation data are obtained according to different point cloud labels in the first annotation data; different annotation data are used as the second annotation data sequence; according to the first point cloud and the second annotation data sequence, the second point cloud is obtained through data integration. Through the technical solution of the present disclosure, the first part of the annotation data and the second part of the annotation data of the first point cloud are received, and there is no intersection between the first part of the annotation data and the second part of the annotation data. The collaborative annotation of the point cloud sample data annotators is guaranteed, and the problem of multiple people annotating the same point at the same time is avoided. Based on the first annotation data composed of the first part of the annotation data and the second part of the annotation data, different annotation data are obtained according to different point cloud labels in the first annotation data; different annotation data are used as the second annotation data sequence; the annotation data is uniformly managed based on the label category, thereby improving the annotation efficiency. According to the first point cloud and the second annotation data sequence, the second point cloud is obtained through data integration; the labels of the annotation data and the point cloud data are uniformly sorted, thereby improving the point cloud annotation efficiency.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0038] Figure 1 It is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure;

[0039] Figure 2 A flowchart of determining the number of points and annotation accuracy of a modified point cloud according to an embodiment of the present disclosure;

[0040] Figure 3 It is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure;

[0041] Figure 4 It is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure;

[0042] Figure 5 A schematic diagram of communication between a point cloud data annotation cloud server and a front end according to an embodiment of the present disclosure;

[0043] Figure 6 It is a structural schematic diagram of a point cloud data annotation device according to an embodiment of the present disclosure;

[0044] Figure 7 It is a block diagram of an electronic device for implementing point cloud data annotation of the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar originals or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0046] With the continuous development of deep learning technology, it has brought far-reaching impacts to the field of machine vision. For tasks such as classification, detection, segmentation, and tracking of targets in traditional images, methods based on deep neural networks have achieved very significant breakthroughs. Using massive sample data to train neural networks to obtain neural network models has become the main operating mode in the field of machine vision. Among them, the quality of sample data, especially in machine learning based on supervised learning, has a significant impact on the performance of the final training model. Sample data consists of data and labels. For example, in image classification, each picture is used as data, and the category corresponding to the picture is used as a label. The classification data thus constructed can be used to train the neural network of the classification function to obtain a classification neural network model to achieve image classification.

[0047] As the perception of objects has evolved from images to three-dimensional data, more abundant information can be obtained through the collection of three-dimensional data. In the field of autonomous driving, in addition to image sensors, cars are also equipped with sensors such as lidar, three-dimensional scanners, and depth sensors to obtain three-dimensional data. Usually, the three-dimensional data collected by these sensors is point cloud data. Point cloud data includes the three-dimensional spatial coordinates x, y, and z information of a large number of points, as well as the point's serial number, color, and other attribute information. After collecting point cloud data, deep learning is usually used to combine point cloud data to identify or segment three-dimensional objects in the recognition or segmentation of the external environment. Among them, point cloud sample data is particularly important for training three-dimensional target recognition or style networks.

[0048] In the related art, the annotation of point cloud sample data is completed by loading the point cloud pcd file locally and modifying the point label. This modification actually occurs in the current point cloud file, so this operation method is not suitable for large-scale manual operations, and data management and maintenance are also inconvenient. For example, when a pcd file is modified multiple times, or multiple people modify a pcd at the same time, it will affect the accuracy of the point cloud label. In addition, it is impossible to complete statistical indicators on the quality of point cloud annotation, such as the point accuracy rate of the point cloud.

[0049] The method proposed in the present disclosure is applied to point cloud labeling tasks, and can be specifically used for data labeling of three-dimensional data, especially for sample data labeling in scenarios such as three-dimensional target classification, detection, segmentation, and tracking in the field of deep learning and machine vision. The application scenarios are not limited in the embodiments of the present disclosure.

[0050] The point cloud data annotation method provided in this application is described in detail below with reference to the accompanying drawings.

[0051] Figure 1 Flow chart of a point cloud data annotation method according to an embodiment of the present disclosure. Figure 1 In the embodiment shown, the point cloud data annotation method includes the following steps:

[0052] Step 101 : receiving a first portion of annotation data and a second portion of annotation data of a first point cloud, wherein the first portion of annotation data and the second portion of annotation data do not have any intersection.

[0053] In the present disclosure, the cloud server includes a cloud and / or backend server, and the server includes but is not limited to a computer, a network device, and a storage device. The cloud server is used to process the collaborative execution of point cloud labeling tasks. The first point cloud is the overall point cloud data in the point cloud labeling operation, the first part of the labeling data is the data labeled based on a part of the point cloud of the first point cloud, and the second part of the point cloud data is the data labeled based on another part of the point cloud of the first point cloud. The first part of the labeling data and the second part of the labeling data are used to indicate that the first point cloud has labeling data of different regions, and the number of point cloud labeling is not limited. The labeling data can be stored in files of various formats, such as txt, xml, csv, json and other formats, preferably stored in JavaScript Object Notation (json) files. For the first part of the labeling data and the second part of the labeling data, they are respectively used as marks made for different regions of the first point cloud, and there is no identical point cloud data in the different regions, that is, there are no multiple labelings of the same point between the first part of the labeling data and the second part of the labeling data.

[0054] Step 102 , based on the first annotation data formed by the first part of the annotation data and the second part of the annotation data, different annotation data are obtained according to different point cloud labels in the first annotation data.

[0055] In the present disclosure, the first annotation data is composed of a first part of annotation data and a second part of annotation data, and is used to represent the annotation data of the first point cloud.

[0056] After analyzing the first labeled data, a second labeled data sequence can be obtained. The second labeled data sequence here is a sequence of multiple labeled data obtained by merging and arranging the labeled data of the same specific indicator according to the first labeled data according to a specific indicator. The specific indicator includes any indicator of a label, a point cloud color, and a point cloud sequence number.

[0057] In one embodiment of the present disclosure, a large number of point cloud tags are stored in the first annotation data, and different annotation data are created according to different point cloud tags. For example, in the annotation of the human body three-dimensional point cloud, operator A's partial data of the human body three-dimensional point cloud annotation data part1.json is shown in the following table:

[0058] The point ID Point label label 110031 head 9948 leg 120954 head 8212 body 9682 leg

[0059] Part of the data of the human body 3D point cloud annotation data part2.json by operator B is shown in the following table:

[0060] The point ID Point label label 8448 leg 8948 leg 127451 head 7972 body 121090 head

[0061] According to different point cloud labels, here the point cloud labels are head, body, and leg, different annotation data are created, that is, according to the above point cloud labels, the annotation data is classified and saved. For example, the saved annotation data is label_head.json, label_body.json, and label_leg.json.

[0062] Among them, the data saved in the classified label data file label_head.json is:

[0063] label=head,id={110031,120954,127451,121090}.

[0064] The data saved in the labeled data file label_body.json after classification is:

[0065] label=body,id={8212,7972}.

[0066] The data saved in the labeled data file label_leg.json after classification is:

[0067] label=leg,id={9682,9948,8448,8948}.

[0068] Step 103: Use different labeled data as a second labeled data sequence.

[0069] In the present disclosure, after different annotation data are obtained, the different annotation data are used as a second annotation data sequence to modify the first point cloud data and generate sample data.

[0070] In this embodiment, by sorting the labels of the points in the labeled point cloud, different annotation data are created based on the point labels and used as the second annotation data sequence, thereby realizing online collaboration and automatic data sorting for the simultaneous annotation tasks of multiple annotation workers on the server side, thereby improving the efficiency of point cloud annotation.

[0071] Step 104 , obtaining a second point cloud by integrating data based on the first point cloud and the second annotated data sequence.

[0072] In the present disclosure, the second point cloud is the first point cloud data after annotation, which is obtained through the first point cloud and the second annotation data sequence.

[0073] In summary, according to the point cloud data annotation method proposed in the present disclosure, the cloud server receives the first part of the annotation data and the second part of the annotation data of the first point cloud, and there is no intersection between the first part of the annotation data and the second part of the annotation data, which ensures that the point cloud data annotation can be carried out collaboratively on multiple browsers or clients, and avoids annotation anomalies caused by the same annotation data between different browsers or clients. The first annotation data composed of the first part of the annotation data and the second part of the annotation data is analyzed to obtain a second annotation data sequence. Since the server automatically organizes the annotation data to form clearer and easier-to-use annotation data, the efficiency of organizing the point cloud annotation data is improved. According to the first point cloud and the second annotation data sequence, the second point cloud is obtained by data integration, and the point cloud sample data is efficiently obtained.

[0074] In the disclosed embodiment, Figure 1 The first annotation data in the illustrated embodiment includes a point cloud label and a first point cloud attribute, and the first point cloud attribute includes at least one of a point cloud color and a point cloud sequence number.

[0075] The point cloud label is the label corresponding to a certain spatial point in the point cloud. For example, in the three-dimensional point cloud segmentation scene after scanning the point cloud data of multiple human bodies, the point on the head is marked as head, that is, the label of the point is assigned to head, and the point on the leg is marked as leg, that is, the label of the point is assigned to leg. Point cloud attributes are other attributes of the point cloud except the x, y, and z three-dimensional coordinate points of the point, such as the color of the point, the intensity of the point, the normal vector of the point, the tangent direction of the point, the texture coordinates of the point, and other information. The first point cloud attribute is the color of the point and / or the serial number of the point in the point cloud attribute. Since the color of the point and the serial number of the point can be used to distinguish the category of the point, preferably, different point cloud color marks can be used to represent different labels, or the serial numbers of the recorded points can be uniformly classified into one label. In actual operations, different labels often correspond to different point cloud color marks. For example, the point with label = head is marked red, and the point with label = leg is marked green, which is convenient for the operator to mark.

[0076] In one implementation of this embodiment, the first annotation data is a point cloud label, a point cloud color, or the first annotation data is a point cloud label, a point cloud serial number, or the first annotation data is a point cloud label, a point cloud color and a point cloud serial number.

[0077] After the point cloud is labeled, in order to evaluate the annotation quality and efficiency of the point cloud data, the labeled point cloud data needs to be reviewed.

[0078] Figure 2 The present invention is a flowchart for determining the number of points in a modified point cloud and the accuracy of annotations according to an embodiment of the present invention. Figure 2 Based on Figure 1 The second point cloud obtained in step 103 is further processed. Figure 2 In the embodiment shown, the following steps are included:

[0079] The second point cloud and the third point cloud are compared to determine the labeling accuracy of the second point cloud, wherein each point of the third point cloud has a correct label.

[0080] In this embodiment, each point of the third point cloud has a correct label. By comparing the second point cloud and the third point cloud, the number of modified point cloud points m can be obtained, and the annotation quality of the second point cloud can be calculated. The annotation quality can be evaluated by the point accuracy, that is, the point accuracy alpha = 1-m / n, where m is the number of modified point cloud points, that is, the number of points with different labels in the second point cloud and the third point cloud, and n is the total number of points in the second point cloud. The point accuracy alpha value can reflect the annotation quality of the second point cloud after annotation. If alpha is higher than the preset threshold, it is considered that the annotation quality is qualified and can be used as sample data for training. Otherwise, re-annotation is required to ensure that the accuracy of the point cloud annotation can meet the standard.

[0081] In the embodiments of the present disclosure, the cloud server includes at least one of a backend and a cloud, wherein the cloud refers to the central node of cloud computing, the cloud data center; the backend refers to the backend part of the server, including databases, applications, etc.

[0082] The above embodiments describe the functions of the cloud server. The front end corresponding to the cloud server serves as a platform for direct interaction with point cloud data annotators, and its functions are as follows.

[0083] Figure 3 This is a flow chart of a point cloud data annotation method according to an embodiment of the present disclosure. Applied to the front end, Figure 3 Functions implemented and Figure 1 The cloud server corresponds to the cloud server and communicates data with the cloud server. Figure 3 In the embodiment shown, the following steps are included:

[0084] Step 301: obtain and render a first part or a second part of a first point cloud.

[0085] The first part or the second part of the first point cloud is loaded into the front end, and rendered and displayed locally. The first part and the second part here do not limit the composition of the first point cloud, but are used to indicate that the first point cloud is distributed to multiple front ends for annotation. In other words, the first part and the second part of the first point cloud are loaded into different front ends respectively.

[0086] Step 302 : determining, in response to a triggering marking instruction, a first portion of marking data of the first portion, or a second portion of marking data of the second portion.

[0087] The annotation instruction is triggered to annotate the first part or the second part of the first point cloud, and the first part of the annotation data or the second part of the annotation data is generated during the annotation.

[0088] In this embodiment, the point cloud data annotator annotates a part of the first point cloud allocated by the cloud server at the front end, and annotates the points in the point cloud by using the mouse, keyboard and other operations based on the visualized point cloud.

[0089] In one implementation of this embodiment, the annotator can annotate the point cloud by selecting batch annotation, deleting annotations, modifying annotations, annotating single points, annotating custom areas, etc. using a mouse and / or a mouse.

[0090] Step 303: Send the first part of the labeled data or the second part of the labeled data, where there is no intersection between the first part of the labeled data and the second part of the labeled data.

[0091] After the front end annotates the first part of the first point cloud, the first part of the annotated data is obtained. After the second part of the first point cloud is annotated, the second part of the annotated data is obtained. The first part of the annotated data or the second part of the annotated data is sent to the cloud server. There is no intersection between the first part of the annotated data and the second part of the annotated data, that is, each point in the first point cloud has only a unique label.

[0092] In this embodiment, after the first point cloud is loaded into the front end, the first point cloud is annotated, and after the annotation is completed, the annotated data is sent to the cloud server. Different parts of the first point cloud are annotated by different front ends, which ensures that multiple people will not modify the same point cloud data at the same time during the annotation process, avoiding system abnormalities. In the process of annotating the point cloud data by the front end, in order to save the marking results in real time, the modified annotation data needs to be synchronized to the cloud server.

[0093] Figure 4 The present invention is a flowchart of a method for labeling point cloud data according to an embodiment of the present invention. Figure 4 In addition to implementing Figure 3 Another technical solution besides the function of Figure 4 In the embodiment shown, the following steps are included:

[0094] Step 401, rendering the first part of the annotated data or the second part of the annotated data;

[0095] The first part or the second part of the point cloud data is loaded, and the first part of the point cloud is rendered based on the first part of the annotated data, and the second part of the point cloud is rendered based on the second part of the annotated data, and the first part and the second part are loaded in different front ends.

[0096] Step 402, in response to triggering the marking instruction, modifying the first part of the marking data or the second part of the marking data;

[0097] After the labeling command is triggered by means of keyboard and / or mouse, the label of the point cloud data can be modified, and the modification will cause the first part of the labeling data or the second part of the labeling data to be changed.

[0098] Step 403: Send the modified first part of the annotation data or the modified second part of the annotation data, where there is no intersection between the modified first part of the annotation data and the modified second part of the annotation data.

[0099] The first part of the labeled data with the label data changed or the second part of the labeled data with the label data modified is sent to the cloud server to synchronize the label data changes made in the front end. There is no intersection between the modified first part of the labeled data and the modified second part of the labeled data, that is, each point in the first point cloud has only a unique label.

[0100] In the embodiments of the present disclosure, Figure 3 and Figure 4 Each step of the embodiment is applied to the front end, which includes a browser and a client. The browser refers to the part of the website that the user accesses through the browser, including the content of the web page and the interactive effects. The client refers to a program that corresponds to the server and provides local services to the client. Except for some applications that only run locally, they are generally installed on ordinary client computers and need to cooperate with the server to run.

[0101] Figure 5 Schematic diagram of communication between a point cloud data annotation cloud server and a front end according to an embodiment of the present disclosure. Figure 6 In the embodiment shown, the cloud server has stored the pcd file to be annotated, and when the front end requests to load the point cloud data, the pcd file is sent to the front end and rendered. After the front end renders the point cloud data, the point cloud data is manually annotated, and the interface is saved after the manual annotation operation to transmit the modified json, that is, after the manual annotation operation is completed, the communication interface with the cloud server is saved, and the modified json is sent to the cloud server, and the annotation data is saved in the json. After receiving the json annotation data in this cloud server, the original json configuration file is modified and saved, and after the modification is completed, the front end is notified that the saving is completed. Using the json annotation data, the original json annotation data is sorted using the label category data contained therein, and a annotation json configuration file is generated, which is sent to the front end to render the annotated categories in the point cloud. At the same time, according to the batch point cloud sequence number based on the label in the json configuration file obtained by the label, combined with the pcd file to be annotated, the pcd file is batch processed, and the pcd result file is output, thereby generating a pcd file with annotation.

[0102] The embodiment applied to the cloud server is combined with the embodiment applied to the front end to complete the collaborative labeling of the point cloud data, thereby improving the labeling efficiency and statistically analyzing the accuracy of the labeling, thereby improving the ability to control the labeling quality.

[0103] Corresponding to the methods provided in the above-mentioned embodiments, the present disclosure also provides an image processing device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation method of the method is also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.

[0104] Figure 6 FIG. 6 is a schematic diagram of the structure of a point cloud data annotation device 600 according to an embodiment of the present disclosure. Figure 6 As shown, the device comprises:

[0105] A receiving module 610 is used to receive a first part of the annotation data and a second part of the annotation data of the first point cloud, where the first part of the annotation data and the second part of the annotation data do not have any intersection;

[0106] An analysis module 620 is configured to obtain different annotation data based on the first annotation data formed by the first part of the annotation data and the second part of the annotation data according to different point cloud labels in the first annotation data;

[0107] A determination module 630, configured to use different annotated data as second annotated data;

[0108] The integration module 640 is used to obtain a second point cloud by integrating the first point cloud and the second labeled data sequence.

[0109] In some embodiments, the first annotation data of the analysis module 620 includes a point cloud label and a first point cloud attribute, and the first point cloud attribute includes at least one of a point cloud color and a point cloud sequence number.

[0110] In some embodiments, the point cloud data annotation device 600 is further used for:

[0111] The second point cloud and the third point cloud are compared to determine the labeling accuracy of the second point cloud, wherein each point of the third point cloud has a correct label.

[0112] In some embodiments, the cloud server includes at least one of a backend and a cloud.

[0113] In some embodiments, the point cloud data annotation device 600 for the front end is also used for:

[0114] Obtaining a first part or a second part of a first point cloud and rendering it;

[0115] In response to a triggering annotation instruction, determining first portion of annotation data of the first portion, or second portion of annotation data of the second portion;

[0116] The first part of the labeled data or the second part of the labeled data is sent, and there is no intersection between the first part of the labeled data and the second part of the labeled data.

[0117] In some embodiments, the point cloud data annotation device 600 is further used for:

[0118] Rendering the first part of the labeled data or the second part of the labeled data;

[0119] In response to triggering the marking instruction, modifying the first portion of the marking data or the second portion of the marking data;

[0120] The modified first part of the annotation data or the modified second part of the annotation data is sent, and there is no intersection between the modified first part of the annotation data and the modified second part of the annotation data.

[0121] In some embodiments, the front end includes at least one of a browser side and a client side.

[0122] In summary, through the point cloud data annotation device, the cloud server receives the first part of the annotation data and the second part of the annotation data of the first point cloud, and there is no intersection between the first part of the annotation data and the second part of the annotation data, which ensures that the point cloud data annotation can be carried out collaboratively on multiple browsers or clients, and avoids the annotation anomaly caused by the same annotation data between different browsers or clients. Based on the first annotation data composed of the first part of the annotation data and the second part of the annotation data, different annotation data are obtained according to different point cloud tags in the first annotation data, and different annotation data are used as the second annotation data sequence. Since the server automatically organizes the annotation data to form clearer and easier-to-use annotation data, the efficiency of organizing the point cloud annotation data is improved. According to the first point cloud and the second annotation data sequence, the second point cloud is obtained by data integration, and the point cloud sample data is efficiently obtained. The front end obtains the first part or the second part of the first point cloud and renders it; in response to the triggering annotation instruction, the first part of the annotation data of the first part, or the second part of the annotation data of the second part, is determined; and the first part of the annotation data or the second part of the annotation data is sent. The collaborative and efficient point cloud annotation is ensured, and the efficiency of point cloud annotation is improved.

[0123] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0124] Figure 7 It is a block diagram of an electronic device 600 for implementing the above-mentioned point cloud data annotation method according to an exemplary embodiment.

[0125] For example, the electronic device 700 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0126] Reference Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0127] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0128] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0129] The power supply component 706 provides power to the various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0130] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0131] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0132] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0133] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0134] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0135] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0136] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 720 of an electronic device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0137] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image processing method described in the above embodiment of the present disclosure.

[0138] An embodiment of the present disclosure further provides a computer program product, including a computer program, which executes the image processing method described in the above embodiment of the present disclosure when a processor is used.

[0139] An embodiment of the present disclosure also proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals include computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the image processing method described in the above embodiments of the present disclosure.

[0140] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0141] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0142] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0143] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0144] It should be understood that the various parts of the embodiments of the present disclosure can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0145] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0146] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0147] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A point cloud data annotation method, characterized in that: Applied to a cloud server, the method includes: Receive a first portion of annotation data and a second portion of annotation data of a first point cloud, wherein the first portion of annotation data and the second portion of annotation data do not have any intersection; Based on first annotated data formed by the first part of the annotated data and the second part of the annotated data, different annotated data are obtained according to different point cloud labels in the first annotated data; Using the different labeled data as a second labeled data sequence; A second point cloud is obtained by data integration according to the first point cloud and the second labeled data sequence.

2. The method according to claim 1, characterized in that The first annotation data includes a point cloud label and a first point cloud attribute, and the first point cloud attribute includes at least one of a point cloud color and a point cloud sequence number.

3. The method according to claim 1, characterized in that The method further comprises: The second point cloud and the third point cloud are compared to determine the labeling accuracy of the second point cloud, wherein each point of the third point cloud has a correct label.

4. The method according to any one of claims 1 to 3, characterized in that The cloud server includes at least one of a backend and a cloud.

5. A point cloud data annotation method, characterized in that: Applied to the front end, the method includes: Obtaining a first part or a second part of a first point cloud and rendering it; In response to a triggering annotation instruction, determining a first portion of annotation data of the first portion, or a second portion of annotation data of the second portion; The first portion of the labeled data or the second portion of the labeled data is sent, and there is no intersection between the first portion of the labeled data and the second portion of the labeled data.

6. The method according to claim 5, characterized in that The method further comprises: Rendering the first part of the annotated data or the second part of the annotated data; In response to triggering the marking instruction, modifying the first portion of the marking data or the second portion of the marking data; The modified first portion of the annotation data or the modified second portion of the annotation data is sent, wherein the modified first portion of the annotation data and the modified second portion of the annotation data do not have any intersection.

7. The method according to claim 5 or 6, characterized in that: The front end includes at least one of a browser end and a client end.

8. A point cloud data annotation device, characterized in that: Applied to a cloud server, the device comprises: A receiving module, configured to receive a first portion of annotation data and a second portion of annotation data of a first point cloud, wherein the first portion of annotation data and the second portion of annotation data do not have any intersection; An analysis module, configured to obtain different annotation data based on first annotation data formed by the first part of annotation data and the second part of annotation data according to different point cloud labels in the first annotation data; A determination module is configured to use the different labeled data as a second labeled data sequence; A generating module is used to obtain a second point cloud by data integration according to the first point cloud and the second annotated data sequence.

9. A point cloud data annotation device, characterized in that: Applied to the front end, the device comprises: A rendering module, used for obtaining and rendering the first part or the second part of the first point cloud; a marking module, configured to determine, in response to a triggering marking instruction, a first portion of marking data of the first portion, or a second portion of marking data of the second portion; A sending module is used to send the first part of the marked data or the second part of the marked data.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

13. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 7.

Images