Point cloud data annotation method, device, equipment and storage medium

Abstract

The present disclosure provides a method for labeling point cloud data, which relates to the field of artificial intelligence, specifically to cloud computing, 3D vision and data labeling technology, and can be applied in intelligent cloud scenarios. The method includes: obtaining point cloud data to be labeled; generating a box selection labeling box based on the box selection information of the point cloud data to be labeled; generating a semantic labeling result of the object in the semantic labeling box of the point cloud data to be labeled based on the box selection labeling result of the object in the box selection labeling box. The method for labeling point cloud data provided by the present disclosure improves the accuracy and efficiency of labeling, and saves the labeling cost.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to cloud computing, 3D vision and data annotation technology, and especially to point cloud data annotation methods, devices, equipment and storage media, which can be applied in smart cloud scenarios. Background Art

[0002] Point cloud data is generated by 3D (3-dimension) scanning equipment (such as LiDAR (2D / 3D), stereo camera, time-of-flight camera), represented by a set of vectors in a 3D coordinate system, and is mainly used to represent the outer surface shape of an object. In addition to the geometric position information represented by (X, Y, Z), point cloud data can also represent the RGB color, grayscale value, depth, segmentation results, etc. of a point. At present, objects in point cloud data are generally annotated through machine learning, artificial intelligence, image recognition and other methods. For example, cars, pedestrians, bicycles, and other types of obstacles can be identified from point cloud data. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for annotating point cloud data.

[0004] According to a first aspect of the present disclosure, a method for annotating point cloud data is provided, including: acquiring point cloud data to be annotated; generating a selection annotation box based on selection information of the point cloud data to be annotated; and generating semantic annotation results of objects in a semantic annotation box of the point cloud data to be annotated based on selection annotation results of objects in the selection annotation box.

[0005] According to a second aspect of the present disclosure, a point cloud data annotation device is provided, including: an acquisition module, configured to acquire point cloud data to be annotated; a first generation module, configured to generate a selection annotation box based on the selection information of the point cloud data to be annotated; and a second generation module, configured to generate a semantic annotation result of an object in a semantic annotation box of the point cloud data to be annotated based on the selection annotation result of the object in the selection annotation box.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner in the first aspect.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation manner of the first aspect.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the method described in any implementation manner in the first aspect.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

[0012] Figure 2 is a flow chart of an embodiment of a method for labeling point cloud data according to the present disclosure;

[0013] Figure 3 is a flow chart of another embodiment of a method for labeling point cloud data according to the present disclosure;

[0014] Figure 4 is a flow chart of another embodiment of a method for labeling point cloud data according to the present disclosure;

[0015] Figure 5 is an application scenario diagram of the point cloud data annotation method according to the present disclosure;

[0016] Figure 6 is a structural schematic diagram of an embodiment of a point cloud data annotation device according to the present disclosure;

[0017] Figure 7 It is a block diagram of an electronic device used to implement the point cloud data annotation method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of a method for annotating point cloud data or an apparatus for annotating point cloud data of the present disclosure may be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0022] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send information, etc. Various client applications can be installed on the terminal devices 101, 102, 103.

[0023] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above electronic devices. They may be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0024] The server 105 can provide various services. For example, the server 105 can analyze and process the point cloud data to be annotated obtained from the terminal devices 101, 102, and 103, and generate processing results (such as box selection annotation results and semantic annotation results).

[0025] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be noted that the point cloud data annotation method provided in the embodiment of the present disclosure is generally executed by the server 105 , and accordingly, the point cloud data annotation device is generally set in the server 105 .

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0028] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a method for labeling point cloud data according to the present disclosure. The method for labeling point cloud data comprises the following steps:

[0029] Step 201: Obtain point cloud data to be labeled.

[0030] In this embodiment, the execution subject of the point cloud data annotation method (for example Figure 1 The server 105 shown in the figure can obtain the point cloud data to be annotated. The point cloud data to be annotated is the point cloud data that needs to be annotated. The point cloud data can be collected by a sensor, wherein the type of the sensor can be a point cloud sensor or an image sensor. The point cloud sensor is a sensor that can collect point cloud data, generally a 3D sensor, and the image sensor is a sensor that can collect images, generally a 2D (2-dimension) sensor.

[0031] It should be noted that point cloud data refers to a set of vectors in a three-dimensional coordinate system. Each point in the point cloud data contains three-dimensional coordinates. In addition to representing the geometric position information of a point, point cloud data can also represent the RGB (Red-Green-Blue, three primary colors) color, grayscale value, depth, segmentation results, etc. of a point.

[0032] After collecting the point cloud data, the point cloud sensor will send the collected point cloud data to the server, and the above-mentioned execution entity will obtain the point cloud data to be labeled.

[0033] In some scenarios, after the client's sensor collects point cloud data, it will package the point cloud data and then access the packaged data packet. The above-mentioned execution entity will obtain the accessed data packet and use it as the point cloud data to be annotated.

[0034] Step 202: Generate a selection annotation box based on the selection information of the point cloud data to be annotated.

[0035] In the present embodiment, the above-mentioned execution entity will generate a frame selection annotation box based on the frame selection information of the point cloud data to be annotated. After obtaining the point cloud data to be annotated, the above-mentioned execution entity will display the obtained point cloud data to be annotated so that relevant staff (such as annotators) can annotate it. When annotating, a frame selection will first be performed on the point cloud data to be annotated, that is, a frame to be annotated is obtained through the frame selection operation. In the present embodiment, after the relevant staff selects the point cloud data to be annotated, the above-mentioned execution entity will obtain the frame selection information of the point cloud data to be annotated. The frame selection information may include the size and shape of the frame selection area, etc., and generate a frame selection annotation box based on the frame selection information. Optionally, the above-mentioned execution entity will mark the frame selection annotation box with a color to facilitate identification by the annotator.

[0036] Optionally, the execution subject may identify the object in the box selection and labeling frame, determine the category information of the object, and thereby obtain the box selection and labeling result of the object in the box selection and labeling frame. Here, multiple category information may be pre-configured, such as trucks, buses, small non-motorized vehicles, other obstacles, etc.

[0037] Step 203 , based on the box selection and annotation results of the objects in the box selection and annotation box, generate semantic annotation results of the objects in the semantic annotation box of the point cloud data to be annotated.

[0038] In this embodiment, the execution subject generates the semantic annotation result of the object in the semantic annotation box in the point cloud data to be annotated based on the box selection annotation result of the object in the box selection annotation box. Here, the execution subject first performs semantic segmentation on the point cloud data to be annotated, thereby obtaining a semantic segmentation area, where the semantic segmentation area can be a polygon, a full selection, a rectangle, etc., which is not specifically limited in this embodiment. Then the execution subject generates a semantic annotation box corresponding to the semantic segmentation area.

[0039] It should be noted that if an area already has a box selection annotation result, it will not be semantically annotated. When performing semantic annotation, the box selection annotation result will be displayed to avoid repeated annotation of the same obstacle, thereby improving annotation efficiency.

[0040] Afterwards, when semantically annotating the objects in the semantic annotation box, the annotated box selection annotation results will be displayed on the point cloud data to be annotated. The above-mentioned execution entity will semantically annotate the objects in the semantic segmentation area based on the box selection annotation results, thereby obtaining the semantic annotation results, and then assisting in semantically annotating the point cloud data based on the box selection annotation results.

[0041] For example, assuming that the box selection annotation result is a bus, when the semantic annotation box is a small annotation box next to the box selection annotation box, the object in the semantic annotation box can be annotated as a pedestrian by referring to the box selection annotation result of the box selection annotation box.

[0042] It should be noted that during the labeling process, the above-mentioned execution entity will use a binary file to record the label information of all points in the current frame, and convert it in real time from PCD (a picture storage format) to real-time writing to the binary file, thereby greatly reducing storage time.

[0043] The method for labeling point cloud data provided by the embodiment of the present disclosure first obtains the point cloud data to be labeled; then generates a box selection labeling box based on the box selection information of the point cloud data to be labeled; finally, based on the box selection labeling result of the object in the box selection labeling box, generates the semantic labeling result of the object in the semantic labeling box in the point cloud data to be labeled. The method for labeling point cloud data in this embodiment can simultaneously perform box selection labeling and semantic labeling on the point cloud data to be labeled, thereby integrating the original distributed labeling process into a one-step labeling mode, saving the labeling cost; and performs semantic labeling based on the box selection labeling result, thereby improving the labeling accuracy and efficiency.

[0044] Continue to refer Figure 3 , Figure 3 A process 300 of another embodiment of a method for labeling point cloud data according to the present disclosure is shown. The method for labeling point cloud data comprises the following steps:

[0045] Step 301: Obtain point cloud data to be annotated.

[0046] In this embodiment, the execution subject of the point cloud data annotation method (for example Figure 1 The server 105 shown in the figure will obtain the point cloud data to be annotated. Step 301 is basically the same as step 201 in the above embodiment. The specific implementation method can refer to the above description of step 201, which will not be repeated here.

[0047] Step 302: Generate a 3D annotation frame based on the frame selection information of the point cloud data to be annotated.

[0048] In this embodiment, the above-mentioned execution entity can generate a 3D annotation box based on the box selection information of the point cloud data to be annotated. The above-mentioned execution entity will generate a 3D annotation box based on the box selection information of the point cloud data to be annotated. After obtaining the point cloud data to be annotated, the above-mentioned execution entity will display the acquired point cloud data to be annotated so that relevant staff (such as annotators) can annotate it. When annotating, a box will be selected on the point cloud data to be annotated first, that is, a box to be annotated is obtained through the box selection operation. In this embodiment, after the relevant staff selects the point cloud data to be annotated, the above-mentioned execution entity will obtain the box selection information of the point cloud data to be annotated. The box selection information may include the size and shape of the box selection area, etc., and generate a 3D annotation box based on the box selection information.

[0049] Step 303 , projecting the point cloud data to be annotated onto a two-dimensional view, obtaining a 2D annotation box corresponding to the 3D annotation box, and using the 2D annotation box as a selection annotation box.

[0050] In this embodiment, the execution subject projects the point cloud data to be annotated into a two-dimensional view, obtains a 2D annotation box corresponding to the 3D annotation box, and uses the 2D annotation box as a selection annotation box. The three-dimensional point cloud data can be projected into a two-dimensional view, and then the obstacles in the two-dimensional view are annotated, thereby generating a 2D annotation box corresponding to the obstacles in the point cloud data, and the 2D annotation box can assist in the identification of obstacles.

[0051] Step 304 , determining the category information of the object in the 2D annotation box, and obtaining the box selection annotation result of the object in the box selection annotation box.

[0052] In this embodiment, the above-mentioned execution entity will determine the category information of the object in the 2D annotation box, that is, determine which of the pre-configured categories the object in the 2D annotation box belongs to, for example, a passenger car, a van, a truck, etc., thereby obtaining the box selection annotation result of the object in the box selection annotation box.

[0053] Step 305 , based on the box selection and annotation results of the objects in the box selection and annotation box, generate semantic annotation results of the objects in the semantic annotation box of the point cloud data to be annotated.

[0054] Step 305 is basically the same as step 203 of the aforementioned embodiment. The specific implementation method can refer to the aforementioned description of step 203, which will not be repeated here.

[0055] from Figure 3 It can be seen that Figure 2 Compared with the corresponding embodiments, the method for labeling point cloud data in this embodiment highlights the step of generating box selection labeling results, thereby improving the accuracy of the box selection labeling results and the labeling efficiency.

[0056] Continue to refer Figure 4 , Figure 4 A process 400 of another embodiment of a method for labeling point cloud data according to the present disclosure is shown. The method for labeling point cloud data comprises the following steps:

[0057] Step 401: Obtain point cloud data to be annotated.

[0058] Step 402: Generate a 3D annotation frame based on the frame selection information of the point cloud data to be annotated.

[0059] Step 403 , projecting the point cloud data to be annotated onto a two-dimensional view, obtaining a 2D annotation box corresponding to the 3D annotation box, and using the 2D annotation box as a selection annotation box.

[0060] Step 404 , determining the category information of the object in the 2D annotation box, and obtaining the box selection annotation result of the object in the box selection annotation box.

[0061] Steps 401-404 are basically consistent with steps 301-304 of the aforementioned embodiment. For specific implementation methods, reference may be made to the aforementioned description of steps 301-304, which will not be repeated here.

[0062] Step 405: semantically segment the point cloud data to be annotated to obtain a semantically segmented area.

[0063] In this embodiment, the execution subject of the point cloud data annotation method (for example Figure 1 The server 105 shown in the figure will perform semantic segmentation on the point cloud data to be annotated, thereby obtaining multiple semantic segmentation regions, where the semantic segmentation region can be a polygon, a full selection, or a rectangle, etc., which is not specifically limited in this embodiment. Semantic segmentation can be implemented using a segmentation network, such as RSNet (Recurrent Slice Networks for 3D Segmentation on Point Clouds), or other models or networks can be used for segmentation, which is not specifically limited in this embodiment.

[0064] Step 406: Generate a semantic annotation box corresponding to the semantic segmentation area.

[0065] In this embodiment, the execution entity generates a semantic annotation box corresponding to the semantic segmentation area. Optionally, the execution entity marks the semantic annotation box with a color to facilitate identification by the annotator.

[0066] Step 407: Display the box selection annotation result on the point cloud data to be annotated.

[0067] In this embodiment, the above-mentioned execution entity will display the selection annotation results and the selection annotation box on the point cloud data to be annotated, that is, the annotated area and results will be displayed on the point cloud data to be annotated, so as to avoid repeated annotation of the same area or the same obstacle.

[0068] In some optional implementations of this embodiment, the above-mentioned point cloud data annotation method further includes: hiding the points of the point cloud data in the marking box.

[0069] In this implementation, the execution subject will hide the point cloud points in the box selection annotation box while displaying the box selection annotation result and the box selection annotation box, so as to facilitate the annotation of other areas. For example, after the car is box-selected and annotated, when performing semantic annotation, multiple points in the car's annotation box will be hidden, so as to facilitate the annotation of "pedestrians" and "ground".

[0070] Step 408 , semantically annotate the objects in the semantic segmentation area based on the box selection annotation result to obtain a semantic annotation result.

[0071] In this embodiment, the execution subject will semantically annotate the objects in the semantic segmentation area based on the box selection annotation result, thereby obtaining the corresponding semantic annotation result. When semantically annotating the objects in the semantic annotation box, the annotated box selection annotation result will be displayed on the point cloud data to be annotated, and the execution subject will semantically annotate the objects in the semantic segmentation area based on the box selection annotation result, thereby obtaining the semantic annotation result, thereby assisting in semantically annotating the point cloud data based on the box selection annotation result. For example, assuming that the box selection annotation result is a bus, then when the semantic annotation box is a small annotation box next to the box selection annotation box, the object in the semantic annotation box can be annotated as a pedestrian with reference to the box selection annotation result of the box selection annotation box.

[0072] Step 409: Display the box selection annotation results and the semantic annotation results.

[0073] In this embodiment, the above-mentioned execution entity will also display the box selection annotation results and semantic annotation results on the point cloud data to be annotated, so as to facilitate the annotator to view and accept the annotation results.

[0074] from Figure 4 It can be seen that Figure 3 Compared with the corresponding embodiments, the point cloud data annotation method in this embodiment highlights the step of generating semantic annotation results. The method annotates the objects in the semantic annotation box based on the box selection annotation results, thereby integrating the box selection annotation and the semantic annotation, saving the annotation cost, avoiding repeated annotation of the same obstacle, and improving the annotation efficiency.

[0075] Further references Figure 5 , Figure 5 An application scenario diagram of the point cloud data annotation method according to the present disclosure is shown. In this application scenario, the annotation type is pre-configured, such as a box, region, or line, and the annotation category is pre-set, such as a truck, a bus, a small vehicle, a non-motorized vehicle, etc.

[0076] First, the execution entity 502 obtains the point cloud data 501 to be annotated, and then annotates the point cloud data 501 to be annotated in the frame selection mode, that is, based on the frame selection information of the point cloud data to be annotated, generates a frame selection annotation box, and identifies the object in the frame selection annotation box to determine the frame selection annotation result 503.

[0077] After that, after switching to the semantic mode, the points in the box selection annotation box will be hidden, and new semantic annotations can be added. Here, the point cloud data to be annotated will be semantically segmented to obtain the semantic segmentation area. The semantic segmentation mode has a variety of ways to draw 2D areas, including polygons, full selection, rectangles, etc. You can draw a 2D area in the point cloud window. After drawing, it will be projected to the 2D area. The points projected in the 2D area will be selected, and the selected point cloud will be displayed in red. In temporary mode, you can select the category to set the area pair classification. If the first-level classification has been selected, you can continue to operate the point cloud interface. The temporary area will set the field to the selected category. The semantic segmentation mode supports cropping the point cloud data with the current box selection answer to assist in annotation. The semantic segmentation mode also supports the combined display and hiding of multiple area pairs.

[0078] Finally, the execution body 502 generates a semantic annotation result 504 of the object in the semantic annotation box of the to-be-annotated point cloud 501 based on the box selection and annotation result of the object in the box selection and annotation box 503 .

[0079] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a point cloud data annotation device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0080] like Figure 6 As shown, the point cloud data annotation device 600 of this embodiment includes: an acquisition module 601, a first generation module 602, and a second generation module 603. The acquisition module 601 is configured to acquire the point cloud data to be annotated; the first generation module 602 is configured to generate a selection annotation box based on the selection information of the point cloud data to be annotated; the second generation module 603 is configured to generate a semantic annotation result of the object in the semantic annotation box of the point cloud data to be annotated based on the selection annotation result of the object in the selection annotation box.

[0081] In the point cloud data annotation device 600, the specific processing of the acquisition module 601, the first generation module 602 and the second generation module 603 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of steps 201 - 203 in the corresponding embodiment are not repeated here.

[0082] In some optional implementations of the present embodiment, the first generation module includes: a first generation sub-module, configured to generate a 3D annotation box based on the box selection information of the point cloud data to be annotated; a second generation sub-module, configured to project the point cloud data to be annotated onto a two-dimensional view, obtain a 2D annotation box corresponding to the 3D annotation box, and use the 2D annotation box as a box selection annotation box; and the above-mentioned point cloud data annotation device 600 also includes: a determination module, configured to determine the category information of the object in the 2D annotation box, and obtain the box selection annotation result of the object in the box selection annotation box.

[0083] In some optional implementations of the present embodiment, the point cloud data annotation device 600 also includes: a segmentation module, configured to perform semantic segmentation on the point cloud data to be annotated to obtain a semantic segmentation area, wherein the semantic segmentation area includes: a polygonal area, a fully selected area or a rectangular area; and a third generation module, configured to generate a semantic annotation box corresponding to the semantic segmentation area.

[0084] In some optional implementations of this embodiment, the second generation module includes: a display module, configured to display the box selection annotation results on the point cloud data to be annotated; and an annotation module, configured to semantically annotate objects within the semantic segmentation area based on the box selection annotation results to obtain semantic annotation results.

[0085] In some optional implementations of this embodiment, the point cloud data annotation device 600 further includes: a hiding module configured to hide the points of the point cloud data in the selected annotation box.

[0086] In some optional implementations of this embodiment, the point cloud data annotation device 600 further includes: a display module configured to display the box selection annotation results and the semantic annotation results.

[0087] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0088] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0089] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0090] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0091] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the annotation method of point cloud data. For example, in some embodiments, the annotation method of point cloud data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the annotation method of the point cloud data described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the annotation method of point cloud data in any other appropriate manner (e.g., by means of firmware).

[0092] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0094] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0096] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0097] Cloud computing refers to a technology system that uses the network to access a shared pool of physical or virtual resources that is elastic and scalable. Resources may include servers, operating systems, networks, software, applications, or storage devices, and can be deployed and managed in an on-demand, self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain, as well as model training.

[0098] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0099] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0100] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for labeling point cloud data. include: Get the point cloud data to be annotated; Generate a selection annotation box based on the selection information of the point cloud data to be annotated; Generate a semantic annotation result of the object in the semantic annotation box of the point cloud data to be annotated based on the box selection annotation result of the object in the box selection annotation box; Wherein, the method further comprises: Performing semantic segmentation on the point cloud data to be annotated to obtain a semantic segmentation area, wherein the semantic segmentation area includes: a polygonal area, a fully selected area or a rectangular area; Generate a semantic annotation box corresponding to the semantic segmentation area; Wherein, the generating the semantic annotation result of the object in the semantic annotation box of the point cloud data to be annotated based on the box selection annotation result of the object in the box selection annotation box comprises: Displaying the box selection annotation result on the point cloud data to be annotated; Semantic annotation is performed on the objects in the semantic segmentation area based on the box selection annotation result to obtain a semantic annotation result.

2. The method according to claim 1, in, The generating a box selection annotation box based on the box selection information of the point cloud data to be annotated includes: Generate a 3D annotation frame based on the frame selection information of the point cloud data to be annotated; Projecting the point cloud data to be annotated onto a two-dimensional view to obtain a 2D annotation box corresponding to the 3D annotation box, and using the 2D annotation box as a selection annotation box; and The method further comprises: Determine the category information of the object in the 2D annotation box, and obtain the box selection annotation result of the object in the box selection annotation box.

3. The method according to claim 1, further comprising: include: Hide the point cloud data points in the selected annotation box.

4. The method according to any one of claims 1 to 3, further comprising: include: The box selection annotation result and the semantic annotation result are displayed.

5. A point cloud data annotation device, include: An acquisition module is configured to acquire point cloud data to be annotated; A first generating module is configured to generate a selection annotation box based on the selection information of the point cloud data to be annotated; A second generating module is configured to generate a semantic annotation result of the object in the semantic annotation box of the point cloud data to be annotated based on the box selection annotation result of the object in the box selection annotation box; Wherein, the device further comprises: A segmentation module is configured to perform semantic segmentation on the point cloud data to be annotated to obtain a semantic segmentation area, wherein the semantic segmentation area includes: a polygonal area, a fully selected area or a rectangular area; A third generating module is configured to generate a semantic annotation box corresponding to the semantic segmentation area; Wherein, the second generation module includes: A display module, configured to display the box selection annotation result on the point cloud data to be annotated; The labeling module is configured to perform semantic labeling on the objects in the semantic segmentation area based on the box selection labeling result to obtain a semantic labeling result.

6. The device according to claim 5, in, The first generation module comprises: A first generating submodule is configured to generate a 3D annotation frame based on the frame selection information of the point cloud data to be annotated; A second generating submodule is configured to project the point cloud data to be annotated onto a two-dimensional view, obtain a 2D annotation box corresponding to the 3D annotation box, and use the 2D annotation box as a selection annotation box; and The device also includes: The determination module is configured to determine the category information of the object in the 2D annotation box and obtain the box selection annotation result of the object in the box selection annotation box.

7. The device according to claim 5, further comprising: include: The hiding module is configured to hide the point cloud data points in the selection and annotation box.

8. The device according to any one of claims 5 to 7, further comprising: include: The display module is configured to display the box selection annotation result and the semantic annotation result.

9. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images