Point cloud data classification labeling method and device

Abstract

The application discloses a kind of point cloud data classification labeling method and device. Among them, the method includes: obtaining the point cloud data set on the driving path of target vehicle;Remove ground point cloud data in point cloud data set to obtain target point cloud data set;The point cloud data set is clustered to obtain envelope box set, the object corresponding to the first target envelope box in envelope box set and the detection frame of point cloud data recognition device in target vehicle is labeled as first type, the object corresponding to the second target envelope box in envelope box set and not coinciding with detection frame is labeled as second type, to complete the classification labeling of point cloud data, wherein, detection frame is used to extract point cloud data.The application solves the technical problem that the accuracy of obstacle detection is low due to the lack of pre-classification labeling of the point cloud data corresponding to the obtained obstacles during vehicle driving.

Description

Technical Field

[0001] This application relates to the field of point cloud data processing, and more specifically, to a method and apparatus for classifying and labeling point cloud data. Background Technology

[0002] With the development of intelligent driving technology, obstacle perception solutions have become increasingly diverse. Traditional detection methods cluster nearby points into a group based on point cloud data distribution characteristics such as the distance between points, which is considered a single detected object. Then, target classification is performed based on features such as the size and shape of the clusters. This approach can extract all obstacles in the driving process relatively comprehensively with few missed detections. However, classification based on simple features such as size and shape has low accuracy and is difficult to perform detailed classification. It also cannot detect irregular objects. Deep learning-based detection solutions use the entire frame of point cloud as input and extract features at various levels through the network for target regression and classification. However, due to limitations in network performance, it is difficult to detect some small, uncommon, or irregular objects, resulting in a high rate of missed detections.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method and apparatus for classifying and labeling point cloud data, which at least solves the technical problem of low obstacle detection accuracy caused by the lack of pre-classification and labeling of point cloud data corresponding to obstacles during vehicle operation.

[0005] According to one aspect of the embodiments of this application, a method for classifying and labeling point cloud data is provided, comprising: acquiring a set of point cloud data along the driving path of a target vehicle; removing ground point cloud data from the point cloud data set to obtain a target point cloud data set; clustering the point cloud data set to obtain a set of envelope boxes; labeling objects corresponding to a first target envelope box that overlaps with the detection box of a point cloud data recognition device in the target vehicle as a first type; and labeling objects corresponding to a second target envelope box that does not overlap with the detection box as a second type, thereby completing the classification and labeling of the point cloud data, wherein the detection box is used to extract point cloud data.

[0006] Optionally, the detection box and the target envelope are determined to coincide in the following cases: the center of the projection of the detection box on the plane is inside the target envelope, or the center of the target envelope is inside the projection.

[0007] Optionally, the object corresponding to the first target envelope that overlaps with the detection frame of the point cloud data acquisition device in the target vehicle is labeled as a first type, and the object corresponding to the second target envelope that does not overlap with the detection frame is labeled as a second type. This includes: in a first case, labeling the object corresponding to the first target envelope in the detection frame as the first type, the first case including: the detection frame includes one envelope; in a second case, labeling the objects corresponding to multiple first target envelopes in the detection frame as the first type, the second case including: the detection frame includes multiple envelopes; in a third case, extracting the target point cloud data in the third target envelope, re-clustering the target point cloud data to obtain a target envelope set, and labeling the object corresponding to the first target envelope that overlaps with the detection frame of the point cloud data recognition device in the target vehicle as the first type, wherein the third case includes: the third target envelope includes multiple detection frames.

[0008] Optionally, extracting target point cloud data from the third target envelope includes: determining the height of all points in the point cloud corresponding to the third envelope; identifying points whose height falls within the height range of the target detection boxes in multiple detection boxes and satisfy a first set condition as target point cloud data, wherein the first set condition includes that the number of rays along a set direction passing through the border of the detection box is odd.

[0009] Optionally, after completing the classification and labeling of the point cloud data, the method further includes: obtaining the distance between all objects corresponding to the classified and labeled point cloud data and the ground corresponding to the ground point cloud; identifying objects with a distance greater than zero as floating objects, and otherwise identifying them as ground objects.

[0010] Optionally, after completing the classification and labeling of the point cloud data, the method further includes: matching the shape of the object labeled as the first type with the object shape in a pre-stored database, and determining the object type of the first type of object based on the matching result, wherein the database includes at least the shape and corresponding name of vehicles, pedestrians and roadblocks.

[0011] Optionally, after completing the classification and labeling of the point cloud data, the method further includes: using a target model to identify objects on the driving path of the target vehicle to determine the type of objects on the driving path, wherein the target model is trained using the labeled point cloud data as the training dataset; if the type of the object on the driving path is the second type and the object on the driving path is a floating object, determining the height of the object on the driving path above the ground; if the height of the object on the driving path above the ground is less than the height of the target vehicle, changing the driving direction of the target vehicle to avoid the object on the driving path.

[0012] According to another aspect of the embodiments of this application, a method for classifying and labeling point cloud data is also provided, including: obtaining a target point cloud data set; clustering the target point cloud data set to obtain an envelope set; labeling objects corresponding to a first target envelope set that overlaps with a pre-determined detection box in the envelope set as a first type; and labeling objects corresponding to a second target envelope set that do not overlap with a detection box in the envelope set as a second type, so as to complete the classification and labeling of the point cloud data, wherein the detection box is used to extract point cloud data.

[0013] According to another aspect of the embodiments of this application, a point cloud data classification and labeling device is also provided, comprising: an acquisition module for acquiring a set of point cloud data along the driving path of a target vehicle; a removal module for removing ground point cloud data from the point cloud data set to obtain a target point cloud data set; and a labeling module for clustering the target point cloud data set to obtain an envelope set, labeling objects corresponding to a first target envelope set that overlaps with the detection frame of the point cloud data recognition device in the target vehicle as a first type, and labeling objects corresponding to a second target envelope set that do not overlap with the detection frame as a second type, thereby completing the classification and labeling of the point cloud data, wherein the detection frame is used to extract point cloud data.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-mentioned point cloud data classification and annotation method.

[0015] According to another aspect of the embodiments of this application, a processor is also provided, which is used to run a program, wherein the program executes the above-described point cloud data classification and annotation method during runtime.

[0016] In this embodiment, a point cloud data set along the driving path of the target vehicle is acquired; ground point cloud data is removed from the point cloud data set to obtain a target point cloud data set; the point cloud data set is clustered to obtain an envelope set; objects corresponding to the first target envelope set that overlap with the detection frame of the point cloud data recognition device in the target vehicle are labeled as a first type; objects corresponding to the second target envelope set that do not overlap with the detection frame are labeled as a second type, thereby completing the classification and labeling of the point cloud data. The detection frame is used to extract point cloud data. By clustering the point cloud data set to obtain the envelope set, comparing the detection frame with the envelope frame, and finally classifying and labeling the point cloud data based on whether the detection frame and the envelope frame overlap, the point cloud data is accurately labeled, thus achieving the technical effect of improving the accuracy of point cloud data classification and labeling. This solves the technical problem of low obstacle detection accuracy caused by the lack of pre-classification and labeling of the point cloud data corresponding to the acquired obstacles during vehicle driving. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for a point cloud data classification and annotation method according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a point cloud data classification and annotation method according to this application;

[0020] Figure 3a This is a schematic diagram illustrating an optional point cloud data clustering scenario according to an embodiment of this application;

[0021] Figure 3b This is a schematic diagram illustrating another optional point cloud data clustering scenario according to an embodiment of this application;

[0022] Figure 3c This is a schematic diagram illustrating another optional clustering of point cloud data according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of another optional point cloud data classification and annotation method according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an optional point cloud data classification and labeling device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] According to an embodiment of this application, an embodiment of a point cloud data classification and labeling method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a classification and annotation method for point cloud data is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the point cloud data classification and labeling method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned application vulnerability detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0032] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0033] According to an embodiment of this application, an embodiment of a point cloud data classification and annotation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 2 This is a flowchart of a point cloud data classification and annotation method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0035] Step S202: Obtain the point cloud data set along the driving path of the target vehicle;

[0036] Step S204: Remove the ground point cloud data from the point cloud data set to obtain the target point cloud data set;

[0037] Step S206: Cluster the point cloud data set to obtain an envelope set. Label the object corresponding to the first target envelope set that overlaps with the detection frame of the point cloud data recognition device in the target vehicle as the first type. Label the object corresponding to the second target envelope set that does not overlap with the detection frame as the second type, so as to complete the classification and labeling of the point cloud data. The detection frame is used to extract the point cloud data.

[0038] Through the above steps, the point cloud data set can be clustered to obtain the envelope box, and then the detection box is compared with the envelope box. Finally, the point cloud data is classified and labeled according to whether the detection box and the envelope box overlap. This achieves the goal of accurately labeling the point cloud data, thereby solving the technical problem of low obstacle detection accuracy caused by the lack of pre-classification and labeling of the point cloud data corresponding to the acquired obstacles during vehicle operation.

[0039] It should be noted that the first type in this application embodiment includes conventional objects, such as motor vehicles, two-wheeled vehicles, pedestrians, traffic cones, and other objects pre-stored in the database; the second type includes unconventional objects, such as cardboard boxes, construction fences, unknown obstructions, and partially obstructed objects.

[0040] In step S202, a set of point cloud data along the target vehicle's driving path is obtained, which can be acquired using devices such as vehicle-mounted LiDAR. Obtaining the data directly from vehicle-mounted devices provides a more accurate point cloud dataset that better reflects the actual driving scenario.

[0041] In step S204, removing ground point cloud data from the point cloud dataset yields the target point cloud dataset. This process removes interference from ground point clouds on obstacle point clouds, improving annotation accuracy. Some optional methods include RANSAC (Random Sample Consistency) plane fitting, CSF (Cloth Simulation Filtering), and line fitting to remove ground point cloud data.

[0042] In step S206, envelope boxes are obtained by clustering the point cloud dataset. All envelope boxes are then compared with detection boxes to avoid missed detections during classification and labeling, thereby improving the accuracy of classification and labeling. Detection boxes can be 3D rectangles used to extract point cloud data and identify objects corresponding to the point cloud data. Clustering methods include, but are not limited to, DBSCAN (density-based clustering) and Meanshift (mean shift).

[0043] Understandably, there are usually two ways to determine whether the detection box coincides with the envelope. In the case that the center of the projection box of the detection box on the plane is inside the envelope or the center of the envelope is inside the projection box, the detection box and the envelope can be polygons.

[0044] During clustering, there are three possible scenarios: First, a detection box contains one envelope box, meaning one detection box corresponds to one object; second, a detection box contains multiple envelope boxes, meaning one object is clustered into multiple objects; and third, a detection box contains multiple detection boxes, meaning multiple closely spaced objects are clustered into one object. Figure 3a The first case is shown: one detection box corresponds to one envelope box, that is, one object corresponds to one envelope box. The dashed box is the detection box, and the solid box is the envelope box obtained by clustering. Figure 3b The image shows an optional second case: a detection box contains two envelope boxes, that is, one object corresponds to two envelope boxes; Figure 3c The diagram illustrates an optional third case: an envelope contains two detection boxes, meaning two objects correspond to one envelope.

[0045] In the first case, the object corresponding to the envelope that overlaps with the detection box can be labeled as the first type according to the method in step S206; otherwise, it can be labeled as the second type.

[0046] In the second case, since an object is clustered into multiple bounding boxes, the type of the object corresponding to one of the bounding boxes can be determined. If it is the first type, then the objects corresponding to the multiple bounding boxes in the detection box are labeled as the first type.

[0047] It should be noted that in the third scenario, since multiple objects are clustered into a single bounding box, the point clouds corresponding to these objects need to be re-extracted and labeled. For example: extract the target point cloud data from the third target bounding box, re-cluster the target point cloud data to obtain target bounding boxes, and label the objects corresponding to the first target bounding box that overlaps with the detection box of the point cloud data recognition device in the target vehicle as the first type. The objects corresponding to the second target bounding box are labeled as, for example, of the first and second types. The third target bounding box can be a single bounding box formed by clustering multiple objects in the third scenario.

[0048] Specifically, firstly, the height of all points in the point cloud corresponding to the third envelope is determined; then, points whose heights are within the height range of the target detection box in multiple detection boxes and that satisfy the first set condition are determined as target point cloud data. The first set condition includes that the number of rays along the set direction passing through the border of the detection box is odd.

[0049] It should be noted that, in this embodiment, "a point within the height range of the target detection box" can be understood as a point whose height is greater than the minimum height (the smallest border) of the target detection box and less than the maximum height (the largest border). For example, if the height of point m is h, and the height range of the target detection box is (ha, h+a), where a is a positive integer, then it can be determined that the height of point m is within the height range of the target detection box.

[0050] It is understandable that, taking a 3D rectangle as an example, once it is determined that the height of point m is within the height range of the target detection box, if the border of the projection of point m through the target detection box on the set plane is 1, then point m is considered to be inside the target detection box.

[0051] Point clouds within the same detection bounding box are labeled as a single object. The point clouds extracted from multiple detection bounding boxes are then labeled with the object type corresponding to the determined third target envelope. The object type corresponding to the third target envelope is determined using the method described in step S206.

[0052] In some optional embodiments, after the point cloud data is classified and labeled, a secondary labeling can be performed on the classified and labeled point cloud. Specifically, the distances between all objects corresponding to the classified and labeled point cloud data and the ground corresponding to the ground point cloud are obtained; then, objects with a distance greater than zero are identified as floating objects, and those without are identified as ground objects.

[0053] Ground objects can be things that are close to the ground, such as stones, roadblocks, and fences. Floating objects can be objects that float in the air, such as balloons, plastic bags, and objects suspended in mid-air, which are not in contact with the ground.

[0054] It should be noted that the purpose of secondary labeling is to better distinguish obstacles in order to determine whether the target vehicle needs to avoid them. For example, if the obstacle is a floating object and its height is higher than that of the target vehicle (the height of the highest point of the target vehicle from the ground), then avoidance is not required.

[0055] Furthermore, the shape of an object labeled as type 1 can be matched with the object shapes in a pre-stored database, and the object type of type 1 object can be determined based on the matching results. The database includes at least the shapes and corresponding names of vehicles, pedestrians, and roadblocks.

[0056] Specifically, the point cloud shape of the target first-type object in the first-type object set is compared with the object shape in the database. If the shapes match, the name of the target first-type object is determined according to the name mapping relationship corresponding to the object shape.

[0057] In some embodiments of this application, a target model is used to identify objects on the driving path of the target vehicle to determine the type of objects on the driving path. The target model is trained using labeled point cloud data as the training dataset. If the type of object on the driving path is the second type and the object on the driving path is a floating object, the height of the object on the driving path above the ground is determined. If the height of the object on the driving path above the ground is less than the height of the target vehicle, the driving direction of the target vehicle is changed to avoid the object on the driving path.

[0058] Specifically, when an object on the driving path is identified as a Type 1 object, a mapping relationship in the database can be used for matching. Based on the matching result, the object type is determined, thus deciding whether to avoid it. When a Type 2 object is identified, since Type 2 objects are unconventional, secondary annotations can be used to determine whether it is a floating object or a ground-hugging object. If it is determined to be a ground-hugging object and its height is less than the target vehicle's chassis height, the target vehicle does not need to avoid it. Accurate judgment improves the continuity of vehicle driving, avoiding excessive ineffective avoidance that causes the vehicle to change its route multiple times, thereby improving the traffic efficiency of the target vehicle.

[0059] This application also provides a method for classifying and labeling point cloud data, such as... Figure 4 As shown, it includes:

[0060] Step S402: Obtain the point cloud data set;

[0061] Step S404: Cluster the target point cloud data set to obtain an envelope set. Label the object corresponding to the first target envelope set that overlaps with the pre-determined detection box as the first type. Label the object corresponding to the second target envelope set that does not overlap with the detection box as the second type to complete the classification and labeling of the point cloud data. The detection box is used to extract the point cloud data.

[0062] The point cloud data classification and annotation method provided in this application embodiment can also be applied to a point cloud data classification and annotation device provided in this application embodiment, such as... Figure 5 As shown, it includes: an acquisition module 50, used to acquire a set of point cloud data along the driving path of the target vehicle; a removal module 52, used to remove ground point cloud data from the point cloud data set to obtain a target point cloud data set; and a labeling module 54, used to cluster the target point cloud data set to obtain an envelope set, label the objects corresponding to the first target envelope set that overlap with the detection frame of the point cloud data recognition device in the target vehicle as a first type, and label the objects corresponding to the second target envelope set that do not overlap with the detection frame as a second type, so as to complete the classification and labeling of the point cloud data, wherein the detection frame is used to extract point cloud data.

[0063] The annotation module 54 includes: a first annotation submodule, an extraction submodule, a second annotation submodule, a matching submodule, and a control submodule. The first annotation submodule is used to: in a first case, annotate the object corresponding to the first target envelope in the detection box as a first type, the first case including: the detection box includes one envelope; in a second case, annotate the objects corresponding to multiple first target envelopes in the detection box as the first type, the second case including: the detection box includes multiple envelopes; in a third case, extract the target point cloud data in the third target envelope, re-cluster the target point cloud data to obtain a target envelope set, and annotate the objects corresponding to the first target envelopes in the target envelope set that overlap with the detection box of the point cloud data recognition device in the target vehicle as the first type, wherein the third case includes: the third target envelope includes multiple detection boxes.

[0064] The extraction submodule is used to determine the height of all points in the point cloud corresponding to the third envelope box; points whose height is within the height range of the target detection box in multiple detection boxes and meet the first set condition are determined as target point cloud data. The first set condition includes that the number of borders of the projection of the target detection box onto the plane by the ray along the set direction is odd.

[0065] The second annotation submodule is used to obtain the distance between all objects in the point cloud data after classification and annotation and the ground corresponding to the ground point cloud; objects with a distance greater than zero are identified as floating objects, otherwise they are identified as ground objects.

[0066] The matching submodule is used to match the shape of an object labeled as type 1 with the object shapes in a pre-stored database, and determine the object type of type 1 based on the matching result. The database includes at least the shapes and corresponding names of vehicles, pedestrians and roadblocks.

[0067] The control submodule is used to identify objects on the driving path of the target vehicle using the target model to determine the type of objects on the driving path. The target model is trained using labeled point cloud data as the training dataset. If the type of object on the driving path is type 2 and the object on the driving path is a floating object, the height of the object on the driving path above the ground is determined. If the height of the object on the driving path above the ground is less than the height of the target vehicle, the driving direction of the target vehicle is changed to avoid the object on the driving path.

[0068] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute a classification and labeling method for point cloud data.

[0069] The aforementioned non-volatile storage medium is used to store a program that performs the following functions: acquiring a set of point cloud data along the driving path of the target vehicle; removing ground point cloud data from the point cloud data set to obtain a target point cloud data set; clustering the point cloud data set to obtain a set of envelope boxes; labeling the objects corresponding to the first target envelope box that overlaps with the detection box of the point cloud data recognition device in the target vehicle as a first type; labeling the objects corresponding to the second target envelope box that does not overlap with the detection box as a second type, thereby completing the classification and labeling of the point cloud data, wherein the detection box is used to extract the point cloud data.

[0070] According to another aspect of the embodiments of this application, a processor is also provided, which is used to run a program, wherein the program performs the determination of the target emotion classification model as described above when it runs.

[0071] The processor described above is used to run a program that performs the following functions: acquiring a set of point cloud data along the driving path of the target vehicle; removing ground point cloud data from the point cloud data set to obtain a target point cloud data set; clustering the point cloud data set to obtain a set of envelope boxes; labeling the objects corresponding to the first target envelope box that overlaps with the detection box of the point cloud data recognition device in the target vehicle as a first type; labeling the objects corresponding to the second target envelope box that does not overlap with the detection box as a second type, so as to complete the classification and labeling of the point cloud data, wherein the detection box is used to extract point cloud data.

[0072] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0073] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0077] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0078] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for classifying and labeling point cloud data, characterized in that, The method comprises: acquiring a point cloud data set on a driving path of a target vehicle; eliminating ground point cloud data in the point cloud data set to obtain a target point cloud data set; clustering the point cloud data set to obtain an envelope box set, labeling objects corresponding to first target envelope boxes in the envelope box set that coincide with a detection box of a point cloud data recognition device in the target vehicle as a first type, and labeling objects corresponding to second target envelope boxes in the envelope box set that do not coincide with the detection box as a second type, to complete classification labeling of the point cloud data, wherein the detection box is used to extract the point cloud data; wherein the clustering of the point cloud data set to obtain the envelope box set, and the labeling of objects corresponding to the first target envelope boxes in the envelope box set that coincide with the detection box of the point cloud data recognition device in the target vehicle as the first type, comprises: in a first case, labeling objects corresponding to the first target envelope boxes in the detection box as the first type, the first case including that the detection box includes one envelope box; in a second case, labeling objects corresponding to multiple first target envelope boxes in the detection box as the first type, the second case including that the detection box includes multiple envelope boxes; and in a third case, extracting target point cloud data in a third target envelope box, re-clustering the target point cloud data to obtain a target envelope box set, and labeling objects corresponding to first target envelope boxes in the target envelope box set that coincide with the detection box of the point cloud data recognition device in the target vehicle as the first type, wherein the third case includes that the third target envelope box includes multiple detection boxes.

2. The method of claim 1, wherein, The detection box is determined to coincide with the envelope box in the following cases: in a case where a center of a projection box of the detection box on a plane is within the envelope box or a center of the envelope box is within the projection box, the detection box is determined to coincide with the envelope box.

3. The method of claim 1, wherein, The extraction of the target point cloud data in the third target envelope box comprises: determining the height of all points in the point cloud corresponding to the third target envelope box; determining points whose height is within a height range of a target detection box in the multiple detection boxes and that satisfy a first set condition as the target point cloud data, the first set condition including that a number of edge boxes of a projection of the target detection box on a plane through which a ray in a set direction passes is odd.

4. The method of claim 1, wherein, After the classification labeling of the point cloud data is completed, the method further comprises: acquiring distances between all objects corresponding to the point cloud data after the classification labeling and a ground surface corresponding to the ground point cloud; determining objects whose distance is greater than zero as floating objects, and otherwise as ground objects.

5. The method of claim 1, wherein, After the classification labeling of the point cloud data is completed, the method further comprises: matching a shape of an object labeled as the first type with object shapes in a pre-stored database, and determining an object category in the first type according to a matching result, wherein the object category is a subclass of the first type, and the database at least includes shapes and corresponding names of vehicles, pedestrians, and roadblocks.

6. The method of claim 4, wherein, After the classification labeling of the point cloud data is completed, the method further comprises: identifying the object on the driving path of the target vehicle by using a target model to determine the type of the object on the driving path, wherein the target model is obtained by training the labeled point cloud data as a training data set; in the case that the type of the object on the driving path is a second type and the object on the driving path belongs to a floating object, determining the height of the object on the driving path from the ground; in the case that the height of the object on the driving path from the ground is less than the height of the target vehicle, changing the driving direction of the target vehicle to avoid the object on the driving path.

7. A classification labeling method of point cloud data, characterized by, comprises: obtaining a target point cloud data set; clustering the target point cloud data set to obtain an envelope box set, labeling the object corresponding to the first target envelope box in the envelope box set that coincides with a predetermined detection box as a first type, and labeling the object corresponding to the second target envelope box in the envelope box set that does not coincide with the detection box as a second type, to complete the classification labeling of the point cloud data, wherein the detection box is used to extract the point cloud data; wherein the clustering of the point cloud data set to obtain an envelope box set, and the labeling of the object corresponding to the first target envelope box in the envelope box set that coincides with the detection box of the point cloud data recognition device as a first type, comprises: in a first case, labeling the object corresponding to the first target envelope box in the detection box as the first type, the first case comprising: the detection box includes one envelope box; in a second case, labeling the objects corresponding to multiple first target envelope boxes in the detection box as the first type, the second case comprising: the detection box includes multiple envelope boxes; in a third case, extracting target point cloud data in a third target envelope box, re-clustering the target point cloud data to obtain a target envelope box set, and labeling the object corresponding to the first target envelope box in the target envelope box set that coincides with the detection box of the point cloud data recognition device as the first type, wherein the third case comprises: the third target envelope box includes multiple detection boxes. 8.A device for classifying and labeling point cloud data, characterized by comprising: comprises: an acquisition module configured to obtain a point cloud data set on a driving path of a target vehicle; an elimination module configured to eliminate ground point cloud data in the point cloud data set to obtain a target point cloud data set; a labeling module configured to cluster the target point cloud data set to obtain an envelope box set, label the object corresponding to the first target envelope box in the envelope box set that coincides with a detection box of a point cloud data recognition device in the target vehicle as a first type, and label the object corresponding to the second target envelope box in the envelope box set that does not coincide with the detection box as a second type, to complete the classification labeling of the point cloud data, wherein the detection box is used to extract the point cloud data; The labeling module is further configured to perform the following steps: in a first case, labeling an object corresponding to the first target bounding box in the detection frame as the first type, the first case including that the detection frame includes one bounding box; in a second case, labeling objects corresponding to a plurality of the first target bounding boxes in the detection frame as the first type, the second case including that the detection frame includes a plurality of bounding boxes; and in a third case, extracting target point cloud data in a third target bounding box, re-clustering the target point cloud data to obtain a target bounding box set, and labeling an object corresponding to a first target bounding box in the target bounding box set that coincides with a detection frame of a point cloud data recognition device in the target vehicle as the first type, the third case including that the third target bounding box includes a plurality of detection frames.

9. A non-volatile storage medium, comprising: The non-volatile storage medium includes a stored program, wherein the program, when executed, controls a device in which the non-volatile storage medium is located to perform the point cloud data classification and labeling method of any one of claims 1 to 7.

10. A processor, comprising: The processor is configured to execute a program, wherein the program, when executed, performs the point cloud data classification and labeling method of any one of claims 1 to 7.

Images