A point cloud data processing method and device

Abstract

The application provides a point cloud data processing method and device. The method comprises the following steps: determining a characteristic parameter of an original point cloud, wherein the characteristic parameter comprises a first parameter for indicating a category feature of the original point cloud and / or a second parameter for indicating an environment feature of the original point cloud; determining a first target point cloud of background points in the original point cloud according to the characteristic parameter of the original point cloud; and removing the first target point cloud in the original point cloud to obtain a second target point cloud. The point cloud data is processed according to the point cloud data processing method provided by the application, the background points in the point cloud data can be removed, the interference caused by the background points on the subsequent identification and detection of the point cloud data is reduced, and then the accuracy and efficiency of the subsequent processing are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a point cloud data processing method and apparatus. Background Technology

[0002] Point cloud data is a collection of vectors in a three-dimensional coordinate system. These vectors are typically represented in three-dimensional coordinates and are generally used to represent the shape of an object's outer surface. Currently, most point cloud data can be obtained by scanning with 3D (3D) scanning equipment.

[0003] Point cloud data can generally be divided into foreground points and background points. Foreground points are points within various target objects, while background points are all points other than the target objects. Currently, when performing recognition and detection processing based on point cloud data (such as object detection), the presence of background points in the point cloud data can interfere with the recognition and detection process, leading to reduced accuracy and efficiency, and consuming significant computational resources. Summary of the Invention

[0004] This application provides a point cloud data processing method and apparatus to remove background points from point cloud data, reduce the interference of background points on the recognition and detection of point cloud data, and thereby improve the accuracy and efficiency of processing.

[0005] In a first aspect, this application provides a point cloud data processing method, the method comprising: determining feature parameters of an original point cloud, wherein the feature parameters include: a first parameter for indicating the category features of the original point cloud, and / or a second parameter for indicating the environmental features of the original point cloud; determining a first target point cloud belonging to background points in the original point cloud based on the feature parameters of the original point cloud; and removing the first target point cloud from the original point cloud to obtain a second target point cloud.

[0006] In this method, by distinguishing between foreground and background points in the point cloud based on the category features and environmental features of the point cloud, background points in the point cloud data can be removed. This reduces the interference of background point information in the point cloud data on the subsequent point cloud data processing, thereby improving the accuracy of the processing results. At the same time, it reduces the amount of data to be processed in the subsequent point cloud data, which can speed up the subsequent processing of point cloud data and reduce the consumption of computing resources.

[0007] In one possible design, the feature parameters of the original point cloud include the first parameter; the first parameter satisfies at least one of the following: the first parameter is obtained by classifying the original point cloud through a point cloud classification model; or, the first parameter is the category of a first target object corresponding to the original point cloud, wherein the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

[0008] This method utilizes a point cloud classification model to classify the original point cloud, enabling rapid and accurate determination of its category while maintaining high processing speed. Images, being a higher-resolution representation than point cloud data, facilitate the identification of category features and other information more easily and accurately, especially when the object to which the point cloud belongs contains a small number of points. Furthermore, this method provides multiple methods for determining the feature data of the point cloud, allowing for flexible adjustments based on specific needs.

[0009] In one possible design, the feature parameters of the original point cloud include the second parameter; the second parameter satisfies at least one of the following: the second parameter is the point cloud distribution characteristics within a first defined area where the original point cloud is located; or, the second parameter is the category of a second target object corresponding to the original point cloud, wherein the second target object is an object located at a target position in the target map data corresponding to the original point cloud, and the target position is determined according to the correspondence between the coordinate system where the original point cloud is located and the coordinate system where the target map data is located; or, the second parameter is the object distribution characteristics within a second defined area where the target position in the target map data corresponding to the original point cloud is located.

[0010] In this method, the environmental features of a point cloud reflect the distribution characteristics of the point cloud or the objects to which the point cloud belongs in its neighborhood. Since the features of a point cloud are related to its environmental features, considering these features during feature extraction can further improve the accuracy of the extracted features and ensure the accuracy of subsequent processing based on them. Besides point cloud data, map data can also reflect environmental information relatively accurately. Therefore, referencing point cloud data and map data to extract the environmental features of a point cloud ensures the accuracy and comprehensiveness of the extracted features. Furthermore, this method provides multiple ways to determine the feature data of a point cloud, which can be flexibly adjusted according to actual needs.

[0011] In one possible design, the point cloud distribution features include at least one of the following: the number of point clouds contained in the first defined area, the category of point clouds in the first defined area, the size of the area occupied by each category of point clouds in the first defined area, and the number of point clouds of each category in the first defined area; the object distribution features include at least one of the following: the number of objects in the second defined area, the category of objects in the second defined area, and the size of the area occupied by each object in the second defined area.

[0012] In this method, the distribution features of point clouds and object distribution features are characterized by parameters of various dimensions, which can ensure the comprehensiveness of the extracted environmental feature information, fully reflect the surrounding environmental features of the point cloud, and thus improve the accuracy of point cloud processing based on these features.

[0013] In one possible design, determining the first target point cloud belonging to background points in the original point cloud based on the feature parameters of the original point cloud includes: determining the first target point cloud belonging to background points in the original point cloud based on the feature parameters of the original point cloud and a point cloud analysis model, wherein the point cloud analysis model is used to determine whether the point cloud belongs to background points based on the feature parameters of the point cloud.

[0014] In this method, a point cloud analysis model is used to distinguish the categories of the original point cloud, which can quickly and accurately identify whether the point cloud belongs to the background points. While ensuring high processing speed, it can also ensure the accuracy of point cloud recognition.

[0015] In one possible design, the method further includes: determining a third parameter of the second target point cloud, the third parameter indicating the category of the second target point cloud; dividing the second target point cloud according to the third parameter to obtain at least one set of point clouds, wherein the third parameter of the point clouds in any set of point clouds is the same, and the distance between any two point clouds in any set of point clouds is less than a set threshold; determining a target anchor box for each set of point clouds in the at least one set of point clouds, wherein the target anchor box of any set of point clouds contains all the point clouds in the set of point clouds; determining a third target point cloud located in the target anchor box of the at least one set of point clouds in the first target point cloud; and adding the third target point cloud to the second target point cloud to obtain a modified second target point cloud.

[0016] In this method, the point cloud is divided according to its category. Then, based on the characteristics of the divided point cloud, some point clouds that may belong to the foreground points are selected from the removed background point cloud and added to the second target point cloud to obtain the processed point cloud. This method can avoid the loss of some foreground points when removing background points, retain foreground points as comprehensively as possible, remove background points, and improve the accuracy of point cloud data processing.

[0017] In one possible design, determining the target anchor frame for each point cloud set in the at least one point cloud set includes: determining the circumscribed anchor frame for each point cloud set in the at least one point cloud set, wherein the circumscribed anchor frame of any point cloud set is the smallest anchor frame containing all the point clouds in the point cloud set; for a target point cloud set in the at least one point cloud set, performing the following steps: determining whether the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to a set size; if the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to the set size, then using the circumscribed anchor frame as the target anchor frame of the target point cloud set; otherwise, generating the target anchor frame of the target point cloud set according to the set size; wherein the target point cloud set is any point cloud set in the at least one point cloud set.

[0018] In this method, the size of the target anchor box of the determined point cloud set must meet the requirements in order to further ensure that the point cloud selected from the removed background points based on the target anchor box of the point cloud set is more likely to belong to the foreground points, thereby ensuring that more foreground points and fewer background points can be selected from the removed point cloud.

[0019] In one possible design, generating the target anchor frame of the target point cloud set according to the set size includes: generating the target anchor frame of the target point cloud set with the target reference point as the center and according to the set size, wherein the target reference point is the center point of the outer anchor frame of the target point cloud set or the center point of multiple point clouds contained in the target point cloud set.

[0020] In this method, the target anchor frame of the point cloud set is generated by the center point of the outer anchor frame of the point cloud set or the center point of the point cloud in the point cloud set. This can ensure that the target anchor frame of the point cloud set surrounds the objects to which the point clouds in the point cloud set belong as much as possible, so that the final determined point cloud is as close as possible to the foreground points corresponding to each object.

[0021] Secondly, this application provides a point cloud data processing apparatus, which includes: a feature extraction unit for determining feature parameters of an original point cloud, wherein the feature parameters include: a first parameter for indicating the category features of the original point cloud, and / or a second parameter for indicating the environmental features of the original point cloud; a point cloud selection unit for determining a first target point cloud belonging to background points in the original point cloud based on the feature parameters of the original point cloud; and a point cloud processing unit for removing the first target point cloud from the original point cloud to obtain a second target point cloud.

[0022] In one possible design, the feature parameters of the original point cloud include the first parameter; the first parameter satisfies at least one of the following: the first parameter is obtained by classifying the original point cloud through a point cloud classification model; or, the first parameter is the category of a first target object corresponding to the original point cloud, wherein the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

[0023] In one possible design, the feature parameters of the original point cloud include the second parameter; the second parameter satisfies at least one of the following: the second parameter is the point cloud distribution characteristics within a first defined area where the original point cloud is located; or, the second parameter is the category of a second target object corresponding to the original point cloud, wherein the second target object is an object located at a target position in the target map data corresponding to the original point cloud, and the target position is determined according to the correspondence between the coordinate system where the original point cloud is located and the coordinate system where the target map data is located; or, the second parameter is the object distribution characteristics within a second defined area where the target position in the target map data corresponding to the original point cloud is located.

[0024] In one possible design, the point cloud distribution features include at least one of the following: the number of point clouds contained in the first defined area, the category of point clouds in the first defined area, the size of the area occupied by each category of point clouds in the first defined area, or the number of point clouds of each category in the first defined area; the object distribution features include at least one of the following: the number of objects in the second defined area, the category of objects in the second defined area, or the size of the area occupied by each object in the second defined area.

[0025] In one possible design, when the point cloud selection unit determines the first target point cloud belonging to the background point in the original point cloud based on the feature parameters of the original point cloud, it is specifically used to: determine the first target point cloud belonging to the background point in the original point cloud based on the feature parameters of the original point cloud and the point cloud analysis model, wherein the point cloud analysis model is used to determine whether the point cloud belongs to the background point based on the feature parameters of the point cloud.

[0026] In one possible design, the point cloud processing unit is further configured to: determine a third parameter of the second target point cloud, the third parameter indicating the category of the second target point cloud; divide the second target point cloud according to the third parameter to obtain at least one point cloud set, wherein the third parameter of the point clouds in any point cloud set is the same, and the distance between any two point clouds in any point cloud set is less than a set threshold; determine a target anchor frame for each point cloud set in the at least one point cloud set, wherein the target anchor frame of any point cloud set contains all the point clouds in the point cloud set; determine a third target point cloud located in the target anchor frame of the at least one point cloud set in the first target point cloud; and add the third target point cloud to the second target point cloud to obtain a corrected second target point cloud.

[0027] In one possible design, when the point cloud processing unit determines the target anchor frame for each point cloud set in the at least one point cloud set, it specifically performs the following steps: determining the circumscribed anchor frame for each point cloud set in the at least one point cloud set, wherein the circumscribed anchor frame of any point cloud set is the smallest anchor frame containing all the point clouds in the point cloud set; for a target point cloud set in the at least one point cloud set, performing the following steps: determining whether the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to a set size; if it is determined that the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to the set size, then using the circumscribed anchor frame as the target anchor frame of the target point cloud set; otherwise, generating the target anchor frame of the target point cloud set according to the set size; wherein the target point cloud set is any point cloud set in the at least one point cloud set.

[0028] In one possible design, when the point cloud processing unit generates the target anchor frame of the target point cloud set according to the set size, it is specifically used to: generate the target anchor frame of the target point cloud set with the target reference point as the center and according to the set size, wherein the target reference point is the center point of the outer anchor frame of the target point cloud set or the center point of multiple point clouds contained in the target point cloud set.

[0029] Thirdly, this application provides a point cloud data processing apparatus, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the calculation program stored in the memory to implement the method described in the first aspect or any possible design of the first aspect.

[0030] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a point cloud data processing device, causes the point cloud data processing device to perform the method described in the first aspect or any possible design of the first aspect.

[0031] Fifthly, this application provides a computer program product, which includes a computer program or instructions that, when executed by a point cloud data processing device, implement the method described in the first aspect or any possible design of the first aspect.

[0032] In a sixth aspect, this application provides a terminal that includes the point cloud data processing apparatus described in the second or third aspect above, or the terminal includes a memory and a processor; wherein the memory is used to store a computer program; and the processor is used to execute the computing program stored in the memory to implement the method described in the first aspect or any possible design of the first aspect above.

[0033] In one possible design, the terminal is any of the following: intelligent transportation equipment, smart home equipment, intelligent manufacturing equipment, or robot.

[0034] In one possible design, the intelligent transportation equipment is any of the following: a vehicle, a drone, an automated guided vehicle, or an unmanned transport vehicle.

[0035] In a seventh aspect, this application provides a server that includes the point cloud data processing apparatus described in the second or third aspect above, or the server includes a memory and a processor; wherein the memory is used to store a computer program; and the processor is used to execute the computational program stored in the memory to implement the method described in the first aspect or any possible design of the first aspect above.

[0036] In one possible design, the server is a cloud server or a local server.

[0037] For the beneficial effects described in aspects two through seven above, please refer to the description of the beneficial effects in aspect one above, which will not be repeated here. Attached Figure Description

[0038] Figure 1a A schematic diagram of a target detection method provided in an embodiment of this application;

[0039] Figure 1b A schematic diagram illustrating another target detection method provided in an embodiment of this application;

[0040] Figure 1cA schematic diagram illustrating another target detection method provided in an embodiment of this application;

[0041] Figure 2 This is a schematic diagram of a target detection result;

[0042] Figure 3a A schematic diagram of the architecture of a possible application system for the point cloud data processing method provided in the embodiments of this application;

[0043] Figure 3b A schematic diagram of the architecture of another possible application system for the point cloud data processing method provided in the embodiments of this application;

[0044] Figure 4 A schematic diagram illustrating a point cloud data processing method provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram illustrating a point cloud data processing method provided in an embodiment of this application;

[0046] Figure 6 A schematic diagram of an original point cloud provided for an embodiment of this application;

[0047] Figure 7 This is a schematic diagram illustrating a method for extracting feature data from point clouds, provided in an embodiment of this application.

[0048] Figure 8 A schematic diagram of a second target point cloud provided for an embodiment of this application;

[0049] Figure 9 A schematic diagram illustrating a point cloud data processing method provided in an embodiment of this application;

[0050] Figure 10 A schematic diagram of a modified second target point cloud provided for an embodiment of this application;

[0051] Figure 11 A schematic diagram of a point cloud data processing device provided in an embodiment of this application;

[0052] Figure 12 This is a schematic diagram of the structure of a point cloud data processing device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0054] For ease of understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0055] Point cloud data: A collection of point data on the surface of an object obtained through measurement equipment can be called point cloud data. Point cloud data can be represented as a set of vectors in a three-dimensional coordinate system. These vectors are usually represented in the form of three-dimensional coordinates and are generally used to represent the shape of an object's outer surface. Most point cloud data is obtained by scanning with three-dimensional (3D) scanning equipment, such as LiDAR.

[0056] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0057] Object detection refers to the process of locating multiple target objects from collected data. Point cloud data is widely used in object detection because it can effectively reflect the external surface characteristics of objects. Currently, point cloud-based object detection methods are mainly divided into the following two types:

[0058] Method 1: Target detection method based on pure point cloud data

[0059] like Figure 1aAs shown, current target detection based on pure point cloud data uses point cloud detection algorithms to directly detect targets in the collected point cloud data to obtain the target detection results. Since the data used for target detection is single-source data, this method suffers from low accuracy and poor robustness.

[0060] Method 2: Target detection method based on point cloud data and image data

[0061] like Figure 1b As shown, in one implementation scheme, when performing target detection based on point cloud data and image data, point cloud detection algorithms are first used to perform target detection on the acquired point cloud data to obtain point cloud-based target detection results, and image detection algorithms are used to perform target detection on the acquired images to obtain image-based target detection results. Then, the consistent result between the point cloud-based and image-based target detection results is selected as the final target detection result. This method requires separate detection processing of point cloud data and images, resulting in slow detection speed and high computational complexity. Furthermore, directly fusing the two detection results does not significantly improve detection accuracy.

[0062] like Figure 1c As shown, in another implementation scheme, when performing target detection based on point cloud data and image data, the acquired point cloud data can first be mapped onto the acquired image to obtain new point cloud data that fuses point cloud features and image features. Then, a point cloud detection algorithm is used to perform target detection on the new point cloud data to obtain the target detection result. This method also suffers from high computational complexity and slow detection speed.

[0063] In addition, among the methods mentioned above, point cloud data generally contains a lot of background points, which increases the difficulty of target regression and consumes a lot of computing resources, thus reducing the speed and accuracy of target detection.

[0064] For example, such as Figure 2 As shown, the target object to be detected is actually located in anchor frame 1. However, during the target detection process, the presence of background point P may cause the detected target object to be located in the area corresponding to anchor frame 2, resulting in a discrepancy between the detection result and the actual situation.

[0065] In view of this, embodiments of this application provide a point cloud data processing method for processing point cloud data. This method can remove background points from the point cloud data and retain foreground points, thereby accelerating the efficiency of subsequent processing of point cloud data (such as target detection), saving subsequent computing resources, and reducing the interference or adverse effects of background points on subsequent processing.

[0066] The point cloud data processing method provided in this application can be applied to a point cloud data processing device with data processing capabilities. This device can be a vehicle with data processing functionality, an in-vehicle device within a vehicle with data processing capabilities, or a sensor installed in a sensor capable of collecting and processing point cloud data. In-vehicle devices may include, but are not limited to, in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, in-vehicle units, in-vehicle radar, electronic control units (ECUs), domain controllers (DCs), etc. The data processing device can also be other electronic devices with data processing capabilities, including but not limited to smart home devices (e.g., televisions), smart robots, mobile terminals (e.g., mobile phones, tablets), industrial control computers, wearable devices, and other smart devices. The data processing device can also be a controller, processor (e.g., central processing unit (CPU), graphics processing unit (GPU), etc.), software modem (Mobile Daughter Card, MDC), chip, or other components within a smart device. The data processing device can also be a server, cloud server, etc.

[0067] The point cloud data processing method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described below are only some embodiments of this application, and not all embodiments.

[0068] Figure 3a This is a schematic diagram of the architecture of a possible application system for the point cloud data processing method provided in the embodiments of this application. For example... Figure 3a As shown, the system includes at least a point cloud data processing device, and may further include a lidar and / or a camera.

[0069] The LiDAR (Light Detection and Ranging) system can be used to acquire raw point cloud data and send it to a point cloud data processing device. A camera can be used to acquire images and send them to the point cloud data processing device. The point cloud data processing device can then combine the images acquired by the camera with the point cloud data acquired by the LiDAR to perform the point cloud data processing method provided in this application's embodiments. Alternatively, the point cloud data processing device can integrate the functions of both the LiDAR and the camera; that is, the device can both acquire raw point cloud data and images, and also process raw point cloud data and images.

[0070] In addition, the point cloud data processing device can acquire map data, for example, from a server (such as a local or cloud server) or other devices used for map creation. After acquiring the raw point cloud data and corresponding map data, images, and other data, the point cloud data processing device can execute the point cloud data processing method provided in the embodiments of this application on the acquired data.

[0071] Optionally, the system may also include a target detection device, which can perform target detection on the point cloud data processed by the point cloud data processing device to obtain target detection results.

[0072] It should be noted that, in Figure 3a In this application, LiDAR is merely an example of a device capable of collecting point cloud data. The device for collecting point cloud data in this embodiment is not limited to LiDAR, but can be any other device capable of collecting point cloud data. Similarly, a camera is merely an example of a device capable of capturing images. The device for capturing images in this embodiment is not limited to a camera, but can be any other device capable of capturing images, such as a mobile phone, a vehicle monitoring system, etc. Figure 3a This is just one example, and the application system of the method provided in this application embodiment is not limited to this.

[0073] Figure 3b This is a schematic diagram of the architecture of another possible application system for the point cloud data processing method provided in the embodiments of this application. For example... Figure 3b As shown, the system includes at least a vehicle.

[0074] For example, the vehicle can be a vehicle used in scenarios such as autonomous driving or assisted driving.

[0075] In one possible scenario, the vehicle may include at least one sensor or on-board device. The at least one sensor or on-board device on the vehicle can be used to collect or acquire point cloud data, images, map data, or other data awaiting processing, and can also be used to execute the point cloud data processing method provided in the following embodiments of this application on the acquired data to be processed.

[0076] For example, at least one sensor or onboard device on a vehicle may include a lidar, a camera, a wireless communication device, a processor, etc. The lidar can be used to collect point cloud data, the camera can be used to collect corresponding images, the wireless communication device can be used to receive map data from other devices, and the processor can be used to execute the point cloud data processing method provided in the following embodiments of this application based on the acquired data.

[0077] In another possible scenario, the system may also include a server. This server can be a local server or a cloud server. In this scenario, the vehicle may include at least one sensor or onboard device, which is used solely to collect or acquire point cloud data, images, map data, and other data awaiting processing, and upload this data to the server. The server is used to execute the point cloud data processing method provided in the following embodiments of this application on the received data to be processed, and to send the processing results to the vehicle.

[0078] Understandable, Figure 3b The system shown is only an example, and the application systems of the methods provided in this application embodiment are not limited to this.

[0079] It should be noted that the point cloud data processing method provided in this application embodiment can be applied to any field or scene that requires separating foreground and background points in point cloud data, or extracting foreground or background points, such as point cloud segmentation or object detection scenarios. The following description uses the application of the point cloud data processing method provided in this application embodiment to an object detection scenario as an example to illustrate the method provided in this application embodiment.

[0080] Figure 4 This is a schematic diagram illustrating a point cloud data processing method provided in an embodiment of this application. Figure 4 As shown in the embodiments of this application, the point cloud data processing method mainly includes two stages: point cloud fusion and segmentation, and intelligent point cloud augmentation. When this method is applied to a target detection scenario, it may also include a point cloud target detection stage.

[0081] In the point cloud fusion and segmentation stage, background points in the original point cloud data can be identified by combining the feature data of the point cloud, and the identified background points can be removed so that only foreground points are retained in the original point cloud data. In the point cloud target detection stage, target detection can be performed based on the retained foreground points, thereby reducing the amount of data processing and computational complexity of target detection and improving the speed of target detection.

[0082] During the point cloud intelligent expansion stage, some point clouds can be recovered from the background points removed in the point cloud fusion and cutting stage to compensate for the foreground points lost in the point cloud fusion and cutting stage, further ensuring the accuracy of point cloud cutting. This allows the target detection stage to perform target detection based on a relatively comprehensive and accurate foreground point cloud, thereby ensuring the accuracy of target detection.

[0083] In the target detection stage, the feature data of the point cloud determined in the point cloud fusion and segmentation stage can be combined with the processed point cloud data obtained in the intelligent point cloud augmentation stage to perform target detection processing, thereby obtaining the corresponding target detection results. Since background points are removed from the point cloud data used for target detection in this stage, unnecessary consumption of computing resources and interference from background points on target detection can be avoided, resulting in a significant improvement in both the speed and accuracy of target detection.

[0084] The following section will first provide a detailed explanation of the point cloud data processing methods corresponding to the point cloud fusion and segmentation stage.

[0085] For ease of explanation, the following description uses the point cloud data processing method provided in the embodiments of this application, executed by a data processing device, as an example. The data processing device may be, but is not limited to, the device with data processing capabilities provided in the embodiments of this application.

[0086] Figure 5 This is a schematic diagram illustrating a point cloud data processing method provided in an embodiment of this application. Figure 5 As shown, the method includes:

[0087] S501: The point cloud data processing device determines the feature parameters of the original point cloud, wherein the feature parameters include: a first parameter for indicating the category characteristics of the original point cloud, and / or a second parameter for indicating the environmental characteristics of the original point cloud.

[0088] In this embodiment of the application, the raw point cloud data acquired by the point cloud data processing device includes multiple raw point clouds. After acquiring the raw point cloud data, the point cloud data processing device extracts feature data for each raw point cloud in the raw point cloud data, and then selects or classifies the foreground and background points of the raw point cloud data based on the extracted feature data.

[0089] Figure 6 This is a schematic diagram of a raw point cloud provided for an embodiment of this application. For example, as shown... Figure 6 As shown, the point clouds corresponding to different objects (such as vehicles, pedestrians, etc.) in the original point cloud are foreground points, and the point clouds other than the foreground points are background points.

[0090] like Figure 7 As shown in the embodiments of this application, when the point cloud data processing device determines the feature parameters of the original point cloud, it can adopt at least one of the following methods:

[0091] Method 1: Point cloud feature extraction based on point cloud classification

[0092] In this method, the feature parameters of the original point cloud include a first parameter indicating the category features of the original point cloud, and / or a second parameter indicating the environmental features of the original point cloud.

[0093] The first parameter of the original point cloud is obtained by classifying the original point cloud using a set point cloud classification model, and the second parameter of the original point cloud is the point cloud distribution characteristics within the first set area where the original point cloud is located.

[0094] For example, after acquiring raw point cloud data, the point cloud data processing device can use a pre-defined point cloud classification model to classify multiple raw point clouds in the raw point cloud data. The category corresponding to each raw point cloud is obtained after classification and used as the first parameter of each raw point cloud. The pre-defined point cloud classification model can be a network model obtained after training the model using deep learning, neural networks, or other methods. The category corresponding to the raw point cloud can be used to represent the category of the object to which the raw point cloud belongs.

[0095] For example, if the original point cloud data contains point clouds corresponding to vehicles and pedestrians, and a point cloud is located within the area where the point cloud corresponding to a vehicle is located, then the category of the point cloud can be determined as a vehicle through the classification of the network model.

[0096] After determining the category corresponding to each original point cloud, the aforementioned point cloud data processing device delineates a first defined region of a predetermined size centered on each original point cloud. Then, combining this with the distribution of point clouds within the first defined region, it determines the point cloud distribution characteristics within the first defined region and uses these characteristics as a second parameter corresponding to the original point cloud. Specifically, when determining the point cloud distribution characteristics within the first defined region, the point cloud data processing device can statistically analyze at least one of the following parameters related to the point cloud distribution within the first defined region: the number of point clouds contained within the first defined region, the category of each point cloud within the first defined region, the size of the area occupied by each category of point clouds within the first defined region, or the number of each category of point clouds within the first defined region.

[0097] In this method, the original point cloud is classified by setting a point cloud classification model to determine the feature data of the point cloud. This ensures both high processing speed and accuracy of point cloud classification.

[0098] Method 2: Point cloud feature extraction based on image detection

[0099] In this approach, the feature parameters of the original point cloud include a first parameter used to indicate the category features of the original point cloud.

[0100] The first parameter of the original point cloud is the category of the first target object corresponding to the original point cloud, and the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

[0101] For example, after acquiring the raw point cloud data, the point cloud data processing device acquires the image corresponding to the raw point cloud data and uses it as the target image. The target image corresponds to the same actual scene environment as the raw point cloud data. For instance, when performing target detection on a specific environmental area, point cloud data of that area and images of that area can be acquired simultaneously.

[0102] After acquiring the target image, the point cloud data processing device determines the first target object corresponding to the original point cloud in the target image. Specifically, the point cloud data processing device can project the original point cloud onto the target image to obtain the feature points corresponding to the original point cloud in the target image. Among multiple objects identified from the target image, the object closest to the feature points corresponding to the original point cloud is determined as the first target object. Then, the category of the determined first target object is used as the first parameter of the original point cloud.

[0103] In this application embodiment, the method used by the point cloud data processing device to identify multiple objects from the target image and to identify the category of each object is not specifically limited. For example, a network model or neural network model trained based on deep learning can be used for corresponding identification.

[0104] Method 3: Point cloud feature extraction based on map detection

[0105] In this method, the feature parameters of the original point cloud include a second parameter used to indicate the environmental characteristics of the original point cloud.

[0106] The second parameter of the original point cloud is the category of the second target object corresponding to the original point cloud, and / or the object distribution characteristics within a second defined area where the target location in the target map data corresponding to the original point cloud is located. The second target object is either the object located at the target location in the target map data corresponding to the original point cloud, or the object closest to that target location. The target location is determined based on the correspondence between the coordinate system of the original point cloud and the coordinate system of the target map data.

[0107] Specifically, after acquiring the raw point cloud data, the point cloud data processing device acquires the corresponding map data and uses it as the target map data. This target map data corresponds to the same actual scene environment as the raw point cloud data. For example, when performing target detection on a specific environmental area, both the point cloud data and the map data for that area can be collected simultaneously.

[0108] Optionally, the target map data can be a high-definition map. In a high-definition map, the absolute geographic coordinates, physical dimensions, and other characteristics of traffic participants (such as lane lines, signs, traffic lights, etc.) are all reflected, especially the features of static objects. Therefore, combining high-definition maps with point cloud feature extraction can further improve the accuracy of feature extraction.

[0109] After acquiring the target map data, the point cloud data processing device determines the second target object corresponding to the original point cloud within the target map data, and uses the category of the determined second target object as the second parameter of the original point cloud. Specifically, the point cloud data processing device can determine the target position corresponding to the original point cloud after transformation to the coordinate system of the map data based on the correspondence between the coordinate system of the point cloud data and the coordinate system of the map data. Then, among the objects contained in the map data, it determines the object closest to the target position as the second target object, and determines the category of the second target object as the second parameter of the original point cloud.

[0110] After determining the target location in the target map data corresponding to the original point cloud, the point cloud data processing device can delineate a second defined region of a predetermined size centered on the target location. Then, combining this second defined region with the distribution of objects within it, the device determines the object distribution characteristics within the second defined region and uses these characteristics as the second parameter corresponding to the original point cloud. Specifically, when determining the object distribution characteristics within the second defined region, the point cloud data processing device can statistically analyze at least one of the following parameters related to object distribution within the second defined region: the number of objects within the second defined region, the category of objects within the second defined region, or the size of the area occupied by each object within the second defined region.

[0111] S502: The point cloud data processing device determines the first target point cloud that belongs to the background point in the original point cloud based on the feature parameters of the original point cloud.

[0112] After the point cloud data processing device obtains the feature parameters of the original point cloud in the original point cloud data, it can determine the first target point cloud that belongs to the background point in the original point cloud according to the feature parameters of the original point cloud and the point cloud analysis model. The point cloud analysis model is used to determine whether the point cloud belongs to the background point according to the feature parameters of the point cloud.

[0113] The point cloud analysis model described above can be obtained by training a network model, such as a multi-layer perceptron (MLP), support vector machine (SVM), visual geometry group 16 (VGG 16), or residual network 32 (ResNet32).

[0114] Specifically, in Method 1 above, the first parameter of the original point cloud determined by the point cloud feature extraction method based on point cloud classification is defined as the point cloud category `class_pitclound`, and the second parameter is defined as the point cloud statistical category `class_statistics_pitclound`. In Method 2 above, the first parameter of the original point cloud determined by the point cloud feature extraction method based on image detection is defined as the image category `class_camera`. In Method 3 above, the first parameter of the original point cloud determined by the point cloud feature extraction method based on map detection is defined as the map category `class_map`, and the second parameter is defined as the map statistical category `class_statistics_map`.

[0115] During network model training, sample point cloud data with at least one of the following parameters is first acquired: `class_pitclound`, `class_statistics_pitclound`, `class_camera`, `class_map`, and `class_statistics_map`. The labels of the points in the sample point cloud data are then determined, marking whether a point belongs to the foreground or background. Next, the classification network is trained using the acquired sample point cloud data and its corresponding labels to obtain a point cloud analysis model capable of determining whether a point is foreground or background. The point cloud labels can be generated through 3D detection. For example, 3D detection can be performed on the point cloud data to determine at least one 3D detection bounding box. For each point in the point cloud data, if it is determined that the point is inside any 3D detection bounding box, it is determined to be a foreground point and its label is set to 1; otherwise, it is determined to be a background point and its label is set to 0. Correspondingly, the trained point cloud analysis model can output a label value of 1 to indicate that the point cloud belongs to the foreground point when it determines that the point cloud belongs to the foreground point based on the feature data of the point cloud, and can output a label value of 0 to indicate that the point cloud belongs to the background point when it determines that the point cloud belongs to the background point.

[0116] In this embodiment of the application, after the point cloud data processing device acquires the trained point cloud analysis model, it inputs the feature parameters of the original point cloud determined above into the point cloud analysis model to obtain the judgment result of whether the original point cloud belongs to the background point output by the point cloud analysis model.

[0117] For example, such as Figure 7 As shown, the point cloud data processing device can input the first and second parameters of the original point cloud, determined using the three methods described above, into the point cloud analysis model for point cloud analysis. If the label value of a certain point cloud output by the point cloud analysis model is 1, it can be determined that the original point cloud belongs to the foreground point; if the label value of a certain point cloud output by the point cloud analysis model is 0, it can be determined that the original point cloud belongs to the background point.

[0118] S503: The point cloud data processing device removes the first target point cloud from the original point cloud to obtain the second target point cloud.

[0119] After the point cloud data processing device determines the first target point cloud belonging to the background points from the original point cloud, it can distinguish between foreground and background points in the original point cloud. For example, it can remove the determined first target point cloud from the original point cloud to obtain the second target point cloud.

[0120] In some embodiments of this application, after obtaining the second target point cloud, the point cloud data processing device can perform target detection based on the second target point cloud.

[0121] Figure 8 This is a schematic diagram of a second target point cloud provided in an embodiment of this application. Figure 8 As shown, for Figure 6 After the above processing, the original point cloud shown is basically removed, and the remaining point cloud is mainly composed of foreground points.

[0122] In the above embodiments, during point cloud data processing, the point cloud detection device can distinguish between foreground and background points in the original point cloud based on its feature data. Furthermore, it can remove background points from the original point cloud. On one hand, when performing target detection based on the processed point cloud, it can reduce the interference caused by background points in the point cloud data, thereby improving the accuracy of target detection. On the other hand, removing background points from the point cloud reduces the amount of point cloud data processed during target detection, further accelerating the processing speed and reducing the consumption of computing resources.

[0123] The following section provides a detailed explanation of the point cloud data processing methods corresponding to the aforementioned intelligent point cloud expansion stage.

[0124] like Figure 9 As shown, the point cloud data processing method provided in this application also includes:

[0125] S901: The point cloud data processing device determines a third parameter of the second target point cloud, the third parameter being used to indicate the category of the second target point cloud.

[0126] In some embodiments of this application, after the point cloud data processing device obtains the second target point cloud by executing the method described in steps S501 to S503 above, it can continue to execute the method described in steps S901 to S905 to further process the second target point cloud. A detailed description follows.

[0127] After obtaining the second target point cloud, the point cloud data processing device determines a third parameter to indicate the category of the second target point cloud.

[0128] As an optional implementation, the second target point cloud is the point cloud that has not been removed from the original point cloud, and the first parameter of the original point cloud determined in step S501 can be used to characterize the category features of the point cloud. Therefore, for each point cloud in the second target point cloud, the point cloud data processing device can directly use the first parameter of the point cloud obtained by any of the above methods 1 to 3 as the third parameter of the point cloud, thereby speeding up the point cloud data processing speed.

[0129] As an alternative implementation, the point cloud data processing device can use a trained point cloud category determination model to re-identify the category of the obtained second target point cloud, and use the identified category of the second target point cloud as a third parameter of the second target point cloud. In this method, based on the second target point cloud obtained after removing background points, the category of each point cloud is re-determined, avoiding the influence that the originally existing background points may have on the point cloud category identification, thereby improving the accuracy of point cloud category identification.

[0130] In some embodiments of this application, the point cloud category determination model described above can also be integrated with the aforementioned point cloud analysis model into a data processing model. In this case, when the original point cloud is determined to belong to a foreground point (or not to a background point), the data processing model can further identify and predict the category of the original point cloud, obtain the third parameter of the original point cloud, and use it when processing the second target point cloud.

[0131] S902: The point cloud data processing device divides the second target point cloud according to the third parameter of the second target point cloud to obtain at least one point cloud set.

[0132] Among them, the third parameter of the point clouds in any point cloud set is the same, and the distance between any two point clouds in any point cloud set is less than a set threshold.

[0133] After determining the third parameter of the second target point cloud, the point cloud data processing device can divide the second target point cloud according to the determined third parameter. Specifically, during the division, the point cloud data processing device divides a group of point clouds from the second target point cloud whose inter-point distance is less than a set threshold and which belong to the same category into a point cloud set, thereby dividing the second target point cloud into at least one point cloud set. After the division, the point clouds in each point cloud set belong to the same object, and the point clouds in different point cloud sets belong to different objects.

[0134] S903: The point cloud data processing device determines the target anchor frame of each point cloud set in the at least one point cloud set, wherein the target anchor frame of any point cloud set contains all the point clouds in the point cloud set.

[0135] For each point cloud set, the point cloud data processing device fits a minimum anchor frame containing all points in that set, which is then used as the circumscribed anchor frame of that point cloud set. Then, for each point cloud set, it determines whether the size of the circumscribed anchor frame is greater than or equal to a preset size. If so, the circumscribed anchor frame of that point cloud set is used as the target anchor frame; otherwise, a target anchor frame for the target point cloud set is generated, centered on a target reference point and according to the preset size. The target reference point is either the center point of the circumscribed anchor frame or the center point of any of the point clouds contained in the set. The center point of the circumscribed anchor frame can be its geometric center, and the center points of any of the point clouds in the set can be the points corresponding to the average coordinates of the point clouds.

[0136] In some embodiments of this application, the aforementioned set size can also be flexibly set. For example, a corresponding set size can be set for each object category. Then, when processing the size of the outer anchor frame of the point cloud set, the size of the outer anchor frame of the point cloud set can be compared with the set size corresponding to the object category of the object to which the point cloud belongs, and subsequent processing can be performed. The set size corresponding to each object category can be set according to the actual size of the object corresponding to that object category.

[0137] S904: The point cloud data processing device determines a third target point cloud located in the target anchor frame of the at least one point cloud set in the first target point cloud.

[0138] S905: The point cloud data processing device adds the third target point cloud to the second target point cloud to obtain the corrected second target point cloud.

[0139] In steps S904 and S905 above, after the point cloud data processing device determines the target anchor frame for each point cloud set, it selects the third target point cloud located within each target anchor frame from the first target point cloud belonging to the background points. Then, the selected third target point cloud is added to the second target point cloud to obtain the corrected second target point cloud.

[0140] The point cloud data processing device can perform target detection based on the corrected second target point cloud after obtaining the corrected second target point cloud.

[0141] In the point cloud data processing method corresponding to the aforementioned point cloud fusion and segmentation stage, when removing the first target point cloud from the original point cloud to obtain the second target point cloud, there may be a loss of a few foreground points. For example, compared to... Figure 6 The original point cloud shown and Figure 8 As shown in the second target point cloud, the removed point cloud may contain a small number of foreground points, resulting in incomplete preservation of foreground points. Therefore, in this method, by selecting a third target point cloud that is close to the point cloud identified as foreground points and adding it to the second target point cloud, some point clouds with a higher probability of belonging to the foreground can be restored and preserved, thereby avoiding the problem of erroneous deletion of foreground points and improving the accuracy of point cloud processing. For example, as... Figure 10 The corrected second target point cloud shown is closer to the foreground points in the original point cloud data, thus the accuracy of removing background points and retaining foreground points is higher.

[0142] In the above embodiments, based on the category features and environmental features of the original point cloud, it is possible to determine whether a point cloud is more likely to be a foreground or background point. This allows for the differentiation between foreground and background points in the original point cloud. By removing background points and retaining foreground points with useful information, the processed point cloud can be used for target detection, thus accelerating point cloud-based target detection algorithms, improving target detection efficiency, and saving computational resources. Furthermore, after removing background points, points that may belong to the foreground can be selected from the removed background points based on relevant features. This avoids the loss of some foreground points during background point removal and ensures the accuracy of target detection based on the retained foreground points. In summary, using the point cloud data processing method provided in the above embodiments to process the original point cloud and then performing target detection based on the processed point cloud can achieve better detection results for target objects in complex scenes.

[0143] Based on the above embodiments and the same concept, this application also provides a point cloud data processing device, such as... Figure 11As shown, the point cloud data processing device 1100 may include: a feature extraction unit 1101, a point cloud selection unit 1102, and a point cloud processing unit 1103.

[0144] The feature extraction unit 1101 is used to determine the feature parameters of the original point cloud, wherein the feature parameters include: a first parameter for indicating the category features of the original point cloud, and / or a second parameter for indicating the environmental features of the original point cloud; the point cloud selection unit 1102 is used to determine a first target point cloud belonging to the background points in the original point cloud according to the feature parameters of the original point cloud; the point cloud processing unit 1103 is used to remove the first target point cloud from the original point cloud to obtain a second target point cloud.

[0145] In one possible design, the feature parameters of the original point cloud include the first parameter; the first parameter satisfies at least one of the following: the first parameter is obtained by classifying the original point cloud through a point cloud classification model; or, the first parameter is the category of a first target object corresponding to the original point cloud, wherein the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

[0146] In one possible design, the feature parameters of the original point cloud include the second parameter; the second parameter satisfies at least one of the following: the second parameter is the point cloud distribution characteristics within a first defined area where the original point cloud is located; or, the second parameter is the category of a second target object corresponding to the original point cloud, wherein the second target object is an object located at a target position in the target map data corresponding to the original point cloud, and the target position is determined according to the correspondence between the coordinate system where the original point cloud is located and the coordinate system where the target map data is located; or, the second parameter is the object distribution characteristics within a second defined area where the target position in the target map data corresponding to the original point cloud is located.

[0147] In one possible design, the point cloud distribution features include at least one of the following: the number of point clouds contained in the first defined area, the category of point clouds in the first defined area, the size of the area occupied by each category of point clouds in the first defined area, or the number of point clouds of each category in the first defined area; the object distribution features include at least one of the following: the number of objects in the second defined area, the category of objects in the second defined area, or the size of the area occupied by each object in the second defined area.

[0148] In one possible design, when the point cloud selection unit 1102 determines the first target point cloud belonging to the background point in the original point cloud based on the feature parameters of the original point cloud, it is specifically used to: determine the first target point cloud belonging to the background point in the original point cloud based on the feature parameters of the original point cloud and the point cloud analysis model, wherein the point cloud analysis model is used to determine whether the point cloud belongs to the background point based on the feature parameters of the point cloud.

[0149] In one possible design, the point cloud processing unit 1103 is further configured to: determine a third parameter of the second target point cloud, the third parameter indicating the category of the second target point cloud; divide the second target point cloud according to the third parameter to obtain at least one point cloud set, wherein the third parameter of the point clouds in any point cloud set is the same, and the distance between any two point clouds in any point cloud set is less than a set threshold; determine a target anchor frame for each point cloud set in the at least one point cloud set, wherein the target anchor frame of any point cloud set contains all the point clouds in the point cloud set; determine a third target point cloud located in the target anchor frame of the at least one point cloud set in the first target point cloud; and add the third target point cloud to the second target point cloud to obtain a corrected second target point cloud.

[0150] In one possible design, when the point cloud processing unit 1103 determines the target anchor frame for each point cloud set in the at least one point cloud set, it is specifically configured to: determine the circumscribed anchor frame for each point cloud set in the at least one point cloud set, wherein the circumscribed anchor frame for any point cloud set is the smallest anchor frame containing all the point clouds in the point cloud set; for a target point cloud set in the at least one point cloud set, perform the following steps: determine whether the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to a set size; if it is determined that the size of the circumscribed anchor frame of the target point cloud set is greater than or equal to the set size, then use the circumscribed anchor frame as the target anchor frame for the target point cloud set; otherwise, generate the target anchor frame for the target point cloud set according to the set size; wherein the target point cloud set is any point cloud set in the at least one point cloud set.

[0151] In one possible design, when the point cloud processing unit 1103 generates the target anchor frame of the target point cloud set according to the set size, it is specifically used to: generate the target anchor frame of the target point cloud set with the target reference point as the center and according to the set size, wherein the target reference point is the center point of the outer anchor frame of the target point cloud set or the center point of multiple point clouds contained in the target point cloud set.

[0152] The unit division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into a single processor, exist as separate physical units, or be integrated into a single unit. The integrated units described above can be implemented in hardware or as software functional units.

[0153] Figure 11 One or more of the various units within can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0154] Based on the above embodiments and the same concept, this application also provides a point cloud data processing apparatus for implementing the point cloud data processing method provided in this application. Figure 12 As shown, the point cloud data processing device 1200 may include one or more processors 1201, a memory 1202, and one or more computer programs (not shown in the figure). In one implementation, the above-mentioned devices can be coupled via one or more communication lines 1203. The memory 1202 stores one or more computer programs, which include instructions; the processor 1201 calls the instructions stored in the memory 1202, causing the point cloud data processing device 1200 to execute the point cloud data processing method provided in this application embodiment.

[0155] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0156] In the embodiments of this application, the memory can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. The memory in the embodiments of this application may also be circuitry or any other means capable of implementing storage functions.

[0157] As one implementation, the point cloud data processing device 1200 may further include a communication interface 1204 for communicating with other devices via a transmission medium. For example, when the device collecting raw point cloud data, images, map data, etc., is not the point cloud data processing device 1200, the point cloud data processing device 1200 can communicate with the device collecting raw point cloud data, images, map data, etc., through the communication interface 1204, thereby receiving the raw point cloud data, images, map data, etc., collected by that device. In this embodiment, the communication interface may be a transceiver, circuit, bus, module, or other types of communication interface. In this embodiment, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter, or a transceiver with integrated transceiver functions, or an interface circuit.

[0158] In some embodiments of this application, the processor 1201, memory 1202, and communication interface 1204 can be interconnected via a communication line 1203. The communication line 1203 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication line 1203 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0159] Based on the above embodiments and the same concept, this application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a point cloud data processing device, the point cloud data processing device executes the point cloud data processing method provided in the above embodiments.

[0160] Based on the above embodiments and the same concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a point cloud data processing device, the point cloud data processing method provided in the above embodiments is implemented.

[0161] Based on the above embodiments and the same concept, this application also provides a terminal, which includes the point cloud data processing device described in the above embodiments, or the terminal includes a memory and a processor; wherein, the memory is used to store a computer program; and the processor is used to execute the calculation program stored in the memory to implement the point cloud data processing method provided in the above embodiments.

[0162] In one possible design, the terminal is any of the following: intelligent transportation equipment, smart home equipment, intelligent manufacturing equipment, or robot.

[0163] In one possible design, the intelligent transportation equipment is any of the following: a vehicle, a drone, an automated guided vehicle, or an unmanned transport vehicle.

[0164] Based on the above embodiments and the same concept, this application also provides a server, which includes the point cloud data processing device described in the above embodiments, or the server includes a memory and a processor; wherein, the memory is used to store a computer program; and the processor is used to execute the calculation program stored in the memory to implement the point cloud data processing method provided in the above embodiments.

[0165] In one possible design, the server is a cloud server or a local server.

[0166] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD), or semiconductor media (e.g., SSD), etc.

[0167] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A point cloud data processing method, characterized in that, include: Determine the feature parameters of the original point cloud, wherein the feature parameters include: a first parameter for indicating the category features of the original point cloud, and / or a second parameter for indicating the environmental features of the original point cloud; Based on the feature parameters of the original point cloud, determine the first target point cloud that belongs to the background point in the original point cloud; Remove the first target point cloud from the original point cloud to obtain the second target point cloud; A third parameter is determined for the second target point cloud, wherein the third parameter is used to indicate the category of the object corresponding to the second target point cloud; Based on the third parameter of the second target point cloud, the second target point cloud is divided to obtain at least one point cloud set, wherein the third parameter of the point clouds in any point cloud set is the same, and the distance between any two point clouds in any point cloud set is less than a set threshold. Determine the target anchor box for each point cloud set in the at least one point cloud set; In the first target point cloud, a third target point cloud is determined to be located within the target anchor frame of the at least one point cloud set; The third target point cloud is added to the second target point cloud to correct the second target point cloud; Wherein, determining the target anchor box for each point cloud set in the at least one point cloud set includes: Determine the bounding anchor frame of each point cloud set in the at least one point cloud set, wherein the bounding anchor frame of any point cloud set is the smallest anchor frame that contains all the point clouds in the point cloud set; For the target point cloud set in the at least one point cloud set, perform the following steps: If it is determined that the size of the outer anchor frame of the target point cloud set is greater than or equal to the set size, then the outer anchor frame is used as the target anchor frame of the target point cloud set. If it is determined that the size of the outer anchor frame of the target point cloud set is smaller than the set size, then the target anchor frame of the target point cloud set is generated with the target reference point as the center and according to the set size, wherein the target reference point is the center point of the outer anchor frame of the target point cloud set or the center point of multiple point clouds contained in the target point cloud set; Wherein, the target point cloud set is any one of the at least one point cloud set.

2. The method according to claim 1, characterized in that, The feature parameters of the original point cloud include the first parameter; the first parameter satisfies at least one of the following: The first parameter is obtained by classifying the original point cloud using a point cloud classification model; or The first parameter is the category of the first target object corresponding to the original point cloud, wherein the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

3. The method according to claim 1 or 2, characterized in that, The feature parameters of the original point cloud include the second parameter; the second parameter satisfies at least one of the following: The second parameter is the point cloud distribution characteristics within the first defined region where the original point cloud is located; or The second parameter is the category of the second target object corresponding to the original point cloud, wherein the second target object is an object located at a target position in the target map data corresponding to the original point cloud, and the target position is determined according to the correspondence between the coordinate system of the original point cloud and the coordinate system of the target map data; or The second parameter is the object distribution characteristics within a second defined area where the target location in the target map data corresponding to the original point cloud is located.

4. The method according to claim 3, characterized in that, The point cloud distribution features include at least one of the following: the number of point clouds contained in the first set area, the category of point clouds in the first set area, the size of the area occupied by each category of point clouds in the first set area, or the number of point clouds of each category in the first set area. The object distribution characteristics include at least one of the following: the number of objects in the second set area, the category of objects in the second set area, or the size of the area occupied by each object in the second set area.

5. The method according to claim 1 or 2, characterized in that, The step of determining the first target point cloud belonging to the background points in the original point cloud based on the feature parameters of the original point cloud includes: Based on the feature parameters of the original point cloud and the point cloud analysis model, a first target point cloud belonging to the background point in the original point cloud is determined, wherein the point cloud analysis model is used to determine whether the point cloud belongs to the background point based on the feature parameters of the point cloud.

6. The method according to claim 1 or 2, characterized in that, The set size is the set size corresponding to the object category to which the point cloud in the target point cloud set belongs; The set size for each object category is determined based on the actual size of the object corresponding to each object category.

7. A point cloud data processing device, characterized in that, include: A feature extraction unit is used to determine feature parameters of the original point cloud, wherein the feature parameters include: a first parameter for indicating the category features of the original point cloud, and / or a second parameter for indicating the environmental features of the original point cloud; A point cloud selection unit is used to determine a first target point cloud that belongs to the background point in the original point cloud based on the feature parameters of the original point cloud. A point cloud processing unit is configured to: remove the first target point cloud from the original point cloud to obtain a second target point cloud; determine a third parameter of the second target point cloud, the third parameter indicating the category of the object corresponding to the second target point cloud; divide the second target point cloud according to the third parameter to obtain at least one point cloud set, wherein the third parameter of the points in any point cloud set is the same, and the distance between any two points in any point cloud set is less than a set threshold; determine the target anchor box of each point cloud set in the at least one point cloud set; determine a third target point cloud located in the target anchor box of the at least one point cloud set in the first target point cloud; and add the third target point cloud to the second target point cloud to correct the second target point cloud. Specifically, when the point cloud processing unit determines the target anchor frame for each point cloud set in the at least one point cloud set, it is configured to: determine the circumscribed anchor frame for each point cloud set in the at least one point cloud set, wherein the circumscribed anchor frame of any point cloud set is the smallest anchor frame containing all the point clouds in the point cloud set; and for the target point cloud set in the at least one point cloud set, perform the following steps: If it is determined that the size of the outer anchor frame of the target point cloud set is greater than or equal to the set size, then the outer anchor frame is used as the target anchor frame of the target point cloud set. If it is determined that the size of the outer anchor frame of the target point cloud set is smaller than the set size, then the target anchor frame of the target point cloud set is generated with the target reference point as the center and according to the set size, wherein the target reference point is the center point of the outer anchor frame of the target point cloud set or the center point of multiple point clouds contained in the target point cloud set; Wherein, the target point cloud set is any one of the at least one point cloud set.

8. The apparatus according to claim 7, characterized in that, The feature parameters of the original point cloud include the first parameter; the first parameter satisfies at least one of the following: The first parameter is obtained by classifying the original point cloud using a point cloud classification model; or The first parameter is the category of the first target object corresponding to the original point cloud, wherein the first target object is the object to which the feature points of the original point cloud are projected onto the target image belong.

9. The apparatus according to claim 7 or 8, characterized in that, The feature parameters of the original point cloud include the second parameter; the second parameter satisfies at least one of the following: The second parameter is the point cloud distribution characteristics within the first defined region where the original point cloud is located; or The second parameter is the category of the second target object corresponding to the original point cloud, wherein the second target object is an object located at a target position in the target map data corresponding to the original point cloud, and the target position is determined according to the correspondence between the coordinate system of the original point cloud and the coordinate system of the target map data; or The second parameter is the object distribution characteristics within a second defined area where the target location in the target map data corresponding to the original point cloud is located.

10. The apparatus according to claim 9, characterized in that, The point cloud distribution features include at least one of the following: the number of point clouds contained in the first set area, the category of point clouds in the first set area, the size of the area occupied by each category of point clouds in the first set area, or the number of point clouds of each category in the first set area. The object distribution characteristics include at least one of the following: the number of objects in the second set area, the category of objects in the second set area, or the size of the area occupied by each object in the second set area.

11. The apparatus according to claim 7 or 8, characterized in that, When the point cloud selection unit determines the first target point cloud belonging to the background point in the original point cloud based on the feature parameters of the original point cloud, it is specifically used for: Based on the feature parameters of the original point cloud and the point cloud analysis model, a first target point cloud belonging to the background point in the original point cloud is determined, wherein the point cloud analysis model is used to determine whether the point cloud belongs to the background point based on the feature parameters of the point cloud.

12. The apparatus according to claim 7 or 8, characterized in that, The set size is the set size corresponding to the object category to which the point cloud in the target point cloud set belongs; The set size for each object category is determined based on the actual size of the object corresponding to each object category.

13. A point cloud data processing device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is used to execute the calculation program stored in the memory to implement the method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on the point cloud data processing device, causes the point cloud data processing device to perform the method as described in any one of claims 1 to 6.

15. A terminal, characterized in that, The terminal includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the calculation program stored in the memory to implement the method as described in any one of claims 1 to 6.

16. A server, characterized in that, The server includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the calculation program stored in the memory to implement the method as described in any one of claims 1 to 6.

Images