Point cloud data processing method and device

Abstract

The present application provides a point cloud data processing method and device, the method comprising: obtaining initial point cloud data and determining a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data; performing grid division on the target area to obtain multiple grids, wherein the feature data of the point cloud in each grid satisfies a feature distribution condition, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighboring points of the point cloud; selecting the target point cloud of each grid from the point cloud contained in each grid, and deleting the other point clouds in each grid except the target point cloud; and obtaining target point cloud data based on the target point cloud of each grid. The point cloud data processing method provided in the present application first performs grid division on the point cloud data based on the geometric distribution characteristics of the point cloud, and then uses the target point cloud of the grid as the representative point of the grid. This can reduce the damage to the shape features of the point cloud when compressing the point cloud, thereby improving the compression effect of the point cloud data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a point cloud data processing method and device. Background Art

[0002] Three-dimensional (3D) point cloud data is crucial for various scenarios, including autonomous driving and high-precision map creation. Currently, 3D point cloud data is typically collected using high-precision lidar. A single frame of 3D point cloud data is approximately 1.8 megabytes (MB), and collecting approximately ten minutes of 3D point cloud data results in a 10-gigabyte (GB) file. Consequently, existing methods collect a large amount of 3D point cloud data, requiring significant memory and disk space. Furthermore, the amount of computation required for real-time playback of the 3D point cloud data is substantial. Consequently, compression of the collected 3D point cloud data is necessary.

[0003] Currently, the most commonly used 3D point cloud compression method is the proportional voxel filtering method. This method reduces the number of points in the collected 3D point cloud data by filtering it. However, this method significantly damages the point cloud structure, and the shape of the compressed point cloud data deviates significantly from the original point cloud data, resulting in poor point cloud compression results. Summary of the Invention

[0004] The present application provides a point cloud data processing method and device for reducing the damage to the shape features of point cloud data and improving the point cloud compression effect when performing point cloud compression.

[0005] In a first aspect, the present application provides a point cloud data processing method, the method comprising: obtaining initial point cloud data and determining a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data; performing grid division on the target area to obtain multiple grids, wherein the feature data of the point cloud in each grid satisfies a feature distribution condition, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighborhood points of the point cloud; selecting a target point cloud of each grid in the point cloud contained in each grid, and deleting other point clouds except the target point cloud in each grid; obtaining target point cloud data based on the target point cloud of each grid.

[0006] In this method, by dividing the target area of ​​the point cloud data into grids and then using the target point cloud in each grid to replace all the point clouds in the grid, the number of point clouds can be effectively reduced and point cloud compression can be achieved. At the same time, the spatial geometric characteristics of the point cloud in the divided grid must meet the characteristic distribution conditions. That is, the grid division in this method is based on the geometric distribution characteristics of the point cloud. This ensures that while the point cloud is compressed, it avoids causing significant damage to the geometric distribution characteristics of the point cloud, thereby reducing the damage to the shape characteristics of the point cloud data and improving the point cloud compression effect. In addition, the geometric distribution characteristics of the point cloud are a kind of posture-invariant local features. Therefore, the rotation and translation invariance of the point cloud can be guaranteed during grid division, improving stability and thus reducing the error of point cloud data processing.

[0007] In one possible design, the grid division of the target area includes: dividing the target area into multiple grids according to a set grid size; not performing division processing on each grid that does not meet the division conditions; performing the following steps on each grid that meets the division conditions: dividing the target grid into multiple grids, the target grid being each grid that meets the division conditions; wherein the division condition is that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0008] In this method, when dividing the grid, by setting the division conditions, the grid can be selectively divided, and finally non-uniform grid division can be achieved. Through non-uniform grid division, the point cloud in the point cloud data can be selectively simplified, thereby reducing the elimination of certain point clouds with obvious features and reducing the damage to the shape characteristics of the point cloud data.

[0009] In one possible design, whether the feature data of the point cloud in the grid meets the feature distribution condition is determined according to the following method: a first target parameter is calculated based on the feature data of all point clouds in the grid; if it is determined that the first target parameter is not greater than a set threshold, it is determined that the feature data of the point cloud in the grid meets the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0010] In this method, by determining the relevant parameters reflecting the geometric distribution characteristics of the point cloud in the grid based on the feature data of the extracted point cloud, it is possible to quickly determine whether the divided grid meets the requirements based on the relevant parameters, and then determine whether to further divide the grid.

[0011] In one possible design, after obtaining target point cloud data based on the target point cloud of each grid, the method further includes: using a random sampling consensus algorithm to perform plane fitting on the point cloud contained in the target point cloud data to obtain at least one fitting plane; in the at least one fitting plane, determining a first target fitting plane whose distance from the target plane in the target coordinate system is less than a set distance value; if there is a second target fitting plane in the at least one plane that meets the set conditions, performing voxel filtering on the point cloud data contained in the second target fitting plane; wherein the set conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than a set value.

[0012] In this method, after removing a part of the point clouds in the initial point cloud data to obtain the target point cloud data, some planes with a large number of point clouds are further selected in the target point cloud data, and the selected planes are filtered. Among them, the point clouds in the planes generally have relatively smooth distribution characteristics, so removing some redundant points will not cause great damage to the shape characteristics of the point cloud data. At the same time, the point cloud data can be further streamlined to reduce the size of the point cloud data file.

[0013] In a possible design, before determining the target area of ​​the initial point cloud data, the method further includes: filtering the initial point cloud data to remove outliers in the initial point cloud data.

[0014] In this method, the initially collected point cloud data generally contains a certain amount of noise points and measurement error points. The existence of these outliers will affect the local distribution characteristics of the point cloud in the initial point cloud data. Therefore, by removing outliers, the accuracy of point cloud feature extraction in the point cloud data can be improved, thereby improving the accuracy of point cloud data processing.

[0015] In one possible design, after selecting the target point cloud of each grid in the point cloud contained in each grid, the method further includes: adjusting the second target parameter of the target point cloud in each grid to the average value of the second target parameters of all point clouds contained in the grid.

[0016] In this method, when the target point cloud of the grid is used as the representative point of all the point clouds in the grid, by adjusting the relevant parameters of the target point cloud to the average value of the relevant parameters of all the point clouds in the grid where it is located, the point cloud parameter characteristics of the entire grid can be reflected more accurately, avoiding deviations caused by some extreme parameter values.

[0017] In a possible design, the target point cloud of each grid is the centroid point in the point cloud contained in each grid.

[0018] In this method, the centroid point of the point cloud in the grid is used as the representative point of the grid, which can better preserve the spatial layout characteristics of the point cloud in the point cloud data and reduce the damage to the point cloud distribution characteristics of the point cloud data.

[0019] In a second aspect, the present application provides a point cloud data processing device, which includes an acquisition unit and a processing unit; the acquisition unit is used to acquire initial point cloud data; the processing unit is used to determine a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data; the processing unit is also used to divide the target area into grids to obtain multiple grids, wherein the feature data of the point cloud in each grid meets the feature distribution conditions, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighborhood points of the point cloud; the target point cloud of each grid is selected from the point cloud contained in each grid, and other point clouds except the target point cloud in each grid are deleted; the target point cloud data is obtained based on the target point cloud of each grid.

[0020] In one possible design, the processing unit performs grid division on the target area, including: dividing the target area into multiple grids according to a set grid size; not performing division processing on each grid that does not meet the division conditions; performing the following steps on each grid that meets the division conditions: dividing the target grid into multiple grids, the target grid being each grid that meets the division conditions; wherein the division condition is that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0021] In one possible design, the processing unit determines whether the feature data of the point cloud in the grid meets the feature distribution condition according to the following method: calculating a first target parameter based on the feature data of all point clouds in the grid; if it is determined that the first target parameter is not greater than a set threshold, then determining that the feature data of the point cloud in the grid meets the feature distribution condition; otherwise, determining that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0022] In one possible design, after obtaining the target point cloud data based on the target point cloud of each grid, the processing unit is further used to: use a random sampling consensus algorithm to perform plane fitting on the point cloud contained in the target point cloud data to obtain at least one fitting plane; in the at least one fitting plane, determine a first target fitting plane whose distance from the target plane in the target coordinate system is less than a set distance value; if there is a second target fitting plane that meets the set conditions in the at least one plane, perform voxel filtering on the point cloud data contained in the second target fitting plane; wherein the set conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than the set value.

[0023] In a possible design, before determining the target area of ​​the initial point cloud data, the processing unit is further configured to: filter the initial point cloud data to remove outliers in the initial point cloud data.

[0024] In a possible design, after selecting the target point cloud of each grid in the point cloud contained in each grid, the processing unit is further used to: adjust the second target parameter of the target point cloud in each grid to the average value of the second target parameters of all point clouds contained in the grid.

[0025] In a possible design, the target point cloud of each grid is the centroid point in the point cloud contained in each grid.

[0026] In a third aspect, the present application provides a point cloud data processing device, comprising a memory and a processor; the memory is used to store computer programs; the processor is used to execute the computer programs stored in the memory to implement the method described in the above-mentioned first aspect or any possible design of the first aspect.

[0027] In a fourth aspect, the present application provides a point cloud data processing device, comprising at least one processor and an interface; the interface is used to provide program instructions or data to the at least one processor; the at least one processor is used to execute the program instructions to implement the method described in the above-mentioned first aspect or any possible design of the first aspect.

[0028] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a data processing device, the data processing device executes the method described in the first aspect or any possible design of the first aspect.

[0029] In a sixth aspect, the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a data processing device, it implements the method described in the above-mentioned first aspect or any possible design of the first aspect.

[0030] In the seventh aspect, the present application provides a chip system, which includes at least one processor and an interface, wherein the interface is used to provide program instructions or data to the at least one processor, and the at least one processor is used to execute the program instructions to implement the method described in the above-mentioned first aspect or any possible design of the first aspect.

[0031] In one possible design, the chip system also includes a memory, which is used to store program instructions and data.

[0032] In one possible design, the chip system is composed of a chip, or includes a chip and other discrete devices.

[0033] For the beneficial effects of the second to seventh aspects mentioned above, please refer to the description of the beneficial effects of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a possible application scenario applicable to a point cloud data processing method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a point cloud data processing method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a plane fitting processing method provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a point cloud grid division method provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram showing the comparison of point cloud data before and after processing provided in an embodiment of the present application;

[0039] Figure 6a A schematic diagram of a method for limiting the display range of a point cloud provided in an embodiment of the present application;

[0040] Figure 6b A schematic diagram of another method for limiting the display range of a point cloud provided in an embodiment of the present application;

[0041] Figure 7a A schematic diagram of raw point cloud data provided in an embodiment of the present application;

[0042] Figure 7b A schematic diagram of processed point cloud data provided in an embodiment of the present application;

[0043] Figure 8 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0044] Figure 9 A schematic diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0046] To facilitate understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0047] 1) Point cloud data: Point cloud data is a collection of point data on an object's surface measured by a measuring device. Point cloud data is a collection of points obtained by obtaining the spatial coordinates of each sampling point on the object's surface. It is also known as a massive point collection of the target object's surface characteristics.

[0048] Point cloud data measured using laser measurement principles (also known as laser point cloud data) includes information such as 3D coordinates and laser reflection intensity. Point cloud data obtained using photogrammetry principles includes information such as 3D coordinates and color, where the color information can be in red, green, and blue (RGB) format. Combining laser measurement and photogrammetry principles yields point cloud data including 3D coordinates, laser reflection intensity, and color.

[0049] 2) Point feature histograms (PFH): Point feature histograms are a type of pose-invariant local feature that describes the geometric features of local point cloud data based on the relationship between the point cloud and its neighborhood points contained in the point cloud data and their estimated normals. PFH describes the neighborhood geometric properties of the point cloud by parameterizing the spatial differences between the query point cloud and the neighborhood points, and forming a multidimensional histogram. The high-dimensional hyperspace where the histogram is located provides a measurable information space for feature representation, which is invariant to the 6-dimensional pose of the surface corresponding to the point cloud, and is robust under different sampling densities or noise levels in the neighborhood. Fast point feature histograms (FPFH) are a simplified form of the PFH calculation method.

[0050] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0051] The specific operating methods in the method embodiments may also be applied to the device embodiments or system embodiments.

[0052] 3D point cloud data is essential for various scenarios, including autonomous driving and high-precision mapping. For example, in the field of intelligent driving, high-precision electronic map acquisition systems, as well as autonomous and assisted driving systems, often use radar to collect point cloud data. For example, LiDAR (Light Detection and Ranging) is used to acquire 3D point cloud data with reflection intensity, and then to obtain corresponding environmental information based on this 3D point cloud data.

[0053] LiDAR is a radar system that uses laser beams to detect a target's position, speed, and other characteristic parameters. It operates by transmitting a detection signal, such as a laser beam, toward the target. It then compares the received signal reflected from the target, such as the target echo, with the transmitted signal. After appropriate processing, it obtains relevant target information, such as its range, position, altitude, speed, attitude, and even shape. If a LiDAR scans the laser beam along a specific trajectory, it records the reflected laser points as it scans. Because the scanning process is extremely precise, a large number of laser points can be obtained during the scanning process, forming point cloud data.

[0054] Currently, the amount of 3D point cloud data collected by radar is very large, and it requires a lot of memory and disk space when stored, which is not conducive to point cloud data transmission, point cloud data playback and other processing. Therefore, it is necessary to compress or streamline the point cloud data to reduce the amount of point cloud data.

[0055] Point cloud compression requires maintaining the shape characteristics of the point cloud while reducing the number of points and the size of the point cloud data file. Currently, the commonly used point cloud compression method is the proportional voxel filtering method. This method performs uniform thinning on the point cloud data, which can easily lose the shape or distribution characteristics of the point cloud, resulting in poor point cloud compression results.

[0056] In view of this, an embodiment of the present application provides a point cloud data processing method, which selectively sparsely simplifies point cloud data based on point cloud features, and can reduce damage to the point cloud structure while compressing the data volume of point cloud data, thereby improving the point cloud compression effect.

[0057] It should be noted that, in the embodiment of the present application, compressing point cloud data can be understood as selectively deleting a part of the point cloud from the collected point cloud data and retaining the remaining part of the point cloud, thereby reducing the number of point clouds contained in the point cloud data, reducing the file size of the point cloud data, and realizing compression of the point cloud data. Therefore, compressing point cloud data can also be understood as streamlining the point cloud data.

[0058] The point cloud data processing method provided in the embodiment of the present application can compress the point cloud data collected by the radar. The method can be applied to a data processing device with data processing capabilities. The data processing device can be a vehicle with data processing capabilities, or an on-board device with data processing capabilities in a vehicle, or a sensor that can collect and process point cloud data. On-board devices can include but are not limited to on-board terminals, on-board controllers, on-board modules, on-board modules, on-board components, on-board chips, on-board units, on-board radars and other devices. The data processing device can also be other electronic devices with data processing capabilities, including but not limited to smart home devices (such as TVs, etc.), smart robots, mobile terminals (such as mobile phones, tablets, etc.), wearable devices (such as smart watches, etc.) and other smart devices. The data processing device can also be a controller, chip, radar and other devices in a smart device.

[0059] The point cloud data processing method provided in the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. It can be understood that the embodiments described below are only part of the embodiments of the present application, rather than all the embodiments.

[0060] Figure 1 A schematic diagram of a possible application scenario applicable to a point cloud data processing method provided in an embodiment of the present application. Figure 1 As shown, the application scenario of the point cloud data processing method provided in the embodiment of the present application can be an assisted driving scenario. In this scenario, the vehicle is in a natural environment, such as a vehicle driving on a road, and a radar is installed on the vehicle. The radar can measure the surrounding environment to obtain point cloud data of the vehicle's surrounding environment.

[0061] The radar can send the collected point cloud data to the vehicle or the on-board device on the vehicle, so that the vehicle or the on-board device executes the point cloud data processing method provided in the embodiment of the present application on the collected point cloud data; or, the radar executes the point cloud data processing method provided in the embodiment of the present application on the obtained point cloud data, and sends the processed point cloud data to the vehicle or the on-board device on the vehicle, so that the vehicle or the on-board device performs subsequent operations on the processed point cloud data, such as playing the point cloud data.

[0062] Figure 1 In the present application, radar is merely an example of a device capable of collecting point cloud data. The device for collecting point cloud data in the embodiments of the present application is not limited to radar but can be any other device capable of collecting point cloud data, such as a camera, and the embodiments of the present application do not limit this. The location of the radar on the vehicle is also merely an example and is not limited to this specific location.

[0063] certainly Figure 1 This is just an example, and the application scenarios of the embodiments of the present application are not limited to this. For example, the data processing device that executes the point cloud data processing method provided in the embodiments of the present application may not be a radar but other equipment, and the device may not be installed on the vehicle but on other equipment, or the device may be installed separately.

[0064] The point cloud data processing method provided by this application is described in detail below in conjunction with specific embodiments.

[0065] Figure 2 A schematic diagram of a point cloud data processing method provided in an embodiment of the present application.

[0066] For the sake of convenience, the following description will be given by taking the point cloud data processing method provided by the present application as an example, which is executed by a data processing device. The data processing device may be, but is not limited to, a device with data processing capabilities provided in the embodiment of the present application, for example, Figure 1 Radar, vehicle or vehicle-mounted equipment, or other equipment such as server, cloud server, etc. in the scene shown.

[0067] like Figure 2 As shown, the point cloud data processing method provided by this application includes:

[0068] S201: The data processing device obtains initial point cloud data and determines a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data.

[0069] In the embodiment of the present application, radar is used as an example to illustrate the device for collecting initial point cloud data.

[0070] The initial point cloud data collected by the radar includes at least the location information of the point cloud, which can be the three-dimensional coordinates of the point cloud. The point cloud data can also include color parameters of the point cloud, such as RGB color values ​​and reflection intensity parameters.

[0071] The initial point cloud data is obtained by measuring the environment of the scene by the radar. For example, the radar is the above Figure 1 When the radar is shown, the scene where the radar is located is Figure 1 In the scenario shown, a vehicle is traveling on a road. The radar measures and collects data about the surrounding environment, generating initial point cloud data and sending it to a data processing device. The radar can collect point cloud data for all objects, including the vehicle, the road, other vehicles on the road, roadside signs, and buildings. Alternatively, the radar can collect point cloud data for specific objects or within a specific range, depending on actual needs.

[0072] In the embodiment of the present application, the target region of the initial point cloud data must include all point clouds in the initial point cloud data. The target region can be the minimum circumscribed region of all point clouds in the initial point cloud data. The shape of the region can be any shape or any set shape, such as a cuboid, cube, triangular prism, triangular pyramid, or sphere.

[0073] In the embodiment of the present application, the data processing method provided in the present application is introduced and explained by taking the shape of the target area as a rectangular parallelepiped as an example. The processing method when the target area is of other shapes can refer to the processing method used when the target area is a rectangular parallelepiped.

[0074] When the target area of ​​the initial point cloud data is in the shape of a cuboid, the target area can be understood as the circumscribed cuboid of the initial point cloud data. The following description takes the circumscribed cuboid as the minimum circumscribed cuboid that can contain all point clouds in the initial point cloud data as an example.

[0075] After acquiring the initial point cloud data collected by the radar, the data processing device determines the smallest cuboid that can contain all the point clouds in the initial point cloud data and defines this smallest cuboid as the circumscribed cuboid of the initial point cloud data. When all sides of the circumscribed cuboid are equal in length, the circumscribed cuboid can also be referred to as a circumscribed cube. The circumscribed cuboid (or circumscribed cube) is a bounding box surrounding the object being processed and, in this application, can be understood as a fictitious outer frame surrounding the initial point cloud data.

[0076] In some embodiments of the present application, after the data processing device obtains the initial point cloud data collected by the radar, it can also determine the smallest cube that can contain all the point clouds in the initial point cloud data, and determine the smallest cube as the circumscribed cube of the initial point cloud data, and subsequently perform the processing described below on the circumscribed cube.

[0077] Radar scanning typically generates point cloud collections with varying point cloud densities. Furthermore, noise points generated during measurement and sparse outliers due to measurement errors can affect the distribution of local features in the point cloud data, potentially corrupting the processing results. Therefore, in some embodiments of the present application, the data processing device can filter the initial point cloud data before determining the bounding box of the initial point cloud data. This filtering process performs a statistical analysis of the neighborhood of each point cloud and removes any point clouds that do not meet the criteria, thereby ensuring the accuracy of subsequent processing.

[0078] In some embodiments of the present application, a data processing device may employ a statistical outlier removal filter to identify and delete outliers from a point cloud contained in the initial point cloud data. Specifically, the data processing device may set the parameters of the statistical outlier removal filter as follows: a search neighborhood for the point cloud is a set neighborhood value, and a reference standard deviation multiple is a set multiple value. For example, the set neighborhood value may be 50, and the standard deviation multiple may be 1. For a point cloud within the set search neighborhood, if the average distance between the point cloud and the remaining point clouds within the search neighborhood is greater than a standard range for the distribution of point clouds within the search neighborhood, the point cloud is determined to be an outlier and is deleted from the initial point cloud data. Specifically, the average distance from each point cloud within the search neighborhood to all of its neighboring points is calculated. Assuming that the result is a Gaussian distribution whose shape is determined by the mean and standard deviation, points whose corresponding average distances are outside the standard range are determined to be outliers, where the standard range is the range corresponding to the product of the standard deviation and the set standard deviation multiple.

[0079] S202: The data processing device divides the target area into grids to obtain multiple grids, wherein the feature data of the point cloud in each grid meets the feature distribution condition, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighboring points of the point cloud.

[0080] Specifically, when the data processing device divides the target area into grids, the following method can be adopted: first, the target area is divided into multiple grids according to the set grid size; among the multiple grids obtained, each grid that does not meet the division conditions is not divided; for each grid that meets the division conditions, the following steps are performed: the target grid is divided into multiple grids, wherein the target grid is each grid that meets the division conditions.

[0081] Among them, the division condition is that the characteristic data of the point cloud in the grid does not meet the characteristic distribution condition, and the data processing device determines whether the characteristic data of the point cloud in the grid meets the characteristic distribution condition in the following way: calculate the first target parameter based on the characteristic data of all point clouds in the grid; if it is determined that the first target parameter is not greater than the set threshold, it is determined that the characteristic data of the point cloud in the grid meets the characteristic distribution condition; otherwise, it is determined that the characteristic data of the point cloud in the grid does not meet the characteristic distribution condition.

[0082] The above-mentioned first target parameter can be a variance parameter, a standard deviation parameter, etc., wherein, in the following embodiments of this application, the first target parameter is taken as an example to introduce the solution.

[0083] The detailed description is given below.

[0084] After determining the circumscribed cuboid of the acquired initial point cloud data, the data processing device evenly divides the circumscribed cuboid into a plurality of grids according to a set grid size. The set grid size may have the same or different side lengths for the grids.

[0085] The grid can also be understood as an imaginary frame that contains part of the point cloud or does not contain any point cloud. Set the grid size to any of the following:

[0086] 1) Set the grid size to a fixed size.

[0087] For example, the grid size can be set to a fixed 40×30×20 cm, that is, the grid corresponding to the grid size is set to a rectangular parallelepiped with side lengths of 40, 30, and 20 cm respectively. The data processing device then evenly divides the circumscribed rectangular parallelepiped into multiple grids of size 40×30×20 cm.

[0088] 2) Set the grid size to the size determined by the size of the circumscribed cuboid.

[0089] Exemplarily, the product of each side length of the circumscribed cuboid and the set coefficient can be respectively determined as each grid side length corresponding to the set grid size, and the set coefficient is greater than 0 and not greater than 1.

[0090] For example, if the coefficient is set to 0.1%, when the size of the circumscribed cuboid determined by the data processing device is 100×80×40 meters, the grid side lengths corresponding to the grid size are set to 100×0.1%=0.1 meter, 80×0.1%=0.08 meter, and 40×0.1%=0.04 meter, respectively. That is, the grid size is set to 10×8×4 centimeters, and the data processing device evenly divides the circumscribed cuboid into multiple grids of 10×8×4 centimeters in size. For another example, when the lengths of each side of the circumscribed cuboid determined by the data processing device are 50 meters, that is, when the circumscribed cuboid is also a cube, the grid size is set to 5×5×5 centimeters.

[0091] In the embodiment of the present application, the grid may also be referred to as a voxel grid.

[0092] After the data processing device preliminarily divides the circumscribed cuboid, it determines whether each grid obtained by the division meets the division conditions. If so, the grid is further divided, and for each grid obtained after the division, it continues to determine whether the division conditions are met and continues to divide when the division conditions are met, and so on, until all the grids obtained by the division do not meet the division conditions, and then the grid division is stopped.

[0093] Specifically, after the data processing device preliminarily divides the circumscribed cuboid into multiple grids, it continues to perform the following grid division steps:

[0094] Step 1: The data processing device takes each grid among the multiple grids obtained by division as a target grid.

[0095] Step 2: The data processing device determines whether the target grid meets the division conditions. If so, step 3 is executed; otherwise, the target grid is not divided.

[0096] Step 3: The data processing device divides the target grid into multiple grids, and uses each grid in the multiple grids obtained by division as the target grid, and executes step 2.

[0097] The local geometric features of a point cloud include PFH or FPFH. Both PFH and FPFH are pose-invariant local features that are invariant to the 6-DOF pose of the corresponding surface of the point cloud. Feature extraction is robust under varying sampling densities and neighborhood noise levels. Therefore, during grid partitioning, the PFH or FPFH features of the point cloud can be used to determine whether to further partition the grid.

[0098] Specifically, in the above step 2, when the data processing device determines whether the target grid meets the division condition, it first determines the feature data of each point cloud in the target grid based on the point cloud contained in the target grid, wherein the feature data is used to represent the positional relationship between the point cloud and the neighborhood points of the point cloud; then, the data processing device calculates the variance of the feature data of all point clouds in the target grid, and determines whether the obtained variance is greater than a set threshold. If the variance is greater than the set threshold, it is determined that the feature data of the point cloud in the target grid does not meet the feature distribution condition, and the target grid meets the division condition; otherwise, it is determined that the feature data of the point cloud in the target grid meets the feature distribution condition, and the target grid does not meet the division condition.

[0099] As an optional embodiment, the variance calculated above is the variance calculated based on the PFH of the point cloud within the target grid. Specifically, the feature data of the point cloud includes a distance parameter and three angle parameters, wherein one distance parameter is the Euclidean distance between the point cloud and one of its neighboring points, and the three angle parameters are used to represent the angular deviation between the normal of the point cloud and the normal of one of its neighboring points. For each point cloud in the target grid, the data processing device uses the point cloud and its neighboring points to form point pairs, and calculates the distance parameter and angle parameter corresponding to each point pair. After the calculation is completed, the PFH is obtained, and the variance of the feature data contained in the PFH is calculated. Finally, based on the obtained variance, it is determined whether the target grid meets the division conditions.

[0100] As another optional embodiment, the variance calculated above is the variance calculated based on the FPFH of the point cloud within the target grid. Specifically, the feature data of the point cloud includes three angle parameters representing the angular deviation between the normal of the point cloud and the normals of its neighboring points. For each point cloud in the target grid, the data processing device forms point pairs using the point cloud and its neighboring points, and calculates the angle parameters corresponding to each pair. After the calculations are completed, the FPFH is obtained, and the variance of the feature data included in the FPFH is calculated. Finally, based on the obtained variance, it is determined whether the target grid meets the classification criteria.

[0101] After determining the variance corresponding to the grid as described above, the variance is compared with the set threshold. If the variance is less than the set threshold, it means that the PFH or FPFH feature of the point cloud in the grid has changed little, and the grid does not need to be further divided. If the variance is greater than or equal to the set threshold, it means that the PFH or FPFH feature of the point cloud in the grid has changed dramatically, and the grid can be further divided.

[0102] Generally, point clouds in point cloud data are three-dimensional. Three-dimensional point cloud data is unstructured and characterized by sparsity, disorder, non-uniform distribution, and large variations in quantity. The above method of this embodiment determines feature data of a point cloud based on point pairs consisting of a point cloud and its neighboring points. This method has a certain degree of resistance to interference, such as rotation, and thus can produce more accurate and robust feature data.

[0103] In the above step 3, as an optional implementation, when the data processing device divides the target grid into multiple grids, the target grid can be evenly divided into n grids by reducing the length of each side of the target grid to 1 / n of the original side length and taking the obtained size as the side length. 3 grids, where n is an integer not less than 2.

[0104] For example, if the target grid is a cube with a side length of 40 cm and n is 2, the side length of 40 cm is reduced to 1 / 2 of its original length, resulting in 20 cm. The data processing device then divides the target grid into grids with a side length of 20 cm, resulting in a total of 8 smaller grids. If the target grid is a cube with a size of 90 × 60 × 30 cm and n is 3, the target grid size is 30 × 20 × 10 cm after each side length is reduced to 1 / 3 of its original length. The data processing device then evenly divides the target grid into grids with a size of 30 × 20 × 10 cm, resulting in a total of 27 smaller grids.

[0105] When the data processing device completes the grid division step and determines that all grids obtained by division do not meet the division condition, it determines that the grid division is completed and continues to perform the following point cloud simplification processing.

[0106] S203: The data processing device selects a target point cloud of each grid from the point clouds contained in each grid, and deletes other point clouds except the target point cloud in each grid.

[0107] After the data processing device completes the grid division, the point cloud data can be simplified. Specifically, the data processing device selects the target point cloud of each grid from the point cloud contained in each grid.

[0108] In some embodiments of the present application, the target point cloud is the centroid of the point cloud contained in the grid, where the coordinates of the centroid in the grid are obtained by calculating the average of the coordinates of all the point clouds in the grid. In some embodiments of the present application, the target point cloud can also be the centroid of the point cloud contained in the grid.

[0109] For each grid, after the data processing device determines the target point cloud in the grid, it deletes all point clouds in the grid except the target point cloud and replaces the grid with only the target point cloud. At the same time, the second target parameter of the target point cloud in the grid can be adjusted to the average of the second target parameters of all point clouds contained in the grid, wherein the second target parameter includes the color parameter and / or reflection intensity parameter of the point cloud, and the color parameter can be an RGB color value. Specifically, the data processing device can adjust the RGB color parameter of the target point cloud in the grid to the average of the red, green, and blue color parameters of all point clouds contained in the grid; and / or adjust the reflection intensity parameter of the target point cloud in the grid to the average of the reflection intensity parameters of all point clouds contained in the grid.

[0110] S204: The data processing device obtains target point cloud data based on the target point cloud of each grid.

[0111] After the data processing device performs the above processing on each grid, target point cloud data composed of all retained target point clouds is obtained. The target point cloud data is point cloud data obtained by streamlining or compressing the initial point cloud data.

[0112] In the embodiment of the present application, the target point cloud data obtained by simplifying the initial point cloud data may also contain many points with smooth features but redundant features. Figure 1 For example, in the target point cloud data obtained after streamlining the initial point cloud data collected by the radar, the smooth but redundant points are the road surface and building planes surrounding the vehicle. The road surface generally contains more traffic sign information, so its point cloud data can be retained without further processing. However, the redundant building plane data on both sides of the road can be further streamlined. Therefore, in some embodiments of the present application, the target point cloud data can be further downsampled by plane fitting to remove redundant point clouds of the building planes on both sides of the road, further reducing the number of point clouds.

[0113] Specifically, after the data processing device obtains the target point cloud data, it can use a random sampling consensus algorithm to perform plane fitting on the point cloud contained in the target point cloud data to obtain at least one fitting plane, and then determine in the at least one fitting plane a first target fitting plane whose distance from the target plane in the target coordinate system is less than a set distance value; if there is a second target fitting plane that meets the set conditions in at least one plane, the point cloud data contained in the second target fitting plane is filtered; wherein the set conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than the set value, and the target coordinate system is the coordinate system adopted by the device for collecting the initial point cloud data, and the coordinate system is a coordinate system with the center of the device for collecting the initial point cloud data (such as a radar) as the origin.

[0114] The detailed description is given below.

[0115] The data processing device can perform random sample consensus (RANSAC) plane fitting on the point cloud included in the target point cloud data. The plane fitting uses N points as fitting conditions to determine a plane containing more than N point clouds, where N is a set positive integer.

[0116] As an optional embodiment, the data processing device may first randomly select three point clouds from the target point cloud data, forming a plane from these three point clouds. The device then calculates the distance from each other point cloud in the target point cloud data to the plane. If the distance from the point cloud to the plane is less than a preset distance value, the point cloud is considered to be located in the plane; otherwise, the point cloud is considered not to be located in the plane. Finally, if the number of point clouds located in the plane is determined to be greater than N, the plane is determined to be a fitted plane, and all point clouds located in the fitted plane are marked as matched. Three more point clouds are then selected from the point clouds not marked as matched, and a fitted plane is determined based on these three selected point clouds. If the number of point clouds located in the plane is determined to be less than N, three more point clouds are directly selected from the point clouds not marked as matched, and a fitted plane is determined based on these three selected point clouds. The data processing device iteratively performs plane fitting in the above manner until a termination condition is satisfied, resulting in at least one fitted plane. The termination condition is that after M iterations, the number of point clouds contained in the plane determined is less than N, or three point clouds not marked as matched cannot be found. Wherein, M is the set number of iterations, which is a positive integer.

[0117] After the data processing device determines at least one fitting plane through the above plane fitting, it selects a plane from the at least one fitting plane that needs to be further simplified in terms of the number of point clouds, and further simplifies the selected plane. Figure 3 Provide explanation.

[0118] Figure 3 This is a schematic diagram of a plane fitting processing method provided in an embodiment of the present application. Figure 3 As shown, the X-axis, Y-axis, and Z-axis are three coordinate axes of the three-dimensional coordinate system used by the device for collecting initial point cloud data, wherein any two coordinate axes are perpendicular to each other, and point O is the origin of the three-dimensional coordinate system.

[0119] In application Figure 1 In the scenario shown, the X axis can be the same as the direction of the vehicle, the XOY plane can be the horizontal plane in the natural coordinate system, and the Z axis represents the height. Assume that the point cloud contained in the target point cloud data is distributed in the three-dimensional coordinate system as follows Figure 3 As shown in , the plane obtained by the data processing device performing plane fitting on the target point cloud data includes Figure 3 The points O, P, Q, and R shown in the figure correspond to the plane L1, the points P1, P2, P3, and P4 correspond to the plane L2, and the points P3, P4, and P5 correspond to the plane L3.

[0120] The target plane is the XOY plane. The data processing device first selects, from among planes L1, L2, and L3, a plane whose distance from the target plane in the coordinate system is less than a set distance value, i.e., the plane with the lowest height. After plane L1 is selected, it is designated as the ground plane. Then, from the remaining planes, a plane is selected whose Z-axis height exceeds a set height value, contains a greater than set number of point clouds, and is perpendicular to plane L1. After plane L2 is selected, it is determined as the building plane. Filtering is then used to reduce the resolution of plane L2 to a set value, for example, one-tenth of the original resolution of plane L2. In specific implementation, a voxel grid filter can be used to filter the point cloud contained in the plane. When the voxel filter filters the point cloud contained in the plane, the circumscribed cuboid of the point cloud contained in the plane is evenly divided into multiple grids, and the length of each side of each grid is one tenth of the length of the corresponding side of the circumscribed cuboid. The target point cloud in each grid is used as the representative point of the grid, and the non-target point cloud in the grid is deleted.

[0121] In the above method, by determining the building planes in the point cloud of the vehicle driving environment and performing downsampling processing on them, redundant data of the building planes on both sides of the road can be effectively removed, further reducing the amount of point cloud data.

[0122] It should be noted that the step numbers in the various embodiments described in the embodiments of the present application are only an example of the execution process and do not constitute a restriction on the order of execution of the steps. In the embodiments of the present application, there is no strict execution order between the steps that have no temporal dependency relationship with each other. For example, in the above-mentioned step S203, when it is determined that a certain grid does not meet the grid division requirements, the data processing device in the above-mentioned step S204 selects the target point cloud from the point cloud contained in the grid and deletes the processing steps of other point clouds except the target point cloud. Alternatively, after step S203 is completed and it is determined that all grids do not meet the grid division requirements, step S204 may be executed for each grid. That is, when step S203 and step S204 are executed, step S203 may be executed before step S204, or the two steps may be selectively executed simultaneously.

[0123] In the above embodiment, during the process of streamlining and compressing point cloud data, the point cloud data is divided into grids and the target point cloud determined in each grid is used to replace the grid, thereby reducing the number of point clouds and achieving point cloud compression. At the same time, when dividing the grid, the grid can be selectively divided by setting the division conditions. By non-uniform grid division, the elimination of point clouds with obvious features is reduced, thereby ensuring the shape characteristics of the point cloud data and improving the point cloud compression effect. Among them, the division conditions are set according to the characteristic parameters in the grid that reflect the degree of discreteness of the point cloud distribution. The grid division can be combined with the geometric characteristics of the point cloud data to ensure the posture invariance of the point cloud, improve the robustness, and ensure the accuracy of the point cloud features, thereby avoiding grid division errors caused by changes such as translation and rotation of the point cloud, and further improving the point cloud compression effect.

[0124] The grid division method in the above embodiment of the present application is described in detail below with reference to specific examples.

[0125] Reference Figure 4 , is a schematic diagram of a point cloud grid division method provided in an embodiment of the present application. Figure 4 As shown in Figure (a), it is assumed that the initial point cloud data obtained by the data processing device is a point cloud set S.

[0126] The following description will be made by taking the circumscribed cube of the initial point cloud data as an example and the centroid point in the point cloud contained in the grid as the target point cloud of the grid as an example.

[0127] When the data processing device processes the initial point cloud data, it first determines the circumscribed cube of the initial point cloud data. The circumscribed cube is the smallest cube that contains all the point clouds in the point cloud set S, that is, the grid S1 containing the initial point cloud data shown in the figure.

[0128] The data processing device divides the grid S1 into a plurality of grids according to the set grid size. As an optional embodiment, the data processing device determines the set grid size according to the size of the grid S1. For example, as shown in Figure (a), the data processing device can determine half of the side length of the grid S1 as the side length corresponding to the set grid size, so that the grid S1 is divided into 8 grids of the same size, and the side length of each grid after division is half of the side length of the grid S1, as shown in grid S2, where grids A, B, C, D, E, F, G, and H are the 8 small grids obtained by dividing the grid S2.

[0129] After the data processing device divides the grid S1 into grid S2, it determines whether each grid obtained by dividing the grid S2 meets the division condition. The figure takes grid D and grid F as examples. Specifically, for grid D, Figure 4As shown in Figure (b), if the data processing device determines that grid D does not meet the division conditions, it will no longer divide grid D, but select the centroid point in the point cloud contained in grid D, and delete the other point clouds in grid D except the centroid point to obtain grid D1, wherein the point cloud shown in grid D1 is the centroid point of the retained grid D. Grid D2 shown in Figure (b) is the grid after the grid D is simplified, which is the same as grid D1. For grid F, as shown in Figure (b), if the data processing device determines that grid F meets the division conditions, it will further divide grid F. For example, as shown in Figure (b), the same method as that of dividing grid S1 is used to divide grid F into 2 3 =8 small grids, and grid F1 is obtained, wherein grids F11, F12, F13, F14, F15, F16, F17, and F18 are 8 small grids obtained by dividing grid F1.

[0130] After the data processing device performs grid division on grid F to obtain grid F1, it then determines whether each grid obtained from the division of grid F1 meets the division criteria. The figure uses grids F11, F16, and F17 as examples for illustration. Specifically, if the data processing device determines that grids F11, F16, and F17 do not meet the division criteria, it no longer divides grids F11, F16, and F17. Instead, it selects centroid points from the point clouds contained in grids F11, F16, and F17, respectively, and deletes all other point clouds from the grids except the centroid point, respectively, to obtain grids F21, F26, and F27, respectively, as shown in Figure (b). After the data processing device selects centroid points and deletes non-centroid points for each grid in grid F1, it obtains grid F2, shown in Figure (b), which is a simplified version of grid F1.

[0131] Figure 4 The processing methods of other grids not shown in the figure can refer to the above method and will not be repeated here.

[0132] The data processing device divides the circumscribed cube corresponding to the initial point cloud data, i.e., grid S1, according to the above-described grid division method. When all newly divided grids no longer meet the division conditions, the point cloud simplification is completed. The point clouds in all newly divided grids constitute the target point cloud data, as shown in grid S3. The point cloud in grid S3 is the point cloud obtained by simplifying the point cloud in grid S1.

[0133] The specific implementation of some of the above methods can refer to the relevant description in the above embodiment 1 and will not be repeated here.

[0134] Figure 5 A schematic diagram showing the comparison of point cloud data before and after processing provided in an embodiment of the present application.

[0135] Figure 5 Grid W shows the point cloud distribution of the original simulated point cloud data before point cloud data processing, and grid W1 shows the point cloud distribution of the point cloud data obtained after point cloud data processing using the method provided in the above embodiment of the present application. Comparing the point cloud distribution in grids W and W1, it can be seen that after processing point cloud data with a large amount of data using the point cloud processing method provided in the embodiment of the present application, the number of point clouds can be effectively reduced, thereby reducing the size of the point cloud data file and achieving point cloud compression. At the same time, the point cloud shown in grid W1 after processing is relatively small compared to the point cloud shown in grid W before processing in terms of shape features, density of point cloud distribution, etc. Therefore, the point cloud processing method provided in the embodiment of the present application has little damage to the shape features of the point cloud, and can ensure a good point cloud compression effect.

[0136] In the above embodiment of the present application, when performing grid division on the point cloud, the initial point cloud data is firstly uniformly grid-divided, and then, based on the division conditions, the grids obtained by the initial division are selectively further divided, wherein the uniform division method is still used when further dividing each grid. The above method ultimately achieves non-uniform gridding of the point cloud, and then uses the centroid point replacement method to achieve point cloud compression, which can reduce the damage to the shape and distribution characteristics of the point cloud, thereby improving the point cloud compression effect.

[0137] In the fields of high-precision electronic map collection, autonomous driving, assisted driving, etc., the initial point cloud data of the scene collected by radar may contain data that is not suitable for display, such as sensitive building features. Relevant regulations require that these sensitive data cannot be made public or held on a large scale for a long time. Therefore, there are great restrictions on the viewing of point cloud data.

[0138] In actual applications, point cloud data is mainly used for driving perception. Point clouds that are extremely far away from the radar are not very helpful for driving perception. Especially in intelligent driving fields such as autonomous driving and assisted driving, driving perception generally relies mainly on point cloud data within a certain range with the vehicle center as the origin. Therefore, the range of point cloud data collected by the radar and subsequently displayed can be limited to avoid or blur the point cloud features of sensitive buildings, while not having much impact on driving perception.

[0139] For example, based on the above embodiments, when the radar collects point cloud data of the vehicle's surrounding environment, it can delete the point cloud data outside the set range and only retain the point cloud data within the set range. The point cloud data within the set range is used as the initial point cloud data, and the initial point cloud data is processed using the point cloud data processing method provided in the above embodiments of the present application, thereby effectively streamlining the point cloud data and legally erasing the point cloud within the invalid range.

[0140] Figure 6a A schematic diagram of a method for limiting the display range of a point cloud provided in an embodiment of the present application. Figure 6a As shown in , when the radar is located on the top of the vehicle, the radar is used as the origin, and in the vertical direction, only the point cloud data within the range corresponding to the set distance on both sides of the radar is retained, that is, Figure 6a The point cloud data is within the range limited by the middle dotted line L4 and the dotted line L5. For point cloud data outside this range, the radar does not collect point cloud data or deletes the collected point cloud data.

[0141] Considering that the height of small vehicles is generally below 2 meters and the maximum height limit of large trucks does not exceed 4.2 meters, the height distance value can be set to 2.5 meters, and only the point cloud data within the range of 2.5 on both sides of the radar is retained, that is, Figure 6a The area between the two dotted lines is 5 meters high, which covers both the ground and the height of almost all vehicle types. Buildings on both sides of the lane are largely blocked due to the 2.5-meter height limit, thus preventing sensitive building information from being fully disclosed.

[0142] Figure 6b This is a schematic diagram of another method for limiting the display range of point clouds provided in an embodiment of the present application. Figure 6b As shown, in Figure 6a Based on the restricted range shown, only the point cloud data within the range corresponding to the set distances on the left and right sides and the front and rear sides of the radar are retained, that is, Figure 6b The range enclosed by the dotted box in the middle. For point cloud data beyond this range, the radar does not collect point cloud data or deletes the collected point cloud data. Figure 6b The x-axis shown in FIG is the same as the vehicle's traveling direction, and the y-axis is perpendicular to the x-axis.

[0143] Generally, point clouds outside the range of 100 meters in front and behind a vehicle are sparsely distributed and have no reference value, so they can be discarded. Therefore, the set distance in the radar's front and rear direction can be 100 meters. Relevant standards stipulate that the maximum width of a standard lane is 3.75 meters, and the width of an eight-lane one-way road is less than 16 meters. Point clouds outside the range of 30 meters to the left and right of the radar do not affect target detection and can also be discarded. Therefore, the set distance in the radar's left and right direction can be 30 meters.

[0144] By retaining point cloud data within a range of 100 meters in front and behind, 30 meters to the left and right, and 2.5 meters above and below the radar, it is possible to reduce the collection of invalid information while retaining the necessary detection information within the lane and on both sides of the road.

[0145] In this embodiment, combined with the above Figure 6a 、 Figure 6bThe method shown in the figure only includes point cloud data within a set range around the vehicle for the point cloud data collected around the vehicle. This can shield the building information on both sides of the road while covering the useful information range, thus avoiding the disclosure of sensitive building information. It can also further reduce the amount of point cloud data and improve the efficiency of point cloud data transmission and playback.

[0146] Figure 7a This is a schematic diagram of the original point cloud data provided in the embodiment of the present application. Figure 7a As shown, this is the point cloud data of the actual environment collected by the radar. The white points in the figure are point clouds. It can be seen that the number of point clouds in the point cloud data in the figure is relatively large.

[0147] Figure 7b This is a schematic diagram of processed point cloud data provided in an embodiment of the present application. Figure 7b As shown, the point cloud data processing method provided by the above embodiment of the present application is used to process Figure 7a The original point cloud data shown is processed to obtain point cloud data. The white points in the figure are point clouds.

[0148] contrast Figure 7a and Figure 7b It can be seen that after processing point cloud data with a large amount of data using the point cloud processing method provided in the embodiment of the present application, the number of point clouds is reduced to about one tenth of the original amount. Therefore, the solution of the present application can effectively reduce the number of point clouds and achieve point cloud compression; at the same time, Figure 7b The processed point cloud shown is the same as Figure 7a Compared with the original point cloud shown, the differences in shape features, density of point cloud distribution, etc. are relatively small. Therefore, the point cloud processing method provided in the embodiment of the present application has little damage to the shape features of the point cloud and can ensure a better point cloud compression effect.

[0149] Based on the above embodiments and the same concept, the present application also provides a data processing device, such as Figure 8 As shown, the data processing device 800 may include: an acquisition unit 801 and a processing unit 802.

[0150] The acquisition unit 801 is used to acquire initial point cloud data.

[0151] The processing unit 802 is configured to determine a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data.

[0152] The processing unit 802 is further configured to divide the target area into grids to obtain a plurality of grids, wherein the feature data of the point cloud in each grid satisfies a feature distribution condition, and the feature data is used to represent the spatial geometric relationship between the point cloud and the neighborhood points of the point cloud; select the target point cloud of each grid from the point cloud contained in each grid, and delete other point clouds except the target point cloud in each grid; and obtain target point cloud data based on the target point cloud of each grid.

[0153] In one possible design, the processing unit 802 performs grid division on the target area, including: dividing the target area into multiple grids according to a set grid size; not performing division processing on each grid that does not meet the division conditions; performing the following steps on each grid that meets the division conditions: dividing the target grid into multiple grids, the target grid being each grid that meets the division conditions; wherein the division condition is that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0154] In one possible design, the processing unit 802 determines whether the feature data of the point cloud in the grid meets the feature distribution condition according to the following method: calculating a first target parameter based on the feature data of all point clouds in the grid; if it is determined that the first target parameter is not greater than a set threshold, then determining that the feature data of the point cloud in the grid meets the feature distribution condition; otherwise, determining that the feature data of the point cloud in the grid does not meet the feature distribution condition.

[0155] In one possible design, after obtaining the target point cloud data based on the target point cloud of each grid, the processing unit 802 is further used to: use a random sampling consensus algorithm to perform plane fitting on the point cloud contained in the target point cloud data to obtain at least one fitting plane; in the at least one fitting plane, determine a first target fitting plane whose distance from the target plane in the target coordinate system is less than a set distance value; if there is a second target fitting plane in the at least one plane that meets the set conditions, perform voxel filtering on the point cloud data contained in the second target fitting plane; wherein the set conditions include: perpendicular to the first target fitting plane, the length along the target direction is greater than the set length value, and the number of point clouds located in the plane is greater than the set value.

[0156] In a possible design, before determining the target area of ​​the initial point cloud data, the processing unit 802 is further configured to: filter the initial point cloud data to remove outliers in the initial point cloud data.

[0157] In one possible design, after selecting the target point cloud of each grid in the point cloud contained in each grid, the processing unit 802 is further used to: adjust the second target parameter of the target point cloud in each grid to the average value of the second target parameters of all point clouds contained in the grid.

[0158] In a possible design, the target point cloud of each grid is the centroid point in the point cloud contained in each grid.

[0159] As an implementation, the data processing device 800 may further include a storage unit 803 for storing the program code and data of the data processing device 800. The processing unit 802 may be a processor or controller, such as a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and the like described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The storage unit 803 may be a memory. The acquisition unit 801 may be an interface circuit of the data processing device, configured to receive data from other devices, such as initial point cloud data sent by a point cloud data acquisition device. When the data processing device is implemented in the form of a chip, the transceiver unit 801 may be an interface circuit of the chip used to receive data from other chips or devices or to send data to other chips or devices.

[0160] The division of units in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0161] Figure 8One or more of the various units in the system may be implemented in software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or codes, and may be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0162] Based on the above embodiments and the same concept, the present application also provides a data processing device for implementing the point cloud data processing method provided in the present application. Figure 9 As shown, the data processing device 900 may include: one or more processors 901, a memory 902, and one or more computer programs (not shown in the figure). As an implementation, the above-mentioned components may be coupled via one or more communication lines 903. The memory 902 stores one or more computer programs, each of which includes instructions. The processor 901 invokes the instructions stored in the memory 902, causing the data processing device 900 to execute the point cloud data processing method provided in the embodiments of the present application.

[0163] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0164] In the embodiments of the present application, the memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. The memory in the embodiments of the present application can also be a circuit or any other device capable of implementing a storage function.

[0165] As an implementation method, the data processing device 900 may also include a communication interface 904 for communicating with other devices via a transmission medium. For example, when the device for collecting the initial point cloud data is not the data processing device 900, the data processing device 900 may communicate with the device for collecting the initial point cloud data via the communication interface 904, thereby receiving the initial point cloud data collected by the device. In an embodiment of the present application, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interfaces. In an embodiment of the present application, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter; or it may be a transceiver with integrated transceiver functions, or an interface circuit.

[0166] In some embodiments of the present application, the processor 901, memory 902, and communication interface 904 can be interconnected via a communication line 903; the communication line 903 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication line 903 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0167] The methods provided in the embodiments of the present application may be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD), or a semiconductor medium (e.g., an SSD), etc.

[0168] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

[0169] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method can be implemented in other ways without exceeding the scope of this application. For example, the embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0174] Furthermore, the described devices and methods, as well as the schematic diagrams of various embodiments, may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present application. Furthermore, any coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device, or unit, which may be electronic, mechanical, or other.

[0175] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A point cloud data processing method, characterized in that: include: Acquire initial point cloud data, and determine a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data; Performing grid division on the target area to obtain a plurality of grids, wherein feature data of a point cloud in each grid satisfies a feature distribution condition, the feature data is used to represent a spatial geometric relationship between a point cloud and neighboring points of the point cloud, and the feature data is used to determine local geometric features of the point cloud in each grid; Select the target point cloud of each grid in the point cloud contained in each grid, and delete the other point clouds except the target point cloud in each grid; According to the target point cloud of each grid, the target point cloud data is obtained; Wherein, whether the feature data of the point cloud in the grid meets the feature distribution condition is determined according to the following method: Calculating a first target parameter based on the feature data of all point clouds in the grid, wherein the first target parameter is used to characterize the degree of change of the local geometric features of the point clouds in the grid; If it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid meets the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not meet the feature distribution condition.

2. The method according to claim 1, characterized in that The grid division of the target area includes: Dividing the target area into multiple grids according to a set grid size; For each grid that does not meet the division conditions, no division is performed; For each grid that meets the division condition, the following steps are performed: dividing the target grid into a plurality of grids, wherein each target grid is a grid that meets the division condition; The division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.

3. The method according to claim 1 or 2, characterized in that After obtaining target point cloud data according to the target point cloud of each grid, the method further includes: Using a random sampling consensus algorithm, plane fitting is performed on the point cloud included in the target point cloud data to obtain at least one fitting plane; Determine, among the at least one fitting plane, a first target fitting plane whose distance to the target plane in the target coordinate system is less than a set distance value; If a second target fitting plane that meets the set conditions exists in the at least one fitting plane, performing voxel filtering on the point cloud data contained in the second target fitting plane; The setting conditions include: being perpendicular to the first target fitting plane, the length along the target direction being greater than a set length value, and the number of point clouds located in the plane being greater than a set value.

4. The method according to claim 1 or 2, characterized in that Before determining the target area of ​​the initial point cloud data, the method further includes: The initial point cloud data is filtered to remove outliers in the initial point cloud data.

5. The method according to claim 1 or 2, characterized in that After selecting a target point cloud of each grid from the point clouds contained in each grid, the method further includes: The second target parameter of the target point cloud in each grid is adjusted to the average value of the second target parameters of all point clouds contained in the grid.

6. The method according to claim 1 or 2, characterized in that The target point cloud of each grid is the centroid point in the point cloud contained in each grid.

7. A data processing device, characterized in that: including an acquisition unit and a processing unit; The acquisition unit is used to acquire initial point cloud data; The processing unit is configured to determine a target area of ​​the initial point cloud data, wherein the target area includes each point cloud in the initial point cloud data; The processing unit is further configured to perform grid division on the target area to obtain a plurality of grids, wherein feature data of a point cloud in each grid satisfies a feature distribution condition, the feature data is used to represent a spatial geometric relationship between a point cloud and neighboring points of the point cloud, and the feature data is used to determine local geometric features of the point cloud in each grid; select a target point cloud in each grid from the point cloud contained in each grid, and delete other point clouds in each grid except the target point cloud; and obtain target point cloud data based on the target point cloud in each grid; The processing unit is configured to determine whether the feature data of the point cloud in the grid meets the feature distribution condition according to the following method: Calculating a first target parameter based on the feature data of all point clouds in the grid, wherein the first target parameter is used to characterize the degree of change of the local geometric features of the point clouds in the grid; If it is determined that the first target parameter is not greater than the set threshold, it is determined that the feature data of the point cloud in the grid meets the feature distribution condition; otherwise, it is determined that the feature data of the point cloud in the grid does not meet the feature distribution condition.

8. The device according to claim 7, characterized in that The processing unit performs grid division on the target area, including: Dividing the target area into multiple grids according to a set grid size; For each grid that does not meet the division conditions, no division is performed; For each grid that meets the division condition, the following steps are performed: dividing the target grid into a plurality of grids, wherein each target grid is a grid that meets the division condition; The division condition is that the feature data of the point cloud in the grid does not satisfy the feature distribution condition.

9. The device according to claim 7 or 8, characterized in that After obtaining target point cloud data according to the target point cloud of each grid, the processing unit is further configured to: Using a random sampling consensus algorithm, plane fitting is performed on the point cloud included in the target point cloud data to obtain at least one fitting plane; Determine, among the at least one fitting plane, a first target fitting plane whose distance to the target plane in the target coordinate system is less than a set distance value; If a second target fitting plane that meets the set conditions exists in the at least one fitting plane, performing voxel filtering on the point cloud data contained in the second target fitting plane; The setting conditions include: being perpendicular to the first target fitting plane, the length along the target direction being greater than a set length value, and the number of point clouds located in the plane being greater than a set value.

10. The device according to claim 7 or 8, characterized in that Before determining the target area of ​​the initial point cloud data, the processing unit is further configured to: The initial point cloud data is filtered to remove outliers in the initial point cloud data.

11. The device according to claim 7 or 8, characterized in that After selecting a target point cloud of each grid from the point clouds contained in each grid, the processing unit is further configured to: The second target parameter of the target point cloud in each grid is adjusted to the average value of the second target parameters of all point clouds contained in the grid.

12. The device according to claim 7 or 8, characterized in that The target point cloud of each grid is the centroid point in the point cloud contained in each grid.

13. A data processing device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computing program stored in the memory to implement the method according to any one of claims 1 to 6.

14. A data processing device, characterized in that: comprising at least one processor and an interface; The interface is used to provide program instructions or data to the at least one processor; The at least one processor is configured to execute the program instructions to implement the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on a data processing device, the data processing device is caused to execute the method according to any one of claims 1 to 6.

Images