Point cloud processing method and device, equipment, storage medium and program product

Abstract

The invention relates to a point cloud processing method and device, equipment, a storage medium and a program product. The method comprises the steps that after environment point clouds in a target area where a vehicle is located are obtained, the target area is divided into at least two columnar areas with the same size, the height feature of each columnar area is determined according to the environment point clouds contained in each columnar area, and the height of each columnar area is determined according to the point cloud feature and the point cloud number of the environment point clouds contained in each columnar area. And determining a three-dimensional feature tensor corresponding to each columnar region according to the height feature of each columnar region and the height feature of each columnar region, and determining environmental information in the target region according to the three-dimensional feature tensor corresponding to each columnar region. By adopting the method, the accuracy of determining the environment information can be improved.

Description

Technical Field

[0001] The present application relates to the field of radar detection technology, and in particular to a point cloud processing method, apparatus, device, storage medium, and program product. Background Art

[0002] With the rapid development of autonomous driving technology, in order to ensure the reliability of driving path planning, lidar can be used to collect environmental point clouds during vehicle driving, and based on the environmental point clouds, the environmental information of the vehicle's environment can be determined to generate a driving path.

[0003] However, existing point cloud detection methods generally determine the environmental information of the vehicle's environment directly based on the point cloud features of the environmental point cloud. However, in complex scenarios, lidar is susceptible to external influences such as noise, which will reduce the accuracy of point cloud feature determination and thus reduce the accuracy of the determined environmental information. Summary of the Invention

[0004] Based on this, it is necessary to provide a point cloud processing method, device, equipment, storage medium and program product that can improve the accuracy of environmental information determination in response to the above technical problems.

[0005] In a first aspect, the present application provides a point cloud processing method, comprising:

[0006] After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two cylindrical areas of equal volume;

[0007] Determine the height characteristics of each columnar area based on the environmental point cloud contained in each columnar area;

[0008] Determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and point cloud quantity of the environment point cloud contained in each columnar area, as well as the height features of each columnar area;

[0009] The environmental information within the target area is determined based on the three-dimensional feature tensor corresponding to each columnar area.

[0010] In one embodiment, determining the height characteristics of each columnar region based on the environmental point cloud contained in each columnar region includes:

[0011] For each columnar area, the height features corresponding to different types of objects in the columnar area are determined based on the distribution of the environmental point cloud contained in the columnar area;

[0012] The height features corresponding to different types of objects are weighted to obtain the height features of the columnar area.

[0013] In one embodiment, determining a three-dimensional feature tensor corresponding to each columnar region based on the point cloud features and the number of point clouds contained in each columnar region and the height features of each columnar region includes:

[0014] For each columnar region, determine the center coordinates of the region center corresponding to the columnar region according to the position information of the columnar region;

[0015] Determine the point cloud features of each environmental point cloud contained in the columnar region based on the point cloud coordinates and reflection intensity of each environmental point cloud contained in the columnar region, the center coordinates of the region center, and the offset between each environmental point cloud contained in the columnar region and the region center;

[0016] According to the point cloud features and the number of point clouds of the environment point cloud contained in the columnar region and the height features of the columnar region, a three-dimensional feature tensor corresponding to the columnar region is determined.

[0017] In one embodiment, determining a three-dimensional feature tensor corresponding to the columnar region based on the point cloud features and the number of point clouds of the environment point cloud contained in the columnar region and the height features of the columnar region includes:

[0018] For each environmental point cloud in the columnar area, the height feature and the point cloud feature of the environmental point cloud are spliced ​​to obtain the environmental feature of the environmental point cloud. The environmental feature, the number of point clouds in the columnar area, and the number of columnar areas containing environmental point clouds in the target area are spliced ​​to obtain the point cloud tensor of the environmental point cloud.

[0019] The point cloud tensors of each environmental point cloud contained in the columnar area are combined to obtain the three-dimensional feature tensor corresponding to the columnar area.

[0020] In one embodiment, dividing the target area into at least two columnar areas of equal volume comprises:

[0021] The ground size corresponding to the target area is evenly divided to obtain the ground size corresponding to the columnar area;

[0022] The area height corresponding to the target area is evenly divided to obtain the area height corresponding to the columnar area;

[0023] Generate at least two cylindrical areas with the same volume based on the ground size and area height.

[0024] In one embodiment, determining environmental information within the target area based on the three-dimensional feature tensor corresponding to each columnar area includes:

[0025] Project the three-dimensional feature tensor corresponding to each columnar area into a pseudo image;

[0026] The pseudo image is convolved to obtain the environmental information within the target area.

[0027] In a second aspect, the present application further provides a point cloud processing device, comprising:

[0028] A region division module is used to obtain the environmental point cloud in the target area where the vehicle is located and divide the target area into at least two columnar areas of equal volume;

[0029] A feature determination module is used to determine the height feature of each columnar area based on the environmental point cloud contained in each columnar area;

[0030] A tensor determination module is used to determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and the number of point clouds of the environment point clouds contained in each columnar area, as well as the height features of each columnar area;

[0031] The information determination module is used to determine the environmental information in the target area based on the three-dimensional feature tensor corresponding to each columnar area.

[0032] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two cylindrical areas of equal volume;

[0034] Determine the height characteristics of each columnar area based on the environmental point cloud contained in each columnar area;

[0035] Determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and point cloud quantity of the environment point cloud contained in each columnar area, as well as the height features of each columnar area;

[0036] The environmental information within the target area is determined based on the three-dimensional feature tensor corresponding to each columnar area.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0038] After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two cylindrical areas of equal volume;

[0039] Determine the height characteristics of each columnar area based on the environmental point cloud contained in each columnar area;

[0040] Determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and point cloud quantity of the environment point cloud contained in each columnar area, as well as the height features of each columnar area;

[0041] The environmental information within the target area is determined based on the three-dimensional feature tensor corresponding to each columnar area.

[0042] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0043] After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two cylindrical areas of equal volume;

[0044] Determine the height characteristics of each columnar area based on the environmental point cloud contained in each columnar area;

[0045] Determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and point cloud quantity of the environment point cloud contained in each columnar area, as well as the height features of each columnar area;

[0046] The environmental information within the target area is determined based on the three-dimensional feature tensor corresponding to each columnar area.

[0047] The above-mentioned point cloud processing method, device, equipment, storage medium and program product introduce the height characteristics of each columnar area. After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two columnar areas of equal volume. The height characteristics of each columnar area are determined based on the environmental point cloud contained in each columnar area. Then, based on the point cloud characteristics and point cloud quantity of the environmental point cloud contained in each columnar area, as well as the height characteristics of each columnar area, the three-dimensional feature tensor corresponding to each columnar area is determined. Then, based on the three-dimensional feature tensor corresponding to each columnar area, the environmental information within the target area is determined. Compared with the related art that only focuses on the point cloud characteristics of the environmental point cloud, the above-mentioned method divides the target area into at least two columnar areas of equal volume, and determines the environmental information within the target area based on the point cloud characteristics and point cloud quantity of the environmental point cloud contained in each columnar area, as well as the height characteristics of each columnar area. This can ensure the accuracy of the environmental information determination. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 1 is a flow chart of a point cloud processing method according to an embodiment;

[0050] Figure 2 A schematic diagram of a process for determining a three-dimensional feature tensor in one embodiment;

[0051] Figure 3 A schematic diagram of a process for determining a three-dimensional feature tensor in another embodiment;

[0052] Figure 4 is a schematic diagram of a columnar region in one embodiment;

[0053] Figure 5 A schematic diagram of a process for determining environmental information in one embodiment;

[0054] Figure 6 A schematic diagram of pseudo image generation in one embodiment;

[0055] Figure 7 is a schematic diagram of convolution processing in one embodiment;

[0056] Figure 8 is a flowchart of a point cloud processing method in another embodiment;

[0057] Figure 9 is a structural block diagram of a point cloud processing device in one embodiment;

[0058] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] With the rapid development of autonomous driving technology, LiDAR (Light Detection and Ranging) (LiDAR) is being used to collect environmental point clouds during vehicle travel to ensure the reliability of driving path planning. Based on these point clouds, environmental information about the vehicle's surroundings can be determined, and a driving path can be generated. LiDAR, a sensing technology that uses laser beams to measure the distance to objects, is widely used in areas such as autonomous vehicles, drones, robot navigation, and intelligent transportation systems. LiDAR emits pulsed laser beams and receives the reflected light, generating three-dimensional point cloud data based on the time difference and intensity of the light's propagation. This point cloud data can be used to detect objects in the surrounding environment and construct high-precision three-dimensional maps, supporting the environmental perception and navigation functions of autonomous driving and other intelligent systems.

[0061] However, existing point cloud detection methods generally determine the environmental information of the vehicle's environment directly based on the point cloud features of the environmental point cloud. However, in complex scenarios, lidar is susceptible to external influences such as noise, which will reduce the accuracy of point cloud feature determination and thus reduce the accuracy of the determined environmental information.

[0062] Based on this, in an exemplary embodiment, a point cloud processing method is provided, which is described by taking the method applied to a terminal deployed in a vehicle as an example. Figure 1 As shown, the specific steps include:

[0063] S101 , after obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two columnar areas of equal volume.

[0064] The so-called target area is the preset range where the vehicle is located; the so-called environmental point cloud is the point cloud within the target area.

[0065] Optionally, after the vehicle's laser radar collects a point cloud within the target area, the collected point cloud can be preprocessed to obtain an environmental point cloud. The preprocessing steps may include, but are not limited to, noise removal, filtering, downsampling, and coordinate conversion.

[0066] Furthermore, in order to analyze the information of the environmental point cloud in more detail, the target area may be divided into at least two columnar areas of the same volume according to the size information of the target area.

[0067] S102 : Determine the height feature of each columnar region according to the environment point cloud contained in each columnar region.

[0068] The so-called height feature is a feature that describes the height information of the environmental point cloud contained in the columnar area.

[0069] Optionally, for each columnar region, a height feature of the columnar region can be determined based on the height information of the environmental point cloud contained within the columnar region and / or the height information of the columnar region within the target region. For example, the environmental point cloud contained within the columnar region and the position information of the columnar region can be input into a trained first feature determination model, which then outputs the height feature of the columnar region based on the point cloud coordinates, position information, and model parameters of each environmental point cloud.

[0070] In another optional manner, for each columnar region, the height information of the environmental point cloud contained in the columnar region may be encoded according to the distribution of the environmental point cloud, and maximum pooling processing may be performed to extract the height features of the columnar region.

[0071] S103 , determining a three-dimensional feature tensor corresponding to each columnar region based on the point cloud features and the number of point clouds of the environment point clouds contained in each columnar region, and the height features of each columnar region.

[0072] Among them, the so-called point cloud feature is the feature that describes the location information of the environmental point cloud contained in the columnar area; the so-called tensor is a multidimensional array used to represent the structure of the data. Furthermore, the three-dimensional feature tensor is a three-dimensional tensor used to describe the characteristics of the environmental point cloud.

[0073] Optionally, for each columnar region, point cloud features of the environmental point cloud contained in the columnar region can be determined based on the position information of the environmental point cloud contained in the columnar region. For example, the position information of the environmental point cloud contained in the columnar region can be input into a trained second feature determination model, which then outputs the point cloud features of the environmental point cloud contained in the columnar region based on the position information of each environmental point cloud and model parameters.

[0074] Furthermore, the point cloud features and the number of point clouds of the environmental point clouds contained in the columnar region, as well as the height features of the columnar region, can be combined to obtain a three-dimensional feature tensor corresponding to the columnar region. For example, the point cloud features and the number of point clouds of the environmental point clouds contained in the columnar region, as well as the height features of the columnar region, can be directly spliced ​​to obtain a three-dimensional feature tensor corresponding to the columnar region.

[0075] S104: Determine the environmental information within the target area according to the three-dimensional feature tensor corresponding to each columnar area.

[0076] The so-called environmental information refers to the relevant information of the environment within the columnar area, which is used to generate the driving route.

[0077] Optionally, after determining the three-dimensional feature tensors corresponding to each columnar region, a deep algorithm can be used to process each three-dimensional feature tensor to obtain environmental information within the target region. For example, the three-dimensional feature tensors corresponding to each columnar region can be encoded and input into a deep learning network and a Single Shot MultiBox Detector (SSD) head. The deep learning network and the SSD detection head then share and learn the encoded three-dimensional feature tensors to obtain environmental information within the target region.

[0078] Furthermore, after determining the environmental information within the target area, operations such as obstacle warning, path generation, and environmental display can be performed based on the environmental information.

[0079] In the above-mentioned point cloud processing method, the height characteristics of each columnar area are introduced. After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two columnar areas of equal volume, and the height characteristics of each columnar area are determined based on the environmental point cloud contained in each columnar area. Then, based on the point cloud characteristics and point cloud quantity of the environmental point cloud contained in each columnar area, as well as the height characteristics of each columnar area, the three-dimensional feature tensor corresponding to each columnar area is determined, and then based on the three-dimensional feature tensor corresponding to each columnar area, the environmental information within the target area is determined. Compared with the related art that only focuses on the point cloud characteristics of the environmental point cloud, the above-mentioned method divides the target area into at least two columnar areas of equal volume, and determines the environmental information within the target area based on the point cloud characteristics and point cloud quantity of the environmental point cloud contained in each columnar area, as well as the height characteristics of each columnar area, thereby ensuring the accuracy of the environmental information determination.

[0080] In order to more reasonably reflect the height characteristics of the columnar area, based on the above embodiments, an optional method for determining the height characteristics is provided in the embodiments of the present application. Specifically, for each columnar area, the height characteristics corresponding to different types of objects in the columnar area are determined according to the distribution of the environmental point cloud contained in the columnar area; the height characteristics corresponding to different types of objects are weighted to obtain the height characteristics of the columnar area.

[0081] The so-called different types of objects may include but are not limited to people, vehicles, obstacles, etc.

[0082] Optionally, for each columnar region, information of different types of objects in the columnar region and height features corresponding to the different types of objects may be determined based on the distribution of the environmental point cloud contained in the columnar region.

[0083] Furthermore, the weighting parameters corresponding to each type of object can be determined based on the degree of influence of different types of objects on the generation of the driving path. Then, the weighting parameters corresponding to each type of object are used to perform weighted processing on the height features corresponding to different types of objects, thereby obtaining the height features of the columnar area.

[0084] In the embodiment of the present application, the height features corresponding to different types of objects are weighted to obtain the height features of the columnar region, which can ensure the rationality of the height feature determination.

[0085] In order to ensure the accuracy of the three-dimensional feature tensor, based on the above embodiment, an optional method for determining the three-dimensional feature tensor is provided in the embodiment of the present application, such as Figure 2 As shown, the specific steps include:

[0086] S201 , for each columnar region, determining the center coordinates of the region center corresponding to the columnar region according to the position information of the columnar region.

[0087] The point cloud center is the center point of the columnar region. Furthermore, the center point of the columnar region provides a global position reference, ensuring spatial consistency across the columnar regions. This helps to more accurately locate the target object, especially in large-scale scenes. The center coordinates are the coordinates of the center point of the columnar region.

[0088] Optionally, for each columnar region, the center coordinates of the region center corresponding to the columnar region can be determined based on the coordinate information of each region vertex in the columnar region's location information. Alternatively, the columnar region's location information can be input into a trained center determination model, which can then output the center coordinates of the region center corresponding to the columnar region based on the columnar region's location information and model parameters.

[0089] S202, determining the point cloud features of each environmental point cloud contained in the columnar area based on the point cloud coordinates and reflection intensity of each environmental point cloud contained in the columnar area, the center coordinates of the area center, and the offset between each environmental point cloud contained in the columnar area and the area center.

[0090] Among them, the point cloud coordinates are the coordinate information of the environment point cloud, which are three-dimensional coordinates. Furthermore, the three-dimensional coordinates of the environment point cloud represent the absolute position of the environment point cloud in the global space, which can be used to identify the position and shape of the object.

[0091] The so-called reflection intensity refers to the intensity of reflected light when the lidar collects environmental point clouds. Furthermore, the reflection intensity is used to distinguish the material or surface characteristics of objects, which helps to distinguish different types of objects in the same scene.

[0092] The so-called offset is used to characterize the relative coordinates between the environment point cloud and the point cloud center. Furthermore, the relative coordinates can capture the local geometric structure and the relative relationship between the point and other points in the columnar unit, thereby improving the expression ability of local spatial features.

[0093] It is understood that in order to ensure the rationality of feature determination between each columnar area, a standard number of point clouds can be preset. For each columnar area, if the number of environmental point clouds in that columnar area is greater than the standard number of point clouds, each environmental point cloud is randomly sampled. That is, the standard number of environmental point clouds is selected and the point cloud features of each environmental point cloud are calculated. If the number of environmental point clouds in that columnar area is less than the standard number of point clouds, the point cloud features are filled with 0 until the number of environmental point clouds reaches the standard number of point clouds. At this point, the number of environmental point clouds in each columnar area is the same.

[0094] Optionally, for each environmental point cloud in each columnar region, the point cloud features of the environmental point cloud can be determined based on the point cloud coordinates and reflection intensity of the environmental point cloud, the center coordinates of the region center of the columnar region where the environmental point cloud is located, and the offset between the environmental point cloud and the region center. Exemplarily, the point cloud features of the environmental point cloud can be obtained by concatenating the point cloud coordinates and reflection intensity of the environmental point cloud, the center coordinates of the region center of the columnar region where the environmental point cloud is located, and the offset between the environmental point cloud and the region center.

[0095] S203 , determining a three-dimensional feature tensor corresponding to the columnar region according to the point cloud features and the number of point clouds of the environment point cloud contained in the columnar region and the height features of the columnar region.

[0096] The so-called point cloud quantity is the quantity of environmental point clouds contained in the columnar area.

[0097] Optionally, for each columnar area, the point cloud features and the number of point clouds of the environmental point cloud contained in the columnar area, as well as the height features of the columnar area, can be input into a trained tensor determination model. The tensor determination model outputs the three-dimensional feature tensor corresponding to the columnar area based on the point cloud features, the number of point clouds, the height features and the model parameters of each environmental point cloud.

[0098] Alternatively, for each environmental point cloud in each columnar area, the point cloud features of the environmental point cloud, the number of point clouds in the columnar area where the environmental point cloud is located, and the height features can be combined to obtain the point cloud tensor of the environmental point cloud; then, the point cloud tensors of each environmental point cloud in any columnar area are combined to obtain the three-dimensional feature tensor corresponding to the columnar area.

[0099] It can be understood that, when the number of environmental point clouds in each columnar area has been uniformly processed, the number of point clouds in each columnar area in step S203 is the standard number of point clouds.

[0100] In an embodiment of the present application, the rationality of the determination of the three-dimensional feature tensor can be ensured by determining the three-dimensional feature tensor corresponding to the columnar area based on the point cloud features and the number of point clouds of the environmental point cloud contained in the columnar area, as well as the height features of the columnar area.

[0101] In order to further ensure the accuracy of the three-dimensional feature tensor, based on the above embodiment, in the embodiment of the present application, another optional method for determining the three-dimensional feature tensor is provided, such as Figure 3 As shown, the specific steps include:

[0102] S301, for each environmental point cloud in the columnar area, the height features and the point cloud features of the environmental point cloud are spliced ​​to obtain the environmental features of the environmental point cloud, and the environmental features, the number of point clouds in the columnar area, and the number of columnar areas containing environmental point clouds in the target area are spliced ​​to obtain the point cloud tensor of the environmental point cloud.

[0103] Among them, the so-called environmental features are the features of the environmental point cloud in multiple dimensions such as height dimension, coordinate dimension, reflection dimension and offset dimension.

[0104] Optionally, for each environmental point cloud in the columnar region, the height feature of the columnar region where the environmental point cloud is located and the point cloud feature of the environmental point cloud can be spliced ​​to obtain the environmental feature of the environmental point cloud. For example, when the coordinates of environmental point cloud A are (x, y, z), the reflection intensity is r, the height feature of the columnar region where the environmental point cloud is located is s, and the center coordinates are (x c ,y c , z c ), the offset is (x p ,y p ), the point cloud features of the environment point cloud A are (x, y, z, x c ,y c , z c , x p ,y p , r), the environmental characteristics are (x, y, z, x c ,y c , z c , x p ,y p ,s,r), that is, the environmental characteristics are ten-dimensional data.

[0105] Furthermore, the number of columnar regions containing the environmental point cloud can be counted before unifying the number of point clouds in each columnar region. Then, the environmental features of the environmental point cloud, the number of point clouds in the columnar region, and the number of regions can be spliced ​​to obtain the point cloud tensor of the environmental point cloud. For example, in the case of environmental point cloud A with environmental features D A , when the number of point clouds is N and the number of regions is S, the point cloud tensor of the environment point cloud A is (P, D A , N).

[0106] S302 , combining the point cloud tensors of each environmental point cloud contained in the columnar region to obtain a three-dimensional feature tensor corresponding to the columnar region.

[0107] Optionally, after determining the point cloud tensor for each environmental point cloud, for each columnar region, the point cloud tensors of each environmental point cloud within that columnar region can be combined to obtain the 3D feature tensor corresponding to that columnar region. For example, the 3D feature tensor for the columnar region containing environmental point cloud A is (P, D, N), where D is the feature combination of the environmental features corresponding to each environmental point cloud within the columnar region containing environmental point cloud A.

[0108] In an embodiment of the present application, the environmental features are obtained by splicing the height features and point cloud features corresponding to the environmental point cloud, and the environmental features, the number of point clouds in the columnar area, and the number of columnar areas containing environmental point clouds in the target area are spliced ​​to obtain a point cloud tensor of the environmental point cloud. Then, the point cloud tensors of each environmental point cloud contained in the columnar area are combined to obtain a three-dimensional feature tensor corresponding to the columnar area, which can ensure the rationality of the three-dimensional feature tensor.

[0109] In order to ensure the rationality of the columnar area division, based on the above embodiment, an optional method of dividing the columnar area is provided in the embodiment of the present application. Specifically, the ground size corresponding to the target area is evenly divided to obtain the ground size corresponding to the columnar area; the area height corresponding to the target area is evenly divided to obtain the area height corresponding to the columnar area; and at least two columnar areas of the same volume are generated according to the ground size and the area height.

[0110] The so-called ground size refers to the size information of the two-dimensional space where the ground is located; the so-called regional height refers to the height information of each space.

[0111] Alternatively, the ground dimensions corresponding to the target area can be evenly divided based on a preset plane step size to obtain the ground dimensions corresponding to the columnar areas. For example, if the ground dimensions corresponding to the target area are X and Y, and the preset plane step sizes are ΔX and ΔY, the ground divisions for each columnar area can be determined by the formula P = {X / ΔX, Y / ΔY}, where P is the set of ground divisions for each columnar area.

[0112] Furthermore, a preset height threshold may be used to evenly divide the region height corresponding to the target region, thereby obtaining the region height corresponding to each columnar region.

[0113] After determining the ground division and height division of each columnar area, at least two columnar areas with the same volume can be generated. Figure 4 Schematic diagram of the columnar area shown.

[0114] In the embodiment of the present application, by generating at least two columnar regions of the same volume according to the ground size and the region height, the accuracy of the columnar region division can be ensured.

[0115] In order to ensure the accuracy of the environmental information determination, based on the above embodiment, in the embodiment of the present application, an optional method for determining the environmental information is provided, such as Figure 5 As shown, the specific steps include:

[0116] S501 : Project the three-dimensional feature tensor corresponding to each columnar region into a pseudo image.

[0117] Optionally, the three-dimensional feature tensors corresponding to the columnar regions may be arranged according to the positions of the columnar regions in the target region, and the arranged three-dimensional feature tensors may be projected into a pseudo image.

[0118] For example, the pseudo image H×W can be determined based on the environmental features of each environmental point cloud in each columnar region with reference to the following formula (1). Here, H and W are the number of rows and columns of the original grid, corresponding to the XY plane grid division; each pixel in the pseudo image contains a multi-dimensional feature vector (i.e., environmental features including height features).

[0119] (1)

[0120] in, is the pixel value of the pseudo image at pixel (X, Y); F i is the environmental feature of the environmental point cloud i in the columnar area.

[0121] For example, reference Figure 6 As shown in the pseudo image generation diagram, after determining the 3D feature tensor (P, D, N) for each columnar region, a fully connected layer is used to extract local features from the environmental features of each environmental point cloud within the columnar region (e.g., from 9 to 64 dimensions). Maximum pooling is then performed on the local features of all environmental point clouds within the same columnar region to obtain a columnar region-level feature (64 dimensions). The pooled columnar region-level feature is then concatenated with the local features of each environmental point cloud (64 + 64 = 128 dimensions), and a fully connected layer is used to compress the concatenated features to a fixed length (e.g., C = 64 dimensions). This converts the 3D feature tensor corresponding to the columnar region into a C-dimensional feature vector.

[0122] Furthermore, the C-dimensional feature vectors of all columnar regions are arranged according to the original grid positions to form a pseudo image. The pseudo image size is: H × W × C (H and W are the number of rows and columns in the original grid, corresponding to the XY plane grid division, and C is the feature dimension).

[0123] S502: Perform convolution processing on the pseudo image to obtain environmental information in the target area.

[0124] Optionally, a backbone network (Backbone) can be used to perform convolution processing on the pseudo image using a two-dimensional convolutional neural network to extract multi-scale features, thereby obtaining environmental information within the target area.

[0125] For example, the encoded pseudo image is fed into a convolutional neural network to learn spatial features. This process often uses the convolution operation shown in the following formula (2) for feature extraction. Through the convolution layer, the network can capture the spatial features in the pseudo image, including subtle differences in height information, and enhance the detection effect of the target.

[0126] (2)

[0127] Among them, O(x,y) is the output after convolution; i(x,y) is the input pseudo image; K(x,y) is the convolution kernel, and K is the convolution kernel size.

[0128] For example, reference Figure 7 As shown in the convolution processing diagram, the pseudo image is input into the convolution layer Block 1. Block 1 adopts 3×3 convolution and performs batch normalization (BatchNorm) and activation (Rectified LinearUnit, ReLU) on the pseudo image, which can capture the local details of the pseudo image (such as object edges) and obtain the first output data; the first output data is input into the convolution layer Block 2. Block 2 adopts 3×3 convolution and performs batch normalization and activation on the pseudo image, which can extract the medium-scale features of the pseudo image (such as object parts) and obtain the second output data; the second output data is input into the convolution layer Block 3. Block 3 adopts 3×3 convolution and performs batch normalization and activation on the pseudo image, which can extract the global semantic features of the pseudo image (such as the object as a whole).

[0129] Furthermore, the resolution can be restored through transposed convolution (deconvolution) and fused with shallow features to generate a multi-scale feature map. Specifically, the deep features are input into upsampling Block 1, which deconvolve the deep features to obtain first deconvolution data, and then perform element-by-element addition / channel concatenation of the first deconvolution data and the second output data output by Block 2 to obtain first processed data, which can fuse deep semantics with mid-level details; the first processed data is input into upsampling Block 2, which deconvolve the first processed data to obtain second deconvolution data, and then perform feature fusion of the second deconvolution data and the first output data output by Block 1 to obtain the final output data (environmental information), which is used to fuse mid-level details with shallow high-resolution features.

[0130] In an embodiment of the present application, the three-dimensional feature tensor corresponding to each columnar area is projected into a pseudo image, and the pseudo image is convolved to obtain the environmental information in the target area, thereby ensuring the accuracy of the environmental information determination.

[0131] Figure 8 This is a flow chart of a point cloud processing method in another embodiment. Based on the above embodiment, this embodiment provides an optional example of a point cloud processing method. Figure 8 The specific implementation process is as follows:

[0132] S801 , after obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two columnar areas of equal volume.

[0133] Optionally, the ground size corresponding to the target area is evenly divided to obtain the ground size corresponding to the columnar area; the area height corresponding to the target area is evenly divided to obtain the area height corresponding to the columnar area; and at least two columnar areas of the same volume are generated based on the ground size and the area height.

[0134] S802: For each columnar region, determine the height features corresponding to different types of objects in the columnar region according to the distribution of the environmental point cloud contained in the columnar region, and perform weighted processing on the height features corresponding to different types of objects to obtain the height features of the columnar region.

[0135] S803: Determine the center coordinates of the area center corresponding to each columnar area according to the position information of each columnar area.

[0136] S804, determining the point cloud features of the environmental point cloud contained in each columnar area based on the point cloud coordinates and reflection intensity of the environmental point cloud contained in each columnar area, the center coordinates of the area center, and the offset between the environmental point cloud contained in each columnar area and the area center.

[0137] S805 , determining a three-dimensional feature tensor corresponding to each columnar region based on the point cloud features and the number of point clouds of the environment point clouds contained in each columnar region, and the height features of the columnar region.

[0138] Optionally, for each environmental point cloud in the columnar area, the height features and the point cloud features of the environmental point cloud are spliced ​​to obtain the environmental features of the environmental point cloud, and the environmental features, the number of point clouds in the columnar area, and the number of columnar areas containing environmental point clouds in the target area are spliced ​​to obtain a point cloud tensor of the environmental point cloud; the point cloud tensors of each environmental point cloud contained in the columnar area are combined to obtain a three-dimensional feature tensor corresponding to the columnar area.

[0139] S806: Project the three-dimensional feature tensor corresponding to each columnar region into a pseudo image.

[0140] S807: Perform convolution processing on the pseudo image to obtain environmental information within the target area.

[0141] The specific process of the above S801-S807 can be found in the description of the above method embodiment. The implementation principle and technical effects are similar and will not be repeated here.

[0142] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0143] Based on the same inventive concept, embodiments of the present application also provide a point cloud processing device for implementing the aforementioned point cloud processing method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more point cloud processing device embodiments provided below can be found in the above-described limitations of the point cloud processing method and will not be further elaborated here.

[0144] In an exemplary embodiment, Figure 9 As shown, a point cloud processing device 1 is provided, comprising: a region division module 10, a feature determination module 20, a tensor determination module 30 and an information determination module 40, wherein:

[0145] The region division module 10 is configured to obtain an environmental point cloud within a target region where the vehicle is located and then divide the target region into at least two columnar regions of equal volume;

[0146] A feature determination module 20 is configured to determine a height feature of each columnar region based on an environmental point cloud contained in each columnar region;

[0147] A tensor determination module 30 is configured to determine a three-dimensional feature tensor corresponding to each columnar region based on the point cloud features and the number of point clouds of the environment point clouds contained in each columnar region and the height features of each columnar region;

[0148] The information determination module 40 is used to determine the environmental information in the target area according to the three-dimensional feature tensor corresponding to each columnar area.

[0149] In an exemplary embodiment, the feature determination module 20 is specifically configured to:

[0150] For each columnar area, the height features corresponding to different types of objects in the columnar area are determined according to the distribution of the environmental point cloud contained in the columnar area; the height features corresponding to different types of objects are weighted to obtain the height features of the columnar area.

[0151] In an exemplary embodiment, the tensor determination module 30 includes:

[0152] A first determining unit is configured to determine, for each columnar region, the center coordinates of the region center corresponding to the columnar region according to the position information of the columnar region;

[0153] a second determining unit, configured to determine point cloud features of each environmental point cloud contained in the columnar region based on the point cloud coordinates and reflection intensity of each environmental point cloud contained in the columnar region, the center coordinates of the region center, and the offset between each environmental point cloud contained in the columnar region and the region center;

[0154] The third determining unit is used to determine a three-dimensional feature tensor corresponding to the columnar area according to the point cloud features and the number of point clouds of the environment point cloud contained in the columnar area and the height features of the columnar area.

[0155] In an exemplary embodiment, the third determining unit is specifically configured to:

[0156] For each environmental point cloud in the columnar area, the height features and the point cloud features of the environmental point cloud are spliced ​​to obtain the environmental features of the environmental point cloud, and the environmental features, the number of point clouds in the columnar area, and the number of columnar areas containing environmental point clouds in the target area are spliced ​​to obtain the point cloud tensor of the environmental point cloud; the point cloud tensors of each environmental point cloud contained in the columnar area are combined to obtain the three-dimensional feature tensor corresponding to the columnar area.

[0157] In an exemplary embodiment, the region division module 10 is specifically configured to:

[0158] The ground size corresponding to the target area is evenly divided to obtain the ground size corresponding to the columnar area; the area height corresponding to the target area is evenly divided to obtain the area height corresponding to the columnar area; and at least two columnar areas with the same volume are generated according to the ground size and the area height.

[0159] In an exemplary embodiment, the information determination module 40 is specifically configured to:

[0160] The three-dimensional feature tensor corresponding to each columnar area is projected into a pseudo image; the pseudo image is convolved to obtain the environmental information in the target area.

[0161] Each module in the aforementioned point cloud processing device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0162] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 10 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means. The wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a point cloud processing method. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0163] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0164] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0167] It should be noted that the data involved in this application (including but not limited to environmental point cloud data, etc.) are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0168] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0169] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A point cloud processing method, characterized in that: The method comprises: After obtaining the environmental point cloud within the target area where the vehicle is located, the target area is divided into at least two columnar areas of equal volume; Determine the height characteristics of each columnar area based on the environmental point cloud contained in each columnar area; Determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and point cloud quantity of the environment point cloud contained in each columnar area, as well as the height features of each columnar area; The environmental information within the target area is determined according to the three-dimensional feature tensor corresponding to each columnar area.

2. The method according to claim 1, characterized in that The step of determining the height characteristics of each columnar region based on the environmental point cloud contained in each columnar region includes: For each columnar region, determining height features corresponding to different types of objects within the columnar region based on the distribution of the environmental point cloud contained within the columnar region; The height features corresponding to different types of objects are weighted to obtain the height features of the columnar region.

3. The method according to claim 1, characterized in that The step of determining the three-dimensional feature tensor corresponding to each columnar region based on the point cloud features and the number of point clouds contained in each columnar region and the height features of each columnar region includes: For each columnar region, determining the center coordinates of the region center corresponding to the columnar region according to the position information of the columnar region; Determining point cloud features of each environmental point cloud contained in the columnar region based on the point cloud coordinates and reflection intensity of each environmental point cloud contained in the columnar region, the center coordinates of the center of the region, and the offset between each environmental point cloud contained in the columnar region and the center of the region; A three-dimensional feature tensor corresponding to the columnar region is determined according to the point cloud features and the number of point clouds of the environment point cloud contained in the columnar region and the height features of the columnar region.

4. The method according to claim 3, characterized in that The determining of a three-dimensional feature tensor corresponding to the columnar region based on the point cloud features and the number of point clouds of the environment point cloud contained in the columnar region and the height features of the columnar region includes: For each environmental point cloud within the columnar region, concatenating the height feature with the point cloud feature of the environmental point cloud to obtain an environmental feature of the environmental point cloud, and concatenating the environmental feature, the number of point clouds within the columnar region, and the number of columnar regions within the target region containing environmental point clouds to obtain a point cloud tensor of the environmental point cloud; The point cloud tensors of each environmental point cloud contained in the columnar region are combined to obtain a three-dimensional feature tensor corresponding to the columnar region.

5. The method according to claim 1, wherein The step of dividing the target area into at least two columnar areas of equal volume comprises: Evenly dividing the ground size corresponding to the target area to obtain the ground size corresponding to the columnar area; Evenly dividing the region height corresponding to the target region to obtain the region height corresponding to the columnar region; At least two columnar regions with the same volume are generated according to the ground size and the region height.

6. The method according to claim 1, characterized in that The determining of the environmental information within the target area according to the three-dimensional feature tensor corresponding to each columnar area includes: Project the three-dimensional feature tensor corresponding to each columnar area into a pseudo image; The pseudo image is subjected to convolution processing to obtain environmental information within the target area.

7. A point cloud processing device, characterized in that: The device comprises: A region division module, configured to, after acquiring an environmental point cloud within a target region where the vehicle is located, divide the target region into at least two columnar regions of equal volume; A feature determination module is used to determine the height feature of each columnar area based on the environmental point cloud contained in each columnar area; A tensor determination module is used to determine the three-dimensional feature tensor corresponding to each columnar area based on the point cloud features and the number of point clouds of the environment point clouds contained in each columnar area, as well as the height features of each columnar area; The information determination module is used to determine the environmental information within the target area according to the three-dimensional feature tensor corresponding to each columnar area.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images