A feature enhancement method and system for laser radar point cloud data

Abstract

The present invention relates to a feature enhancement method for laser radar point cloud data, comprising: acquiring point cloud data in a target environment; performing voxelization processing on the point cloud data to obtain voxelized data; performing feature extraction on each voxel using a three-dimensional sparse convolution feature extraction network to obtain voxel features of multiple scales; calculating the weight between each voxel and each point in the point cloud data to obtain the inverse distance weight between the voxel and the point; interpolating the voxel feature to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weight between the voxel and the point to obtain point features of multiple scales for each point; for each point, using a feature weight learning network to learn the weights corresponding to the point features of each scale to obtain the point feature weights of each scale; using the point feature weights of each scale to fuse and supplement the point features of each scale to obtain enhanced point features of multiple scales, fusing point features at different scales, enhancing the point features, and realizing enhancement of the point cloud data.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and autonomous driving, and in particular to a feature enhancement method and system for laser radar point cloud data. Background Art

[0002] In the past decade, computer vision technology has attracted a lot of attention. As an important application of computer vision technology, environmental perception technology in the field of autonomous driving has been widely studied in recent years and has become a research hotspot. Cameras and lidar are the most commonly used sensors on autonomous vehicles. They are used to obtain high-resolution images and three-dimensional lidar point clouds of the vehicle's surroundings, respectively. In the early years, high-resolution images were the most commonly used information in the field of computer vision, but with the continuous improvement of the requirements for autonomous driving (such as the need to accurately detect the three-dimensional spatial information of the target), two-dimensional image information can no longer meet its needs. Three-dimensional lidar point cloud data has a powerful ability to capture depth information, which has attracted more and more attention in the field of computer vision and autonomous driving. Based on three-dimensional lidar point cloud data, the accuracy of some visual tasks in autonomous driving scenarios, such as semantic segmentation and three-dimensional object detection, has been greatly improved.

[0003] In practical applications, existing deep learning networks usually convert lidar point cloud data into voxels of equal volume, then use a feature extraction skeleton based on sparse convolution to extract voxel features, and finally encode the voxel features into the point cloud data to obtain point cloud features. The point voxel region convolutional neural network (PV-RCNN) proposed by Shaoshuai Shi, Chaoxu Guo, Jiang Li, etc. encodes multi-scale voxel features into point cloud data and concatenates them with the help of the point set abstraction algorithm (Set Abstraction), but the point set abstraction algorithm will cause a lot of computational consumption and video memory occupancy; the structure-aware single-stage detector (SA-SSD) proposed by Chenhang He, Hui Zeng, Jianqiang Huang, etc. encodes voxel features into point cloud data and concatenates them. This method has low video memory occupancy, but only using the interpolation method cannot obtain discriminative point cloud features. In general, existing methods focus on how to learn and encode point cloud features, and do not consider the enhancement of point cloud features. Therefore, how to use appropriate technology to enhance point cloud features and generate discriminative point cloud features is an important problem that needs to be solved in the field of autonomous driving. Summary of the invention

[0004] The purpose of the present invention is to provide a feature enhancement method and system for lidar point cloud data, which can effectively fuse multi-scale information and obtain deep point features for each point in the point cloud data, obtain discriminative point feature information, and thus obtain enhanced features of the point cloud data, which is beneficial to image detection in autonomous driving scenarios.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A feature enhancement method for laser radar point cloud data, comprising:

[0007] Acquire point cloud data in the target environment;

[0008] Performing voxel processing on the point cloud data to obtain voxelized data of the point cloud;

[0009] For each voxel, according to the voxelized data, a three-dimensional sparse convolutional feature extraction network is used to perform feature extraction to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network;

[0010] Calculate the weight between each voxel and each point in the point cloud data based on the inverse distance weight method to obtain the inverse distance weight between the voxel and the point;

[0011] According to the inverse distance weights between the voxels and the points, the voxel features are interpolated to each point of the point cloud data using a cubic linear interpolation method to obtain point features of multiple scales for each point; the number of scales of the point features corresponds one to one to the number of scales of the voxel features;

[0012] For each point, a feature weight learning network is used to learn the weights corresponding to the point features of each scale, and the point feature weights of each scale are obtained;

[0013] For each point, the point features at each scale are fused and supplemented using the point feature weights at each scale to obtain enhanced point features at multiple scales, thereby enhancing the point cloud data.

[0014] A system for feature enhancement method of laser radar point cloud data, comprising:

[0015] Point cloud data acquisition module, used to acquire point cloud data in the target environment;

[0016] A voxelized data acquisition module, used for performing voxelization processing on the point cloud data to obtain voxelized data of the point cloud;

[0017] A feature extraction module, configured to extract features of each voxel using a three-dimensional sparse convolutional feature extraction network according to the voxelized data, so as to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network;

[0018] An inverse distance weight calculation module, used to calculate the weight between each voxel and each point in the point cloud data based on an inverse distance weight method, to obtain an inverse distance weight between the voxel and the point;

[0019] A feature interpolation module, used to interpolate the voxel feature to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weight between the voxel and the point, so as to obtain point features of multiple scales for each point; the scale number of the point feature corresponds to the scale number of the voxel feature one by one;

[0020] A learning weight module is used to learn the weights corresponding to the point features of each scale for each point using a feature weight learning network to obtain the point feature weights of each scale;

[0021] The enhanced feature acquisition module is used to fuse and supplement the point features of each scale using the point feature weights of each scale for each point, so as to obtain enhanced point features of multiple scales, thereby enhancing the point cloud data.

[0022] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0023] The present invention provides a feature enhancement method and system for laser radar point cloud data, comprising: acquiring point cloud data in a target environment; performing voxelization processing on the point cloud data to obtain voxelized data of the point cloud; for each voxel, according to the voxelized data, using a three-dimensional sparse convolutional feature extraction network to perform feature extraction to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network; calculating the weight between each voxel and each point in the point cloud data based on an inverse distance weighted method, and obtaining the weight between the voxel and the point. Inverse distance weight; based on the inverse distance weight between the voxel and the point, the voxel feature is interpolated to each point of the point cloud data using a cubic linear interpolation method to obtain point features of multiple scales for each point; the number of scales of the point feature corresponds one-to-one to the number of scales of the voxel feature; for each point, the weights corresponding to the point features of each scale are learned using a feature weight learning network to obtain point feature weights of each scale; the point features of each scale are fused and supplemented using the point feature weights of each scale to obtain enhanced point features of multiple scales, thereby enhancing the point cloud data. The present invention does not use a point set abstraction algorithm, which effectively reduces computational consumption and video memory occupancy; compared with existing methods that only use interpolation algorithms, the present invention can effectively fuse multi-scale information and obtain deep point cloud features, and obtain discriminative point cloud feature information, providing a good foundation for various prediction tasks based on point cloud data in autonomous driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flow chart of a feature enhancement method for laser radar point cloud data provided in Example 1 of the present invention;

[0026] Figure 2 The original image captured and provided in Embodiment 1 of the present invention;

[0027] Figure 3 The laser radar point cloud data corresponding to the original image provided in Example 1 of the present invention;

[0028] Figure 4 A structural diagram of a feature weight learning network provided in Example 1 of the present invention;

[0029] Figure 5 A feature enhancement process diagram provided in Example 1 of the present invention;

[0030] Figure 6 A block diagram of a feature enhancement system for lidar point cloud data provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The purpose of the present invention is to provide a feature enhancement method and system for lidar point cloud data, which enhances the features of each point in the point cloud to overcome the problems of high computational consumption and low feature discrimination of each point in the point cloud in the actual application of the prior art.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Example 1

[0035] like Figure 1 As shown, this embodiment provides a feature enhancement method for laser radar point cloud data, including:

[0036] S1: Acquire point cloud data in the target environment;

[0037] Use the vehicle-mounted 3D laser radar to scan the surrounding environment and obtain point cloud data of the surrounding environment;

[0038] The method of the present invention is further illustrated using the most commonly used and authoritative KITTI autonomous driving dataset in the field of autonomous driving. The data of this dataset is collected from two grayscale cameras, two color cameras and a laser radar. Take the laser radar point cloud data of sample No. 7312 in the training set of the KITTI autonomous driving dataset as an example. Figure 2 The original image captured is shown in Figure 1, with a resolution of 1242×375. Figure 3 The corresponding lidar point cloud data is shown, which contains 18656 points.

[0039] S2: performing voxel processing on the point cloud data to obtain voxelized data of the point cloud;

[0040] Wherein, step S2 specifically includes:

[0041] Setting the size of each voxel according to the range of the point cloud data;

[0042] Dividing the three-dimensional space where the point cloud data is located into a plurality of small spaces of equal volume according to the size of each voxel to obtain a plurality of voxels;

[0043] Each non-empty voxel is initially encoded using the initial features of the point cloud data to obtain the initial features of the non-empty voxels, that is, voxelized data of the point cloud.

[0044] The initial encoding of each non-empty voxel using the initial features of the point cloud data specifically includes:

[0045] For each of the non-empty voxels, determining whether the point exists in the non-empty voxel, and obtaining a determination result;

[0046] When the judgment result is yes, the average of the initial features of all the points in the non-empty voxel is calculated, and the average is used as the initial feature of the non-empty voxel.

[0047] When the judgment result is no, the non-empty voxel is not encoded.

[0048] The range of the point cloud space in depth, width and height dimensions is set between [0, -40, -1] meters and [70.4, 40, 3] meters, and the size of the generated voxel in depth, width and height dimensions is [0.05, 0.05, 0.1] meters. The three-dimensional voxels are generated according to the above size, and the resolution of the generated three-dimensional voxels is 1408×1600×40. Considering the computational consumption and video memory occupancy, the number of initial non-empty voxels is limited to 20,000 in this embodiment. If the number of non-empty voxels is more than 20,000, the voxels with more than 20,000 are removed. If the number of non-empty voxels is less than 20,000, the actual number of non-empty voxels is used.

[0049] S3: For each voxel, according to the voxelized data, a three-dimensional sparse convolutional feature extraction network is used to perform feature extraction to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network; wherein, Table 1 shows the three-dimensional sparse convolutional feature extraction network structure and its parameters.

[0050] Table 1 Three-dimensional sparse convolutional feature extraction network structure and its parameters

[0051]

[0052] Specifically, the three-dimensional sparse convolutional feature extraction network includes four sparse convolutional blocks, which are sequentially recorded as a first sparse convolutional block, a second sparse convolutional block, a third sparse convolutional block and a fourth sparse convolutional block;

[0053] The first sparse convolution block includes two first-manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the first first-manifold sparse convolution layer are 4 and 16 respectively, and the input and output feature channel sizes of the second first-manifold sparse convolution layer are 16 and 16 respectively;

[0054] The second sparse convolution block includes a first sparse convolution layer with a convolution kernel size of 3 and two second manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the first sparse convolution layer are 16 and 32 respectively, and the input and output feature channel sizes of the two second manifold sparse convolution layers are both 32 and 32;

[0055] The third sparse convolution block includes a second sparse convolution layer with a convolution kernel size of 3 and two third manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the second sparse convolution layer are 32 and 64 respectively, and the input and output feature channel sizes of the two third manifold sparse convolution layers are both 64 and 64;

[0056] The fourth sparse convolution block includes a third sparse convolution layer with a convolution kernel size of 3 and two fourth manifold sparse convolution layers with a convolution kernel size of 3. The input and output feature channel sizes of the third sparse convolution layer and the two fourth manifold sparse convolution layers are both 64 and 64.

[0057] Based on the three-dimensional sparse convolutional feature extraction network in Table 1, the generated voxels are feature extracted to obtain voxel features at multiple scales of the voxels. and That is, voxel features of 4 scales (h=4), among which N1=15324, N2=26561, N3=22623, N4=11073, that is, k=Ni. After each sparse convolution or manifold sparse convolution, batch normalization operation is performed and activated using the ReLU activation function.

[0058] S4: calculating the weight between each voxel and each point in the point cloud data based on an inverse distance weight method to obtain an inverse distance weight between the voxel and the point;

[0059] Wherein, step S4 specifically includes:

[0060]

[0061] Among them, v k represents the kth voxel, k = 1, 2, ..., m; η(v k ) represents the three-dimensional coordinates of the k-th voxel; p j represents the jth point, j = 1, 2, ..., n; η(p j) represents the three-dimensional coordinates of the jth point; r itp Represents the search radius of the IDW method.

[0062] S5: interpolating the voxel feature to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weight between the voxel and the point, to obtain point features of multiple scales for each point; the number of scales of the point feature corresponds one to one to the number of scales of the voxel feature;

[0063] Specifically, step S5 specifically includes:

[0064]

[0065] Wherein, i represents the scale value, i=1,2,...,h.

[0066] For steps S4 and S5, the voxel features are processed using a cubic linear interpolation method based on an inverse distance weighted method to convert the voxel position features and Interpolate to the location of the point cloud data to obtain the point cloud features and In this example, j=18656.

[0067] S6: For each point, use a feature weight learning network to learn the weights corresponding to the point features of each scale, and obtain the point feature weights of each scale;

[0068] Specifically, step S6 specifically includes:

[0069] Inputting the point features of each scale into the feature weight learning network to obtain the weights of the point features of each scale on each component, recorded as component weights;

[0070] Obtaining an adjusted point feature at each scale according to each feature component of the point feature at each scale and the corresponding component weight;

[0071] Adding each feature component of the adjustment point feature at each scale to obtain a feature sum;

[0072] Each of the features is compressed to obtain the point feature weight of each scale.

[0073] Since the feature components in the point features are not equally important, a feature weight learning network based on traditional convolution is constructed to adjust the point cloud feature components, and then obtain the weights of each point in the point cloud data. The network structure and its parameters are shown in Table 2. After each convolution, batch normalization is performed and ReLU activation function is used for activation. The feature weight learning network structure is as follows: Figure 4 shown.

[0074] Table 2 Feature weight learning network and its parameters

[0075] Traditional convolution feature input and output sizes Convolution kernel size Downsampling scale Convolutional Block 1 (16，64)，(64，64) 1 1 Convolutional Block 2 (32，64)，(64，64) 1 2 Convolutional Block 3 (64，64)，(64，64) 1 4 Convolutional Block 4 (64，64)，(64，64) 1 8

[0076] Input the features of each point in the point cloud at different scales obtained in the previous steps Figure 4 The feature weight learning network shown in Table 2 first adjusts the features of each point to the same number of channels. and Then, the softmax function is used to process the point features and obtain the weights of each component. and Multiply each component of the point feature adjusted to the same number of channels by the weight of the point feature on each component to obtain the adjusted point feature As shown in the following formula:

[0077]

[0078] Then add the components of the adjusted point features of each point to get the feature sum of each point at a certain scale. It is used to represent the information contained in each point, and then the sigmoid function is used to sum the features Compress to the [0-1] interval to get the weight of each point in the point cloud data and As shown in the formula:

[0079]

[0080] S7: For each point, the point features at each scale are fused and supplemented using the point feature weights at each scale to obtain enhanced point features at multiple scales, thereby enhancing the point cloud data.

[0081] Wherein, step S7 specifically includes:

[0082]

[0083] in, represents the enhanced point feature of the i-th scale, i = 1, 2, ..., h; represents the point feature weight of the i-th scale of the j-th point, j = 1, 2, ..., n; Represents the point feature of the i-th scale of the j-th point; represents the point feature weight of the g-th scale of the j-th point; Represents the point feature of the g-th scale of the j-th point.

[0084] by Take the enhancement process as an example. Figure 5 As shown, the weight of each point in the point cloud data is used and Point cloud features To integrate and supplement, As a soft mask control and The features of each point flow into the enhanced features Specifically, if the weight of the jth point in the point cloud is is larger, indicating that the feature of this point contains more information at this scale. Most of the features will flow into the enhanced features. will be smaller, which will inhibit and The features in flow into the enhanced features; on the contrary, if If it is smaller, it means that the feature of this point contains less information at this scale. will be suppressed from flowing into the enhancement feature, and in contrast, will be larger, and Some features in will flow into the enhanced features as supplements. The feature enhancement process described in this step is shown in the formula: Similarly, the enhanced features can be obtained and Finally, the enhanced features and The final point cloud enhancement feature ef is connected in series as shown in the formula:

[0085] This embodiment uses the authoritative public autonomous driving dataset KITTI dataset as the data source, and divides the 7481 training samples into a training set containing 3712 samples and a validation set containing 3769 samples. The same experimental environment (Intel Xeon Silver CPU, NVIDIA RTX-2080Ti graphics card, 64GB memory) is used for experiments. In order to illustrate and quantify the effect of the method described in the present invention, the above-mentioned point cloud enhancement feature ef is used in this embodiment to enhance the point cloud features in the SA-SSD three-dimensional target detector. Specifically, two groups of experiments are conducted in this embodiment: one group of experiments uses the training set to train the original version of the SA-SSD three-dimensional target detection network; the other group of experiments adds the feature enhancement network described in the present invention to the network structure of SA-SSD, uses the point cloud feature enhancement method described in the present invention to enhance the point cloud features obtained in SA-SSD, and uses the training set to train the SA-SSD with enhanced point cloud features. In this embodiment, the batch size is 2, the learning rate is 0.003, and the training cycle is 50. The car detection accuracy results of the two versions of SA-SSD on the validation set samples are shown in Table 3, where the cars in the validation set are divided into three levels of difficulty: easy, medium, and difficult according to the size of the car bounding box and the degree of occlusion. Table 3 uses the average accuracy to evaluate the car detection results under the three-dimensional perspective. If the intersection between the final detected car bounding box and the true bounding box is greater than 70%, the detection is considered correct, otherwise it is wrong.

[0086] Table 3 3D object detection results of SA-SSD and point cloud feature enhanced SA-SSD

[0087]

[0088] As shown in Table 3, the training time of the point cloud feature enhanced SA-SSD increased by 48 minutes compared to the training time of the original SA-SSD, that is, the operation time of each image only increased by 48*60 / 50 / 3769=0.016 (seconds). The use of the point cloud feature enhancement method of the present invention improves the overall accuracy of SA-SSD's three-dimensional target detection by 0.26%, of which the simple difficulty car increased by 0.25%, the medium difficulty car increased by 0.28%, and the difficult difficulty car increased by 0.26%. It can be seen that the point cloud feature enhancement method of the present invention has a positive effect on improving the performance of the three-dimensional target detection network, and also reflects that the enhanced point cloud features generated by the method of the present invention have higher discrimination than the original features.

[0089] In this embodiment, a three-dimensional laser radar is used to obtain point cloud data of the surrounding environment; then, it is voxelized and converted into three-dimensional voxels of equal volume, and three-dimensional sparse convolution is used to extract features from the voxels; then, the cubic linear interpolation method is used to interpolate the features from the voxel location to the point cloud location to obtain point cloud features; again, the weights of the point cloud features are learned using traditional convolution; finally, the point cloud features at multiple scales are fused and enhanced using the weights of the point cloud features to obtain discriminative point cloud features. By fusing point cloud features at different scales and enhancing the point cloud features, discriminative enhanced point cloud features can be obtained compared to existing methods. In addition, the enhancement method described in the present invention does not use a point set abstraction algorithm, and has low computational consumption and video memory occupancy, providing a good foundation for computer vision tasks based on three-dimensional laser radar point cloud data in autonomous driving scenarios.

[0090] Example 2

[0091] like Figure 6 As shown, this embodiment provides a system for feature enhancement of laser radar point cloud data, including:

[0092] Point cloud data acquisition module T1, used to acquire point cloud data in the target environment;

[0093] A voxelized data acquisition module T2 is used to perform voxelization processing on the point cloud data to obtain voxelized data of the point cloud;

[0094] A feature extraction module T3 is used to extract features for each voxel using a three-dimensional sparse convolutional feature extraction network according to the voxelized data to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network;

[0095] An inverse distance weight calculation module T4 is used to calculate the weight between each voxel and each point in the point cloud data based on an inverse distance weight method to obtain an inverse distance weight between the voxel and the point;

[0096] A feature interpolation module T5 is used to interpolate the voxel feature to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weight between the voxel and the point, so as to obtain point features of multiple scales for each point; the scale number of the point feature corresponds to the scale number of the voxel feature one by one;

[0097] The learning weight module T6 is used to learn the weights corresponding to the point features of each scale for each point using a feature weight learning network to obtain the point feature weights of each scale;

[0098] Among them, the learning weight module T6 specifically includes:

[0099] A component weight acquisition unit, used to input the point features of each scale into the feature weight learning network to obtain the weight of the point features of each scale on each component, which is recorded as component weight;

[0100] An adjustment point feature acquisition unit, configured to obtain an adjustment point feature of each scale according to each feature component of the point feature of each scale and the corresponding component weight;

[0101] A feature sum acquisition unit, configured to add the feature components of the adjustment point features at each scale to obtain a feature sum;

[0102] The point feature weight acquisition unit is used to compress each of the features and obtain the point feature weight of each scale.

[0103] The enhanced feature acquisition module T7 is used to fuse and supplement the point features of each scale for each point using the point feature weights of each scale to obtain enhanced point features of multiple scales, thereby enhancing the point cloud data.

[0104] As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0105] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A feature enhancement method for laser radar point cloud data, characterized in that: include: Acquire point cloud data in the target environment; Performing voxel processing on the point cloud data to obtain voxelized data of the point cloud; For each voxel, according to the voxelized data, a three-dimensional sparse convolutional feature extraction network is used to perform feature extraction to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network; Calculate the weight between each voxel and each point in the point cloud data based on the inverse distance weight method to obtain the inverse distance weight between the voxel and the point; According to the inverse distance weights between the voxels and the points, the voxel features are interpolated to each point of the point cloud data using a cubic linear interpolation method to obtain point features of multiple scales for each point; the number of scales of the point features corresponds one to one to the number of scales of the voxel features; For each point, a feature weight learning network is used to learn the weights corresponding to the point features of each scale, and the point feature weights of each scale are obtained; For each point, the point features at each scale are fused and supplemented using the point feature weights at each scale to obtain enhanced point features at multiple scales, thereby enhancing the point cloud data.

2. The method according to claim 1, characterized in that: The voxelization processing of the point cloud data to obtain the voxelized data of the point cloud specifically includes: Setting the size of each voxel according to the range of the point cloud data; Dividing the three-dimensional space where the point cloud data is located into a plurality of small spaces of equal volume according to the size of each voxel to obtain a plurality of voxels; Each non-empty voxel is initially encoded using the initial features of the point cloud data to obtain the initial features of the non-empty voxels, that is, voxelized data of the point cloud.

3. The method according to claim 2, characterized in that The initial encoding of each non-empty voxel using the initial features of the point cloud data specifically includes: For each of the non-empty voxels, determining whether the point exists in the non-empty voxel, and obtaining a determination result; When the judgment result is yes, calculating the mean of the initial features of all the points in the non-empty voxel, and taking the mean as the initial feature of the non-empty voxel; When the judgment result is no, the non-empty voxel is not encoded.

4. The method according to claim 1, characterized in that: The three-dimensional sparse convolutional feature extraction network includes four sparse convolutional blocks, which are sequentially recorded as a first sparse convolutional block, a second sparse convolutional block, a third sparse convolutional block and a fourth sparse convolutional block; The first sparse convolution block includes two first-manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the first first-manifold sparse convolution layer are 4 and 16 respectively, and the input and output feature channel sizes of the second first-manifold sparse convolution layer are 16 and 16 respectively; The second sparse convolution block includes a first sparse convolution layer with a convolution kernel size of 3 and two second manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the first sparse convolution layer are 16 and 32 respectively, and the input and output feature channel sizes of the two second manifold sparse convolution layers are both 32 and 32; The third sparse convolution block includes a second sparse convolution layer with a convolution kernel size of 3 and two third manifold sparse convolution layers with a convolution kernel size of 3, the input and output feature channel sizes of the second sparse convolution layer are 32 and 64 respectively, and the input and output feature channel sizes of the two third manifold sparse convolution layers are both 64 and 64; The fourth sparse convolution block includes a third sparse convolution layer with a convolution kernel size of 3 and two fourth manifold sparse convolution layers with a convolution kernel size of 3. The input and output feature channel sizes of the third sparse convolution layer and the two fourth manifold sparse convolution layers are both 64 and 64.

5. The method according to claim 1, characterized in that The method of calculating the weight between each voxel and each point in the point cloud data based on the inverse distance weight method to obtain the inverse distance weight between the voxel and the point specifically includes: Among them, v k represents the kth voxel, k = 1, 2, ..., m; η(v k ) represents the three-dimensional coordinates of the k-th voxel; p j represents the jth point, j = 1, 2, ..., n; η(p j ) represents the three-dimensional coordinates of the jth point; r itp Represents the search radius of the IDW method.

6. The method according to claim 5, characterized in that The method of interpolating the voxel features to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weights between the voxel and the point to obtain point features of multiple scales for each point specifically includes: Wherein, i represents the scale value in the first scale feature, i=1, 2, ..., h.

7. The method according to claim 1, characterized in that For each point, the weights corresponding to the point features of each scale are learned using a feature weight learning network to obtain the point feature weights of each scale, specifically including: Inputting the point features of each scale into the feature weight learning network to obtain the weights of the point features of each scale on each component, recorded as component weights; Obtaining an adjusted point feature at each scale according to each feature component of the point feature at each scale and the corresponding component weight; Adding each feature component of the adjustment point feature at each scale to obtain a feature sum; Each of the features is compressed to obtain the point feature weight of each scale.

8. The method according to claim 1, characterized in that The step of fusing and supplementing the point features at each scale by using the point feature weights at each scale to obtain enhanced point features at multiple scales specifically includes: in, represents the enhanced point feature of the i-th scale, i = 1, 2, ..., h; represents the point feature weight of the i-th scale of the j-th point, j = 1, 2, ..., n; Represents the point feature of the i-th scale of the j-th point; represents the point feature weight of the g-th scale of the j-th point; Represents the point feature of the g-th scale of the j-th point.

9. A system based on the feature enhancement method for laser radar point cloud data according to any one of claims 1 to 8, characterized in that: include: Point cloud data acquisition module, used to acquire point cloud data in the target environment; A voxelized data acquisition module, used for performing voxelization processing on the point cloud data to obtain voxelized data of the point cloud; A feature extraction module, configured to extract features of each voxel using a three-dimensional sparse convolutional feature extraction network according to the voxelized data, so as to obtain voxel features of multiple scales; the number of scales of the voxel features is determined according to the number of sparse convolutional blocks in the three-dimensional sparse convolutional feature extraction network; An inverse distance weight calculation module, used to calculate the weight between each voxel and each point in the point cloud data based on an inverse distance weight method, to obtain an inverse distance weight between the voxel and the point; A feature interpolation module, used to interpolate the voxel feature to each point of the point cloud data using a cubic linear interpolation method according to the inverse distance weight between the voxel and the point, so as to obtain point features of multiple scales for each point; the scale number of the point feature corresponds to the scale number of the voxel feature one by one; A learning weight module is used to learn the weights corresponding to the point features of each scale for each point using a feature weight learning network to obtain the point feature weights of each scale; The enhanced feature acquisition module is used to fuse and supplement the point features of each scale using the point feature weights of each scale for each point, so as to obtain enhanced point features of multiple scales, thereby enhancing the point cloud data.

10. The system according to claim 9, characterized in that The learning weight module specifically includes: A component weight acquisition unit, used to input the point features of each scale into the feature weight learning network to obtain the weight of the point features of each scale on each component, which is recorded as component weight; An adjustment point feature acquisition unit, configured to obtain an adjustment point feature of each scale according to each feature component of the point feature of each scale and the corresponding component weight; A feature sum acquisition unit, configured to add the feature components of the adjustment point features at each scale to obtain a feature sum; The point feature weight acquisition unit is used to compress each of the features and obtain the point feature weight of each scale.

Images