A laser radar point cloud processing method and system based on three-dimensional variable scale convolution
Patent Information
- Application Number
- CN202511188693.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-25
- Publication Date
- 2026-10-23
AI Technical Summary
[0004]本申请的目的在于克服现有技术中的不足，提供一种基于三维变尺度卷积的激光雷达点云处理方法及系统，以解决相关技术中，固定尺度的三维卷积核难以适配不同大小和形状的目标物体，无法自适应地调整感受野，且计算量较大、计算效率低，难以满足实时性要求较高的应用场景等问题
[0041]本申请实施例可以利用VoxelNet将初始激光雷达点云数据划分至预设尺寸的体素方格中得到激光雷达体素数据，以根据体素数据中目标物体的尺寸动态调节变尺度卷积核的参数，有效提升变尺度目标的特征提取能力；进而，通过修正函数将采样点数量动态调整为奇数，能够避免偶数采样点导致的边缘特征提取不均衡，并以目标体素位置为中心构建变尺度卷积核采样位置矩阵，从而利用矩阵运算确定目标采样点的位置，结合三线性插值算法和仿射变换，得到优化后的三维网格特征，有效提高计算效率，实现对目标姿态变化的鲁棒性显著增强。由此，解决了相关技术中，固定尺度的三维卷积核难以适配不同大小和形状的目标物体，无法自适应地调整感受野，且计算量较大、计算效率低，难以满足实时性要求较高的应用场景的问题。
Smart Images
Figure CN122966975A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of lidar point cloud processing technology, and in particular to a lidar point cloud processing method and system based on three-dimensional variable-scale convolution. Background Technology
[0002] In related technologies, during point cloud processing, the 3D convolution method extracts features from point cloud data in local areas by sliding the convolution kernel in 3D space, thereby obtaining the spatial geometric features of the point cloud. It is widely used in tasks such as 3D target detection, classification and segmentation.
[0003] However, in related technologies, due to the sparsity and irregularity of point cloud data acquired by LiDAR, fixed-scale 3D convolutional kernels are difficult to adapt to target objects of different sizes and shapes. Furthermore, fixed-scale 3D convolution cannot adaptively adjust the receptive field when processing point cloud regions with varying density distributions, making it difficult to extract effective features in sparse point cloud regions, thus affecting the accuracy of subsequent target detection and classification. Simultaneously, the computational cost of 3D convolution in related technologies is high, resulting in low computational efficiency when processing large-scale point cloud data, making it difficult to meet the real-time requirements of application scenarios, thus necessitating improvement. Summary of the Invention
[0004] The purpose of this application is to overcome the shortcomings of the prior art and provide a lidar point cloud processing method and system based on three-dimensional variable-scale convolution. This addresses the problems in related technologies, such as the difficulty of adapting fixed-scale three-dimensional convolution kernels to target objects of different sizes and shapes, the inability to adaptively adjust the receptive field, the large amount of computation, low computational efficiency, and the difficulty in meeting the requirements of application scenarios with high real-time requirements.
[0005] To solve the above-mentioned technical problems, this application adopts the following technical solution:
[0006] In a first aspect, this application provides a method for processing lidar point clouds based on three-dimensional variable-scale convolution, comprising the following steps: acquiring initial lidar point cloud data, and using a VoxelNet voxel network to divide the initial lidar point cloud data into voxel squares of a preset size to obtain lidar voxel data; using a target detection network to extract target objects from the lidar voxel data, and dynamically adjusting the parameters of the variable-scale convolution kernel according to the size of the target object; correcting the number of sampling points of the convolution kernel according to the adjusted parameters of the variable-scale convolution kernel, and constructing a variable-scale convolution kernel sampling position matrix with the target voxel position as the center and the corrected number of sampling points; using matrix operations to determine the position of the target sampling point in the variable-scale convolution kernel sampling position matrix, and using a trilinear interpolation algorithm to obtain the three-dimensional sampling features of the target sampling point; using matrix operations to integrate the three-dimensional sampling features of all sampling points in the variable-scale convolution kernel sampling position matrix to obtain three-dimensional mesh features, and performing an affine transformation on the three-dimensional mesh features to obtain optimized three-dimensional mesh features.
[0007] Optionally, in one embodiment of this application, the expression for the lidar voxel data is:
[0008] ,
[0009] in, Represents the voxelization function. P represents a group of local lidar point clouds in the initial lidar point cloud data. i This represents the i-th local lidar point cloud in the initial lidar point cloud data; This represents the lidar voxel data composed of the j-th local lidar point cloud, where... .
[0010] Optionally, in one embodiment of this application, the step of dynamically adjusting the parameters of the variable-scale convolution kernel according to the size of the target object includes: dynamically generating a region of interest covering the target object, wherein the expression of the region of interest is:
[0011] ,
[0012] in, This represents the k-th region of interest. This represents the function that generates the region of interest. This indicates obtaining the target function. This represents a group composed of lidar voxel data;
[0013] The convolution kernel scale is dynamically adjusted according to the size of the target object to obtain parameters of a variable-scale convolution kernel that matches the target object, in combination with the region of interest. The expression for the parameters of the variable-scale convolution kernel is as follows:
[0014] ,
[0015] in, S represents the variable scaling function; m The parameters representing the variable-scale convolution kernel. , and These represent the height, width, and depth of the variable-scale convolution kernel, respectively.
[0016] Optionally, in one embodiment of this application, the step of correcting the number of sampling points of the convolution kernel according to the adjusted kernel scale parameters includes: initially adjusting the number of sampling points of the convolution kernel using an adjustment function based on the adjusted parameters of the variable-scale convolution kernel, wherein the expression for the initially adjusted number of sampling points is:
[0017] ,
[0018] in, Represents the adjustment function. , , This indicates the number of sampling points in the height, width, and depth of the variable-scale convolution kernel after initial adjustment. This represents the average parameter value of the adjusted variable-scale convolution kernel. , and The average height, average width, and average depth of the adjusted variable-scale convolutional kernel;
[0019] Based on the number of sampling points after the initial adjustment, a correction function is used to obtain a value no greater than [amount missing]. The largest odd number is determined, and this largest odd number is used as the corrected number of sampling points, wherein the expression for the correction function is:
[0020] ,
[0021] in, This represents the correction function. This indicates the corrected number of sampling points. , and This is the number of odd samples obtained after correction in the height, width, and depth of the variable-scale convolution kernel.
[0022] Optionally, in one embodiment of this application, the step of constructing a variable-scale convolution kernel sampling position matrix centered on the target voxel location and in combination with the corrected number of sampling points includes: constructing a target convolution kernel sampling position centered on the target voxel location based on the corrected number of sampling points, and traversing all voxel locations to obtain the convolution kernel sampling matrix, wherein the expression for the target convolution kernel sampling position is:
[0023] ,
[0024] in, Represents the convolution kernel sampling matrix The Middle line, number Column, No. Sampling location of the layer;
[0025] Based on the parameters of the adjusted variable-scale convolution kernel corresponding to the target voxel position, the target convolution kernel scale parameters are constructed, and the convolution kernel scale matrix is obtained by traversing all voxel positions. The expression for the target convolution kernel scale parameters is as follows:
[0026] ,
[0027] in, Represents the convolution kernel scale matrix The Middle line, number Column, No. Layer scale parameters; , and Target voxels The height, width, and depth of the scale-dependent convolution kernel;
[0028] Based on the convolution kernel sampling matrix and the convolution kernel scale matrix, a variable-scale convolution kernel sampling position matrix is obtained, wherein the expression for the variable-scale convolution kernel sampling position matrix is:
[0029] ,
[0030] in, This represents the sampling position matrix of the variable-scale convolution kernel. This indicates an element-wise multiplication operation.
[0031] Optionally, in one embodiment of this application, obtaining the optimized three-dimensional mesh features includes: based on the known feature information of the surrounding sampling points of the target sampling point, performing feature interpolation calculation on the target sampling point using a trilinear interpolation algorithm to obtain the three-dimensional sampling features of the target sampling point, wherein the calculation expression of the three-dimensional sampling features is:
[0032] ,
[0033] in, This represents the three-dimensional sampling feature output at the target sampling point c0. This represents the parameters of the variable-scale convolution kernel at the target sampling point c0. Indicates sampling point The sampled value at that location, Representation matrix Elements in;
[0034] The 3D sampling features of all sampling points are integrated through matrix operations to obtain 3D mesh features. An affine transformation is then performed on the spatial distribution of these 3D mesh features to obtain optimized 3D mesh features. The expression for the affine transformation is as follows:
[0035] ,
[0036] in, This represents the optimized 3D mesh feature. It is a three-dimensional mesh feature. For linear transformation parameters, These are the translation transformation parameters.
[0037] Secondly, this application provides a lidar point cloud processing system based on three-dimensional variable-scale convolution, comprising: a data acquisition module for acquiring initial lidar point cloud data and dividing the initial lidar point cloud data into voxel grids of a preset size using a voxel network (VoxelNet) to obtain lidar voxel data; a convolution kernel variable-scale module for extracting target objects from the lidar voxel data using a target detection network, and dynamically adjusting the parameters of the variable-scale convolution kernel according to the size of the target object; a sampling adjustment module for correcting the number of sampling points of the convolution kernel according to the adjusted parameters of the variable-scale convolution kernel, and constructing a variable-scale convolution kernel sampling position matrix centered on the target voxel position and combining the corrected number of sampling points; a feature processing module for determining the position of the target sampling point in the variable-scale convolution kernel sampling position matrix using matrix operations, and obtaining the three-dimensional sampling features of the target sampling point by combining a trilinear interpolation algorithm; and a feature generation module for integrating the three-dimensional sampling features of all sampling points in the variable-scale convolution kernel sampling position matrix using matrix operations to obtain three-dimensional mesh features, and performing an affine transformation on the three-dimensional mesh features to obtain optimized three-dimensional mesh features.
[0038] Thirdly, this application provides an electronic terminal, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it is used to implement the above-mentioned lidar point cloud processing method based on three-dimensional variable-scale convolution.
[0039] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the above-described lidar point cloud processing method based on three-dimensional variable-scale convolution.
[0040] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, are used to implement the above-described lidar point cloud processing method based on three-dimensional variable-scale convolution.
[0041] This application embodiment utilizes VoxelNet to divide initial LiDAR point cloud data into voxel grids of preset sizes to obtain LiDAR voxel data. The parameters of the variable-scale convolution kernel are dynamically adjusted according to the size of the target object in the voxel data, effectively improving the feature extraction capability for variable-scale targets. Furthermore, by dynamically adjusting the number of sampling points to an odd number through a correction function, the uneven edge feature extraction caused by an even number of sampling points can be avoided. A variable-scale convolution kernel sampling position matrix is constructed with the target voxel position as the center, thereby using matrix operations to determine the position of the target sampling points. Combined with trilinear interpolation and affine transformation, optimized 3D mesh features are obtained, effectively improving computational efficiency and significantly enhancing robustness to target pose changes. Therefore, this solves the problems in related technologies where fixed-scale 3D convolution kernels are difficult to adapt to target objects of different sizes and shapes, cannot adaptively adjust the receptive field, and suffer from high computational load and low efficiency, making them unsuitable for applications with high real-time requirements.
[0042] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0043] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0044] Figure 1 This is a flowchart of a lidar point cloud processing method based on three-dimensional variable-scale convolution provided in an embodiment of this application;
[0045] Figure 2 This is a schematic diagram illustrating the key implementation steps of three-dimensional variable-scale convolution according to an embodiment of this application;
[0046] Figure 3A schematic diagram illustrating the accuracy of a 3D variable-scale convolution on the vehicle category of the KITTI dataset (a public benchmark dataset jointly created by the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago) according to an embodiment of this application.
[0047] Figure 4 This is a schematic diagram illustrating the 3D detection performance of a 3D variable-scale convolution according to an embodiment of this application on the pedestrian category of the KITTI dataset;
[0048] Figure 5 This is a schematic diagram illustrating the 2D detection performance of a 3D variable-scale convolution according to an embodiment of this application on the pedestrian category of the KITTI dataset;
[0049] Figure 6 This is a schematic diagram illustrating the accuracy of three-dimensional variable-scale convolution on the pedestrian category of the KITTI dataset according to one embodiment of this application;
[0050] Figure 7 This is a schematic diagram illustrating the accuracy of three-dimensional variable-scale convolution in the cyclist category of the KITTI dataset according to one embodiment of this application;
[0051] Figure 8 This is a heatmap of the average accuracy of three-dimensional variable-scale convolution across categories in the KITTI dataset according to one embodiment of this application.
[0052] Figure 9 This is a schematic diagram of the structure of a lidar point cloud processing system based on three-dimensional variable-scale convolution provided in an embodiment of this application. Detailed Implementation
[0053] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.
[0054] Example 1:
[0055] Figure 1 This is a flowchart illustrating a lidar point cloud processing method based on three-dimensional variable-scale convolution, as provided in an embodiment of this application.
[0056] like Figure 1 As shown, the lidar point cloud processing method based on three-dimensional variable-scale convolution includes the following steps:
[0057] In step S101, initial lidar point cloud data is acquired, and the initial lidar point cloud data is divided into voxel grids of a preset size using a voxel network, thus obtaining lidar voxel data.
[0058] It is understandable that the initial lidar point cloud data refers to the three-dimensional spatial data set generated after the lidar emits a laser beam and receives the reflected signal. These discrete point cloud data contain key information such as the position and shape of the target object. The preset size can be [0.05, 0.05, 0.05], which means that the size of the voxel in the x, y, and z dimensions is 0.05 units (in meters, according to the conventional units of the KITTI dataset).
[0059] Specifically, this application embodiment can acquire initial lidar point cloud data and its corresponding initial feature information, use a voxel network (VoxelNet) to divide the initial lidar point cloud data into several voxel grids, aggregate point clouds within the same voxel to form a voxel group, and then convert the initial lidar point cloud data into lidar voxel data. For each voxel, initial features such as center point coordinates, number of points, and average reflection intensity are calculated to form a feature vector.
[0060] Let N points in the initial lidar point cloud be represented as follows: The subsequent voxelization process can be represented as:
[0061] ,
[0062] in, Represents the voxelization function. P represents a group of local lidar point clouds in the initial lidar point cloud data. i This represents the i-th local lidar point cloud in the initial lidar point cloud data; This represents the lidar voxel data composed of the j-th local lidar point cloud. Accordingly, Represents lidar voxels Corresponding features.
[0063] In step S102, the target object is extracted from the lidar voxel data using a target detection network, and the parameters of the variable-scale convolution kernel are dynamically adjusted according to the size of the target object.
[0064] Understandably, the parameters of a variable-scale convolution kernel include its height, width, and depth.
[0065] Specifically, embodiments of this application can dynamically generate a region of interest covering the target object based on the target object, and record lidar voxel data. The set of voxels constitutes So for a set of voxels The target in the image can be used to generate several regions of interest that cover the entire target. The expression for the region of interest is:
[0066] ,
[0067] in, This represents the k-th region of interest. This represents the function that generates the region of interest. This indicates obtaining the target function. This represents a group composed of lidar voxel data;
[0068] The convolution kernel scale is dynamically adjusted based on the size of the target object to obtain parameters of a variable-scale convolution kernel that matches the target object, combined with the region of interest. The expression for the parameters of the variable-scale convolution kernel is as follows:
[0069] ,
[0070] in, S represents the variable scaling function; m The parameters representing the variable-scale convolution kernel. , and These represent the height, width, and depth of the variable-scale convolution kernel, respectively.
[0071] This application's embodiments can dynamically adjust the convolutional kernel size according to the size and shape of the target object: for small target objects, the convolutional kernel size is automatically reduced to improve the ability to capture local details. For example, for small targets such as pedestrians and bicycles in LiDAR point clouds, edge contours and structural features can be accurately extracted, avoiding detail blurring caused by fixed large-scale convolutional kernels. When processing large targets such as vehicles and buildings or sparse areas of point clouds, the convolutional kernel size is increased to expand the receptive field, ensuring complete extraction of the overall target structure and solving the problem of missing features of large targets due to insufficient receptive field caused by fixed-scale convolution. In addition, by dynamically generating a RoI (Region of Interest) covering the entire target, multi-scale feature extraction of different parts of the same target can be achieved. For example, for vehicle targets, small-scale convolutional kernels can be used to extract details such as headlights and wheel hubs, while large-scale convolutional kernels can be used to extract the overall contour of the vehicle body. The robustness of the fused features to changes in target pose is significantly enhanced.
[0072] In step S103, the number of sampling points of the convolution kernel is corrected according to the adjusted parameters of the variable-scale convolution kernel, and a variable-scale convolution kernel sampling position matrix is constructed with the target voxel position as the center and the corrected number of sampling points.
[0073] It is understandable that the target voxel position refers to the position of a voxel in the lidar voxel space, which can be denoted as... .
[0074] Specifically, in this embodiment, after determining the convolution kernel size, the input data can be sampled based on the new size parameters. Since the target scale in LiDAR voxel data often varies, it is necessary to obtain the average height, average width, and average depth of the convolution kernel after the scale change, denoted as... , and The average value obtained is , and Then, the number of sampling points is adjusted using an adjustment function:
[0075] ,
[0076] in, Represents the adjustment function. , , This indicates the number of sampling points in the height, width, and depth of the variable-scale convolution kernel after initial adjustment. This represents the average parameter value of the adjusted variable-scale convolution kernel. , and The average height, average width, and average depth of the adjusted variable-scale convolutional kernel;
[0077] When processing LiDAR voxel data, the parity of the number of sampling points in a 3D variable-scale convolutional kernel significantly impacts feature extraction performance. An initial, adjusted number of sampling points might be even. However, even numbers have limitations in constructing symmetrical convolutional kernel structures, making it difficult to achieve a centrally symmetrical feature extraction pattern and potentially leading to unbalanced edge feature extraction. An odd number of sampling points, on the other hand, allows for better symmetrical sampling around the center point. When processing symmetrical target objects or point cloud structures, it can more accurately capture symmetrical features, avoiding feature shift or loss. Therefore, further adjustments to the initial sampling point number are needed, dynamically adjusting it to a suitable odd number to optimize the convolutional kernel structure and sampling performance.
[0078] Furthermore, regarding the number of sampling points after initial adjustment It can be obtained that is not greater than 1000 using a correction function. The largest odd number:
[0079] ,
[0080] in, This represents the correction function. This indicates the corrected number of sampling points. , and This is the number of odd samples obtained after correction in the height, width, and depth of the variable-scale convolution kernel.
[0081] Therefore, the number of samples in height, width, and depth of the corrected variable-scale convolution kernel are respectively... , and At this point, the total number of sampling points is Let the position of a voxel in the voxel space of the lidar be denoted as . , then The sampling locations of the variable-scale convolution kernel constructed around the center are:
[0082] ,
[0083] in, Represents the variable-scale convolution kernel sampling matrix The Middle line, number Column, No. Sampling location of the layer.
[0084] Furthermore, record The height, width, and depth of the scale-dependent convolution kernel are respectively , and So, what is the scale matrix of the variable-scale convolution kernel? for:
[0085] ,
[0086] in, Represents the convolution kernel scale matrix The Middle line, number Column, No. Layer scale parameters.
[0087] Furthermore, the scale parameter of the variable-scale convolution kernel at all sampling locations satisfies:
[0088] ,
[0089] in, This represents the sampling position matrix of the variable-scale convolution kernel, i.e., the sampling positions of the variable-scale convolution kernel that can be dynamically scaled. This indicates an element-wise multiplication operation.
[0090] This application employs a symmetrical design with an odd number of sampling points. A correction function dynamically adjusts the number of sampling points to an odd number, ensuring that the convolutional kernel samples symmetrically around the center point. This avoids the uneven edge feature extraction problem caused by an even number of sampling points (such as sampling points on one side shifting from the center, causing feature offset). For targets with symmetrical structures (such as vehicles and pedestrians), an odd number of sampling points can be evenly distributed around the center point, accurately capturing symmetrical features. For example, when processing wheel point clouds, an odd number of sampling points can symmetrically cover the circumference, avoiding circular contour distortion caused by sampling offset, and improving feature extraction quality by optimizing the sampling mechanism.
[0091] In step S104, matrix operations are used to determine the position of the target sampling point in the variable-scale convolution kernel sampling position matrix, so as to obtain the three-dimensional sampling features of the target sampling point by combining the trilinear interpolation algorithm.
[0092] In step S105, matrix operations are used to integrate the three-dimensional sampling features of all sampling points in the variable-scale convolution kernel sampling position matrix to obtain three-dimensional mesh features. Then, the three-dimensional mesh features are subjected to affine transformation to obtain optimized three-dimensional mesh features.
[0093] It is understandable that a target sampling point refers to a specific sampling point within the target voxel.
[0094] In actual implementation, the sampling points in this embodiment are discretely distributed in three-dimensional space and may have irregular intervals, or the sampling point positions after scale adjustment may not completely match the original regular grid, which may cause the loss of details or misalignment when directly acquiring features. Trilinear interpolation is used to perform feature interpolation calculation on discrete three-dimensional sampling points. Based on the feature values of the surrounding known points, the continuous features of the target sampling position can be accurately fitted, ensuring that even if the sampling points change dynamically during the scale convolution process, the spatial features of the LiDAR can be extracted stably and smoothly.
[0095] Optionally, in one embodiment of this application, obtaining the optimized three-dimensional mesh features includes: based on the known feature information of the surrounding sampling points of the target sampling point, performing feature interpolation calculation on the target sampling point using a trilinear interpolation algorithm to obtain the three-dimensional sampling features of the target sampling point, wherein the calculation expression of the three-dimensional sampling features is:
[0096] ,
[0097] in, This represents the three-dimensional sampling feature output at the target sampling point c0. This represents the parameters of the variable-scale convolution kernel at the target sampling point c0. Indicates sampling point The sampled value at that location, Representation matrix The elements in.
[0098] The calculated position After performing variable-scale convolution at a given location, repeat the above operation to calculate variable-scale convolutions at all locations, thereby forming a new 3D mesh. The overall feature corresponding to the new 3D mesh is denoted as .
[0099] To better adapt the initially obtained 3D sampling features to the complex and irregular spatial distribution of LiDAR point clouds and eliminate feature bias caused by sparse and uneven distribution of point clouds, affine transformation is performed on the initially obtained 3D sampling features to flexibly adjust the spatial morphology of the features. This allows the features extracted by variable-scale convolution to better fit the real target structure and improve the robustness of subsequent 3D target detection and classification for targets of different poses and sizes.
[0100] Optionally, in one embodiment of this application, obtaining the optimized 3D mesh features includes: integrating the 3D sampling features of all sampling points through matrix operations to obtain 3D mesh features, and performing an affine transformation on the spatial distribution of the 3D mesh features to obtain the optimized 3D mesh features, wherein the expression for the affine transformation is:
[0101] ,
[0102] in, This represents the optimized 3D mesh feature. It is a three-dimensional mesh feature. For linear transformation parameters, These are the translation transformation parameters.
[0103] This application embodiment can efficiently determine the location of sampling points using matrix operations, and simultaneously generate high-quality 3D sampling features based on the feature information of surrounding points using a trilinear interpolation algorithm. Furthermore, affine transformation processing is performed on the output 3D sampling features. By introducing an affine transformation matrix to perform affine transformation operations on the features, the features generated by variable-scale convolution better adapt to the complex spatial structure of point cloud scenes, enhancing the spatial expressiveness and robustness of the features, and further improving the accuracy of point cloud feature extraction.
[0104] This application's embodiments coordinate the scaling of the convolutional kernel with the sampling points, ensuring a one-to-one correspondence between the kernel scale matrix and the sampling positions. For example, in areas of varying point cloud density (such as the boundary between the target edge and the background), the kernel scale is automatically reduced while the sampling point density is increased, ensuring that features in sparse areas are not lost and features in dense areas are not redundant. Because the feature processing module combines trilinear interpolation and affine transformation, trilinear interpolation performs continuous feature fitting on discrete sampling points, resolving the feature fragmentation problem caused by point cloud sparsity. For example, in point cloud hole regions, features with smooth transitions are generated through interpolation of surrounding point features, improving semantic segmentation accuracy. Affine transformation adapts features to the complex spatial structure of the point cloud scene. For example, for tilted vehicle targets, affine transformation can rotate features to a standard coordinate system, reducing the impact of pose changes on detection.
[0105] Furthermore, the embodiments of this application integrate and output the optimized 3D features to provide high-quality feature data for subsequent 3D target detection, classification and other tasks, ensuring the integrity and effectiveness of the entire lidar point cloud processing flow.
[0106] Combination Figures 3 to 8 This demonstrates the performance of this application in various target detection categories, including, for example... Figure 3 As shown, the radar chart comparing the accuracy of 3D variable-scale convolution in the car category of the KITTI dataset demonstrates the performance of various evaluation metrics across different training epochs (epochs 51 to 60). The metrics cover Bbox AP (BoundingBox-Average Precision), BEV AP (Bird's Eye View-Average Precision), 3D AP (Three Dimensions-Average Precision), and AOS AP (Average Orientation Similarity-Average Precision), with each metric categorized into three difficulty levels: Easy, Moderate, and Hard, comprehensively measuring the model's detection performance. Different colored lines represent the training results from epochs 51 to 60, allowing for a visual observation of the trends in accuracy across different metrics as training epochs progress, aiding in the assessment of model convergence and performance fluctuations.
[0107] Overall, among the Bbox AP metrics, Easy difficulty shows higher accuracy with fluctuations during training, while Moderate and Hard difficulties remain relatively stable. BEV AP metrics show an upward trend with Easy difficulty as training progresses, while Moderate and Hard difficulties fluctuate. In 3D AP, Easy difficulty shows significant improvement in detection capabilities in the later stages of training, while performance tends to converge with Moderate and Hard difficulties. AOS AP shows outstanding accuracy in Easy difficulty across multiple rounds, while Moderate and Hard difficulties exhibit stable performance.
[0108] like Figure 4 , 5 As shown, the 3D and 2D detection results of 3D variable-scale convolution on the pedestrian category of the KITTI dataset are presented, along with the results of combining... Figure 6 As shown, this is a radar chart of multi-metric accuracy for the pedestrian category on the KITTI dataset, spanning different training epochs (epochs 51 to 60). From a feature perspective, it covers different dimensions of object detection (2D bounding box, bird's-eye view BEV, 3D detection, joint evaluation AOS), with each dimension further subdivided into Easy, Moderate, and Hard difficulties, comprehensively showcasing model performance. In terms of trends, overall, as the training epochs increase from epoch 51 to epoch 60, the accuracy of most metrics generally increases at different difficulty levels. For metrics like Bbox AP and Easy, the lines in later epochs (e.g., epochs 58 to 60) are closer to the outer edge of the radar chart, indicating higher accuracy. This reflects the model training gradually stabilizing, with continuously improving accuracy and multi-dimensional representation capabilities for pedestrian category detection. Even under complex Hard difficulty, performance can be optimized through continuous training, demonstrating the effectiveness of iterative model training.
[0109] like Figure 7 As shown, the radar chart is categorized into four main types: Bbox, BEV, 3D, and AOS. Each category is further subdivided into Easy, Moderate, and Hard difficulty levels with AP (Average Precision) metrics, constructing a multi-dimensional evaluation system. The higher the difficulty, the greater the challenge of the detection task. The line graphs for different rounds (rounds 51 to 60) represent the experimental results for each round. Overall, the AP metrics for simple tasks such as Bbox and Easy are generally higher. As the difficulty increases, the AP value decreases, which is consistent with the pattern of detection tasks: "simple scenarios are easy to be accurate, and complex scenarios are difficult." Comparing the rounds, later rounds (such as rounds 59 and 60) are closer to the edge of the radar chart (i.e., higher AP values) in most metrics, indicating that the algorithm gradually optimizes its multi-dimensional accuracy in cyclist detection with experimental iterations.
[0110] like Figure 8As shown, the heatmap of mAP (mean Average Precision) for 3D variable-scale convolution on various categories of the KITTI dataset is used to visually present the average precision performance under different rounds (left "Rounds" column, rounds 51 to 60) and different categories (bottom, such as Bbox mAP(Car), BEV mAP(Car), etc., covering targets such as cars, pedestrians, and cyclists, and different evaluation dimensions such as Bbox, BEV, 3D, and AOS). The color scale bar on the right, with values ranging from 60.40 to 97.20, shows a change from dark to light, corresponding to different average precision values. The darker the color (e.g., close to 60.40), the lower the precision, and the lighter the color (e.g., close to 97.20), the higher the precision. Through the color distribution, the precision differences between different rounds and categories can be quickly compared, which helps to analyze the performance of 3D variable-scale convolution for various target detection on the KITTI dataset.
[0111] The lidar point cloud processing method based on 3D variable-scale convolution proposed in this application can utilize VoxelNet to divide the initial lidar point cloud data into voxel grids of a preset size to obtain lidar voxel data. The parameters of the variable-scale convolution kernel are dynamically adjusted according to the size of the target object in the voxel data, effectively improving the feature extraction capability of variable-scale targets. Furthermore, by dynamically adjusting the number of sampling points to an odd number through a correction function, the uneven edge feature extraction caused by an even number of sampling points can be avoided. A variable-scale convolution kernel sampling position matrix is constructed with the target voxel position as the center, thereby using matrix operations to determine the position of the target sampling points. Combined with trilinear interpolation and affine transformation, optimized 3D mesh features are obtained, effectively improving computational efficiency and significantly enhancing robustness to target pose changes. This solves the problems in related technologies where fixed-scale 3D convolution kernels are difficult to adapt to target objects of different sizes and shapes, cannot adaptively adjust the receptive field, and have high computational load and low efficiency, making them unsuitable for applications with high real-time requirements.
[0112] Example 2:
[0113] Next, referring to the accompanying drawings, a lidar point cloud processing system based on three-dimensional variable-scale convolution proposed according to an embodiment of this application is described.
[0114] Figure 9 This is a schematic diagram of the structure of a lidar point cloud processing system based on three-dimensional variable-scale convolution, according to an embodiment of this application.
[0115] like Figure 9As shown, the lidar point cloud processing system 10 based on three-dimensional variable-scale convolution includes: a data acquisition module 100, a convolution kernel variable-scale module 200, a sampling adjustment module 300, a feature processing module 400, and a feature generation module 500.
[0116] Specifically, the data acquisition module 100 is used to acquire initial lidar point cloud data and use VoxelNet to divide the initial lidar point cloud data into voxel squares of a preset size to obtain lidar voxel data.
[0117] The variable-scale convolution kernel module 200 is used to extract target objects from lidar voxel data using a target detection network, and to dynamically adjust the parameters of the variable-scale convolution kernel according to the size of the target object.
[0118] The sampling adjustment module 300 is used to correct the number of sampling points of the convolution kernel according to the adjusted parameters of the variable-scale convolution kernel, and construct the sampling position matrix of the variable-scale convolution kernel with the target voxel position as the center and the corrected number of sampling points.
[0119] The feature processing module 400 is used to determine the position of the target sampling point in the variable-scale convolution kernel sampling position matrix using matrix operations, so as to obtain the three-dimensional sampling features of the target sampling point by combining the trilinear interpolation algorithm.
[0120] The feature generation module 500 is used to integrate the three-dimensional sampling features of all sampling points in the sampling position matrix of the variable-scale convolution kernel using matrix operations to obtain three-dimensional mesh features, and then perform affine transformation on the three-dimensional mesh features to obtain optimized three-dimensional mesh features.
[0121] It should be noted that the foregoing explanation of the embodiment of the lidar point cloud processing method based on three-dimensional variable-scale convolution also applies to the lidar point cloud processing system based on three-dimensional variable-scale convolution in this embodiment, and will not be repeated here.
[0122] The lidar point cloud processing system based on 3D variable-scale convolution proposed in this application can utilize VoxelNet to divide the initial lidar point cloud data into voxel grids of preset sizes to obtain lidar voxel data. The parameters of the variable-scale convolution kernel are dynamically adjusted according to the size of the target object in the voxel data, effectively improving the feature extraction capability of variable-scale targets. Furthermore, by dynamically adjusting the number of sampling points to an odd number through a correction function, the uneven edge feature extraction caused by an even number of sampling points can be avoided. A variable-scale convolution kernel sampling position matrix is constructed with the target voxel position as the center, thereby using matrix operations to determine the position of the target sampling points. Combined with trilinear interpolation and affine transformation, optimized 3D mesh features are obtained, effectively improving computational efficiency and significantly enhancing robustness to target pose changes. This solves the problems in related technologies where fixed-scale 3D convolution kernels are difficult to adapt to target objects of different sizes and shapes, cannot adaptively adjust the receptive field, and suffer from high computational load and low efficiency, making them unsuitable for applications with high real-time requirements.
[0123] Example 3:
[0124] This invention also provides an electronic terminal, including a processor and a memory connected to the processor. The memory stores a computer program, and when the computer program is executed by the processor, it performs the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution described in Embodiment 1 above.
[0125] Example 4:
[0126] This embodiment provides a computer-readable storage medium storing a computer program / instruction thereon. When the computer program / instruction is executed by a processor, it implements the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution as described in any of the embodiments.
[0127] Example 5:
[0128] This embodiment provides a computer program product, including a computer program / instruction, characterized in that, when the computer program / instruction is executed by a processor, it implements the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution as described in any one of Embodiment 1.
[0129] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0130] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0133] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A method for processing lidar point clouds based on three-dimensional variable-scale convolution, characterized in that, The method includes: Acquire initial lidar point cloud data, and use VoxelNet to divide the initial lidar point cloud data into voxel squares of a preset size to obtain lidar voxel data. The target object is extracted from the LiDAR voxel data using a target detection network, and the parameters of the variable-scale convolution kernel are dynamically adjusted according to the size of the target object. The number of sampling points of the variable-scale convolution kernel is corrected based on the adjusted parameters of the variable-scale convolution kernel, and a sampling position matrix of the variable-scale convolution kernel is constructed with the target voxel position as the center and the corrected number of sampling points. The position of the target sampling point in the sampling position matrix is determined by matrix operations, and the three-dimensional sampling features of the target sampling point are obtained by combining the trilinear interpolation algorithm. The three-dimensional sampling features of all sampling points in the variable-scale convolution kernel sampling position matrix are integrated by matrix operations to obtain three-dimensional mesh features. The three-dimensional mesh features are then subjected to affine transformation to obtain optimized three-dimensional mesh features.
2. The lidar point cloud processing method based on three-dimensional variable-scale convolution according to claim 1, characterized in that, The expression for the lidar voxel data is: ， in, Represents the voxelization function. P represents a group of local lidar point clouds in the initial lidar point cloud data. i This represents the i-th local lidar point cloud in the initial lidar point cloud data; This represents the lidar voxel data composed of the j-th local lidar point cloud, where... .
3. The lidar point cloud processing method based on three-dimensional variable-scale convolution according to claim 1, characterized in that, The step of dynamically adjusting the parameters of the variable-scale convolution kernel according to the size of the target object includes: A region of interest (ROI) covering the target object is dynamically generated based on the target object, wherein the expression of the ROI is: ， in, This represents the k-th region of interest. This represents the function that generates the region of interest. This indicates obtaining the target function. This represents a group composed of lidar voxel data; The scale of the variable-scale convolution kernel is dynamically adjusted according to the size of the target object to obtain parameters of the variable-scale convolution kernel that match the target object, in combination with the region of interest. The expression for the parameters of the variable-scale convolution kernel is as follows: ， in, S represents the variable scaling function; m The parameters representing the variable-scale convolution kernel. , and These represent the height, width, and depth of the variable-scale convolution kernel, respectively.
4. The lidar point cloud processing method based on three-dimensional variable-scale convolution according to claim 1, characterized in that, The step of adjusting the number of sampling points of the variable-scale convolution kernel based on the adjusted parameters of the variable-scale convolution kernel includes: Based on the adjusted parameters of the variable-scale convolution kernel, the number of sampling points of the variable-scale convolution kernel is initially adjusted using an adjustment function to obtain the initially adjusted number of sampling points. The expression for the initially adjusted number of sampling points is as follows: ， in, Represents the adjustment function. , , This indicates the number of sampling points in the height, width, and depth of the variable-scale convolution kernel after initial adjustment. This represents the average parameter value of the adjusted variable-scale convolution kernel. , and The average height, average width, and average depth of the adjusted variable-scale convolutional kernel; The maximum odd number not greater than the initially adjusted number of sampling points is obtained using a correction function, and the corrected number of sampling points is determined based on the maximum odd number. The expression for the correction function is: ， in, This represents the correction function. This indicates the corrected number of sampling points. , and This is the number of odd samples obtained after correction in the height, width, and depth of the variable-scale convolution kernel.
5. The lidar point cloud processing method based on three-dimensional variable-scale convolution according to claim 1, characterized in that, The step of constructing a sampling position matrix for a variable-scale convolution kernel, centered on the target voxel location and combined with the corrected number of sampling points, includes: Based on the corrected number of sampling points, a target convolution kernel sampling position is constructed centered on the target voxel position, and the convolution kernel sampling matrix is obtained by traversing all voxel positions. The expression for the target convolution kernel sampling position is: ， in, Represents the convolution kernel sampling matrix The Middle line, number Column, No. Sampling location of the layer; Based on the parameters of the adjusted variable-scale convolution kernel corresponding to the target voxel position, the target convolution kernel scale parameters are constructed, and the convolution kernel scale matrix is obtained by traversing all voxel positions. The expression for the target convolution kernel scale parameters is as follows: ， in, Represents the convolution kernel scale matrix The Middle line, number Column, No. Layer scale parameters; , and Target voxels The height, width, and depth of the scale-dependent convolution kernel; Based on the convolution kernel sampling matrix and the convolution kernel scale matrix, a variable-scale convolution kernel sampling position matrix is obtained, wherein the expression for the variable-scale convolution kernel sampling position matrix is: ， in, This represents the sampling position matrix of the variable-scale convolution kernel. This indicates an element-wise multiplication operation.
6. The lidar point cloud processing method based on three-dimensional variable-scale convolution according to claim 1, characterized in that, The optimized 3D mesh features include: Based on the known feature information of the surrounding sampling points of the target sampling point, a trilinear interpolation algorithm is used to perform feature interpolation calculation on the target sampling point to obtain the three-dimensional sampling features of the target sampling point. The calculation expression for the three-dimensional sampling features is as follows: ， in, This represents the three-dimensional sampling feature output at the target sampling point c0. This represents the parameters of the variable-scale convolution kernel at the target sampling point c0. Indicates sampling point The sampled value at that location, Representation matrix Elements in; The 3D sampling features of all sampling points are integrated through matrix operations to obtain 3D mesh features. An affine transformation is then performed on the spatial distribution of these 3D mesh features to obtain optimized 3D mesh features. The expression for the affine transformation is as follows: ， in, This represents the optimized 3D mesh feature. It is a three-dimensional mesh feature. For linear transformation parameters, These are the translation transformation parameters.
7. A lidar point cloud processing system based on three-dimensional variable-scale convolution, characterized in that, The system includes: Data acquisition module: used to acquire initial lidar point cloud data, and use VoxelNet to divide the initial lidar point cloud data into voxel squares of a preset size to obtain lidar voxel data. Variable-scale convolution kernel module: used to extract target objects from the LiDAR voxel data using the target detection network, and dynamically adjust the parameters of the variable-scale convolution kernel according to the size of the target object; Sampling adjustment module: used to correct the number of sampling points of the variable-scale convolution kernel according to the adjusted parameters of the variable-scale convolution kernel, and to construct the sampling position matrix of the variable-scale convolution kernel with the target voxel position as the center and the corrected number of sampling points. Feature processing module: used to determine the position of the target sampling point in the sampling position matrix using matrix operations, so as to obtain the three-dimensional sampling features of the target sampling point by combining the trilinear interpolation algorithm; Feature generation module: Used to integrate the three-dimensional sampling features of all sampling points in the variable-scale convolution kernel sampling position matrix using matrix operations to obtain three-dimensional mesh features, and to perform affine transformation on the three-dimensional mesh features to obtain optimized three-dimensional mesh features.
8. An electronic terminal, characterized in that, The system includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution as described in any one of claims 1 to 6 are performed.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the lidar point cloud processing method based on three-dimensional variable-scale convolution as described in any one of claims 1 to 6.