Lightweight sparse voxel trunk real-time detection method for orchard vehicle-mounted LiDAR

Abstract

The application discloses a lightweight sparse voxel trunk real-time detection method for orchard vehicle-mounted LiDAR, and belongs to the technical field of 3D environment perception of intelligent orchard intelligent agricultural machinery, vehicle-mounted LiDAR 3D target detection and deep learning. The method comprises the following steps: obtaining original LiDAR point clouds in an effective detection area through orchard point cloud collection and preprocessing; completing original point cloud regularization processing through non-isometric voxel sparsification, reducing the calculation cost and adapting to the anisotropy characteristics of LiDAR resolution; extracting deep 3D voxel features by using a lightweight voxel coding network, adapting to the characteristics of slender small targets and reducing the model parameter quantity; obtaining 2D voxel features through BEV projection aggregation, further reducing the calculation amount; and finally outputting the trunk detection result through confidence correction multi-task detection. The method can balance the trunk accurate detection in a complex orchard scene, model lightweight compression and low-latency embedded deployment, and adapt to the real-time operation demand in the field.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional environmental perception of intelligent agricultural machinery in smart orchards and deep learning technology for vehicle-mounted LiDAR3D target detection. Specifically, it relates to a lightweight end-to-end tree trunk detection method for orchard scenarios, which is applicable to intelligent agricultural machinery operation scenarios such as autonomous navigation in orchards, obstacle avoidance between rows, fruit tree phenotypic extraction, and environmental perception of harvesting robots. Background Technology

[0002] With the rapid development of unmanned equipment in smart orchards, 3D environmental perception based on vehicle-mounted LiDAR is the core support for agricultural machinery to navigate between rows and avoid collisions with fruit trees. As the core columnar target in the orchard, high-precision real-time detection of the tree trunk is a fundamental prerequisite for field operations. Existing orchard tree trunk detection technologies fall into two categories, both of which have significant shortcomings:

[0003] (1) Knowledge-driven geometric detection schemes: Represented by RANSAC cylinder fitting, DBSCAN density clustering, elevation threshold screening, and multi-level morphological filtering, these schemes rely entirely on manually setting fixed judgment rules such as tree shape, spacing, and height. These methods do not require labeled datasets, but the adaptability of manual thresholds is poor, and they are prone to missed detections in scenarios with differences in tree shape and heavy occlusion of branches and leaves. Furthermore, the multi-step serial iteration of ground segmentation, clustering, and screening results in high computational overhead and high inference latency, making it difficult to meet the real-time operation requirements of embedded devices.

[0004] (2) General data-driven 3D detection networks: Voxel detection algorithms such as PointPillars, SECOND, CenterPoint, and VoxelNeXt are trained on urban high-density LiDAR datasets and optimized for large-scale targets such as vehicles and pedestrians. Tree trunks in orchard scenes are small, thin targets with weak features, and the general network's fine-grained feature extraction capability is insufficient. At the same time, the general network has multiple downsampling layers and redundant feature channels, resulting in a large number of model parameters. The computing power and memory of economical edge hardware for orchard agricultural machinery are limited, leading to problems such as high power consumption, poor high-temperature stability, and lag after deployment, making it impossible to balance detection accuracy and vehicle-mounted real-time performance.

[0005] In summary, existing technologies cannot simultaneously address the triple requirements of accurate tree trunk detection in complex orchard scenarios, lightweight model design, and low-latency embedded deployment. Therefore, this invention proposes a lightweight sparse voxel tree trunk detection method. Summary of the Invention

[0006] To address the shortcomings of existing orchard trunk detection technologies in meeting the three core requirements of accurate trunk detection, lightweight model compression, and low-latency embedded deployment in complex orchard scenarios, this invention provides a lightweight sparse voxel real-time trunk detection method for orchard vehicle-mounted LiDAR. Through sparse voxelization, lightweight voxel encoding, BEV projection aggregation, and multi-task detection, the method improves trunk detection accuracy with extremely low parameter count and is suitable for real-time inference of edge devices for orchard agricultural machinery.

[0007] The technical solution of this invention is:

[0008] This invention provides a lightweight, sparse voxel-based real-time detection method for tree trunks using vehicle-mounted LiDAR in orchards. The method includes the following steps:

[0009] S1. Orchard Point Cloud Acquisition and Preprocessing Steps: Orchard point clouds are acquired using a vehicle-mounted LiDAR. A hemispherical effective detection area is delineated, and points obscured by the vehicle body and distant sparse invalid measurement points are filtered out to obtain the raw point cloud. N represents the number of points, and the 4-dimensional components include 3D spatial coordinates (x, y, z) and reflectivity ρ.

[0010] S2. Sparse Voxelization Step: Perform sparse voxelization normalization on the original point cloud P to obtain the initial 3D voxel features. N0 represents the effective number of 3D voxels;

[0011] S3. Lightweight Voxel Encoding Step: Perform lightweight voxel encoding on the initial 3D voxel features F0 to mine deep semantic features and obtain the final 3D voxel features. N 3d <N represents the number of valid 3D voxels after encoding, and 128 represents the feature dimension after encoding;

[0012] S4, BEV projection aggregation step: For the final 3D voxel features F... 3d BEV projection dimensionality reduction and feature summation aggregation along the height axis yield 2D voxel features. N 2d <N 3d The effective number of 2D voxels;

[0013] S5. Multi-task detection step: Detect 2D voxel features F 2d Perform multi-task detection processing, solve for the tree trunk detection box parameters and confidence level, and output the tree trunk detection results. M represents the number of tree trunks detected, and the 7-dimensional component includes the center position of the tree trunk detection box (c). x c y c z ), scale (l, w, h) and confidence level s.

[0014] Furthermore, S2 specifically refers to:

[0015] S21. Perform non-isometric voxel partitioning on the original point cloud P to adapt to the anisotropic resolution features of LiDAR in the horizontal and vertical directions; set the reference size v for the horizontal XY axis and the vertical Z axis size to 2v; round down the normalized point coordinates to generate the corresponding voxel index.

[0016] S22. Calculate the theoretical number of points that a single voxel can accommodate based on the LiDAR horizontal and vertical angular resolutions, and set the upper limit τ for voxel sampling. cap For dense voxels at close range, only the maximum number of measurement points are retained, while for sparse voxels at far range, all measurement points are retained, thus reducing the redundant computation of point clouds at close range.

[0017] S23. Eliminate empty voxels without measurement points and retain only effective sparse voxels to participate in subsequent network operations, thereby reducing memory usage and overall computational overhead.

[0018] Furthermore, the S3 design abandons the common 3-5 level downsampling design of networks, setting only 2 levels of downsampling to reduce the loss of fine-grained features in the trunk and significantly compress the number of network parameters. The input layer upscales the original voxel features to 16 channels through 3×3×3 sparse convolutions. Two sets of residual blocks are configured before and after each downsampling level. The residual blocks are composed of two layers of 3×3×3 sparse convolutions combined with BN, ReLU, and residual mapping. The two downsampling channels are increased to 32 and 64 respectively. Finally, 1×1×1 sparse convolutions are used to upscale to 128 channels to output the 3D voxel features F. 3d .

[0019] Furthermore, S5 specifically refers to:

[0020] S51: Features of 2D voxels The base confidence score s0, the localization quality score q, and the trunk bounding box parameters (c) are predicted using the classification confidence branch, the localization quality branch, and the bounding box branch, respectively. x c y c z (l, w, h);

[0021] S52: An exponentially weighted fusion method is used to correct the base confidence level, reducing the confidence score of candidate boxes with poor positioning accuracy. The correction formula is as follows:

[0022]

[0023] In the formula, β s The weights are used to balance the weight ratios of classification confidence and localization quality scores.

[0024] S53: Set the confidence threshold τ sLow-confidence candidate boxes are filtered out, and overlapping detection boxes are eliminated using non-maximum suppression. The final output includes 3D tree trunk 3D detection boxes containing 3D center coordinates, length, width, and height scales, and corrected confidence.

[0025] The beneficial effects of this invention are:

[0026] (1) Higher detection accuracy: The proposed method uses non-equiaxed voxel division to adapt to the vertical shape of the tree trunk, and combines confidence weighted correction to suppress candidate boxes with large positioning errors, resulting in a detection accuracy superior to mainstream algorithms.

[0027] (2) Extremely lightweight model: The method of this invention only uses a two-level downsampling encoder, combined with sparse empty voxel removal, with an overall parameter count of only 0.78M, which is far lower than that of mainstream algorithms. It retains the fine-grained features of the trunk with a very small model size, and the hardware adaptation threshold is lower.

[0028] (3) Excellent edge real-time performance: Relying on sparse convolution and near-distance point cloud truncation and redundancy reduction strategy, the single-frame inference latency is 12.1ms, which takes into account both high precision and low latency. It can be directly deployed in orchard agricultural machinery embedded devices to meet the real-time perception needs in the field.

[0029] (4) Strong anti-interference performance of feature extraction: Convolution calculation is only performed on effective voxels to avoid the dilution of foreground features by blank background zero values. The BEV axis summation fully preserves the tree trunk column geometric features, and the extraction effect of weak features and small targets is better.

[0030] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0031] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the invention.

[0032] Figure 1 The flowchart of the real-time tree trunk detection using a vehicle-mounted LiDAR in orchards according to the present invention is shown.

[0033] Figure 2 The vehicle-mounted LiDAR data acquisition platform in an embodiment of the present invention is shown, and the LiDAR coordinate system is marked.

[0034] Figure 3 A schematic diagram of the network structure for lightweight voxel encoding according to the present invention is shown.

[0035] Figure 4 The tree trunk detection results in an embodiment of the present invention are shown, with red boxes representing truth boxes and green boxes representing prediction boxes. Detailed Implementation

[0036] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0037] like Figure 1 As shown, a lightweight, sparse voxel tree trunk real-time detection method for vehicle-mounted LiDAR in orchards includes the following steps:

[0038] S1. Orchard Point Cloud Acquisition and Preprocessing Steps: Orchard point clouds are acquired using a vehicle-mounted LiDAR. A hemispherical effective detection area is delineated, and points obscured by the vehicle body and distant sparse invalid measurement points are filtered out to obtain the raw point cloud. N represents the number of points, and the 4-dimensional components include 3D spatial coordinates (x, y, z) and reflectivity ρ.

[0039] S2. Sparse Voxelization Step: Perform sparse voxelization normalization on the original point cloud P to obtain the initial 3D voxel features. N0 represents the effective number of 3D voxels, specifically:

[0040] S21. Perform non-isometric voxel partitioning on the original point cloud P to adapt to the anisotropic resolution features of LiDAR in the horizontal and vertical directions; set the reference size v for the horizontal XY axis and the vertical Z axis size to 2v; round down the normalized point coordinates to generate the corresponding voxel index.

[0041] S22. Calculate the theoretical number of points that a single voxel can accommodate based on the LiDAR horizontal and vertical angular resolutions, and set the upper limit τ for voxel sampling. cap For dense voxels at close range, only the maximum number of measurement points are retained, while for sparse voxels at far range, all measurement points are retained, thus reducing the redundant computation of point clouds at close range.

[0042] S23. Eliminate empty voxels without measurement points and retain only effective sparse voxels to participate in subsequent network operations, thereby reducing memory usage and overall computational overhead.

[0043] S3. Lightweight Voxel Encoding Step: Perform lightweight voxel encoding on the initial 3D voxel features F0 to mine deep semantic features and obtain the final 3D voxel features. N 3d<N is the number of effective 3D voxels after encoding, and 128 is the feature dimension after encoding; the encoding network abandons the 3 to 5 level downsampling design of general networks, and only sets 2 levels of downsampling to reduce the loss of fine-grained trunk features while greatly compressing the network parameters; the input layer uses a 3×3×3 sparse convolution to raise the dimension of the original voxel features to 16 channels; 2 sets of residual blocks are arranged before and after each level of downsampling, and the residual block consists of two layers of 3×3×3 sparse convolution combined with BN, ReLU and residual mapping, the channels are sequentially increased to 32 and 64 in the two levels of downsampling, and finally a 1×1×1 sparse convolution is used to raise the dimension to 128 channels to output the 3D voxel feature F 3d .

[0044] S4. BEV projection aggregation step: perform BEV projection dimensionality reduction and feature sum aggregation on the final 3D voxel feature F 3d along the height axis to obtain 2D voxel features N 2d <N 3d is the number of effective 2D voxels.

[0045] S5. Multi-task detection step: perform multi-task detection processing on the 2D voxel feature F 2d to solve the parameters and confidence of the trunk detection frame, and output the trunk detection result M is the number of detected trunks, and the 7-dimensional components include the center position (c x , c y , c z ), scale (l, w, h) and confidence s, specifically:

[0046] S51: For the 2D voxel features , predict the basic confidence s0, the positioning quality score q and the trunk bounding box parameter (c x , c y , c z , l, w, h) through the classification confidence branch, the positioning quality branch, and the bounding box branch respectively;

[0047] S52: Use the exponential weighted fusion method to correct the basic confidence, reduce the confidence score of candidate frames with poor positioning accuracy, and the correction formula is:

[0048]

[0049] In the formula, β s ∈(0, 1) is the fusion weight, which is used to balance the weight proportion of the classification confidence and the positioning quality score;

[0050] S53: Set the confidence threshold τ sLow-confidence candidate boxes are filtered out, and overlapping detection boxes are eliminated using non-maximum suppression. The final output includes 3D tree trunk 3D detection boxes containing 3D center coordinates, length, width, and height scales, and corrected confidence.

[0051] In practice:

[0052] Vehicle-mounted LiDAR data acquisition platform such as Figure 2 As shown, a Velodyne VLP-1616-line LiDAR was used, with an effective horizontal field of view of 240°, a motor speed of 600rpm, corresponding to a horizontal angular resolution of 0.2° and a single frame period of 100ms. Model training and inference were conducted on a 64-bit Windows 10 system: the hardware consisted of an Intel Xeon Gold 6330 (2.00GHz) CPU and an NVIDIA RTX 3090 graphics card with 24GB of video memory; the software system used Ubuntu 20.04 operating system, with the training process built on the PyTorch 2.0.0 deep learning framework and CUDA 11.8 acceleration environment, and the model development relied on the OpenPCDetv0.6.0 open-source point cloud detection framework, and sparse voxel convolution operations were implemented using the spconv-cul 132.3.3 sparse convolution library.

[0053] The test site was a standardized densely planted walnut orchard, with a row spacing of 6m and a tree spacing of 4m. The orchard included typical targets such as shrubs, slopes, protective fences, utility poles, power lines, and workers. The robot traversed the test area at a uniform speed of 0.8m / s along the center line of the tree row, and collected a total of 1778 frames of raw LiDAR point cloud data.

[0054] In the preprocessing stage, a hemispherical effective detection region with a radius of 10m was defined. Tree trunks were labeled using SUSTechPOINTS software and randomly split into training, validation, and test sets in a ratio of 7:1.5:1.5. These sets were used for model training, hyperparameter tuning, and performance evaluation, respectively. To improve the robustness of the model, data augmentation was performed on the point cloud of the training set by randomly fusing 1 to 5 non-overlapping tree trunk samples from the tree trunk ground truth database to expand the number of tree trunks.

[0055] The voxel size v is set to 0.05m, and the upper limit of voxel sampling τ is set to... cap Set to 5, fusion weight β s Set to 0.6, confidence threshold τ s Set to 0.25. The lightweight voxel encoding network structure is as follows: Figure 3 As shown. The tree trunk detection results are as follows. Figure 4 As shown, red represents the truth box and green represents the prediction box.

[0056] Four mainstream vehicle-mounted LiDAR 3D detection algorithms were selected for comparative experiments on a test set: PointPillars served as a lightweight baseline for 2D cylinders; SECOND optimized vertical features using sparse convolution and axial convolution, serving as a comparison of sparse convolution schemes; CenterPoint balanced accuracy and speed using a center point frame; and VoxelNeXt achieved height through 5-level downsampling deep convolution, serving as a reference for the upper limit of accuracy. The results of the comparative experiments on the test set are shown in Table 1. The method of this invention achieved a BEVmIoU of 83.5% and a 3D mIoU of 63.9%, comprehensively outperforming the four comparative algorithms. The model parameter size was only 0.78M, far lower than all the comparative algorithms, and the inference latency was 12.1ms, far lower than VoxelNeXt. This method, balancing detection accuracy, model size, and low edge latency requirements, is fully compatible with the real-time operation requirements of embedded hardware for orchard agricultural machinery.

[0057] Table 1 shows the experimental results compared to the test set.

[0058]

[0059] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A lightweight, sparse voxel-based real-time detection method for tree trunks using vehicle-mounted LiDAR in orchards, characterized in that, The method includes the following steps: S1. Orchard Point Cloud Acquisition and Preprocessing Steps: Orchard point clouds are acquired using a vehicle-mounted LiDAR. A hemispherical effective detection area is delineated, and points obscured by the vehicle body and distant sparse invalid measurement points are filtered out to obtain the raw point cloud. N represents the number of points, and the 4-dimensional components include 3D spatial coordinates (x, y, z) and reflectivity ρ. S2. Sparse Voxelization Step: Perform sparse voxelization normalization on the original point cloud P to obtain the initial 3D voxel features. N0 represents the effective number of 3D voxels; S3. Lightweight Voxel Encoding Step: Perform lightweight voxel encoding on the initial 3D voxel features F0 to mine deep semantic features and obtain the final 3D voxel features. N 3d <N represents the number of valid 3D voxels after encoding, and 128 represents the feature dimension after encoding; S4, BEV projection aggregation step: For the final 3D voxel features F... 3d BEV projection dimensionality reduction and feature summation aggregation along the height axis yield 2D voxel features. N 2d <N 3d The effective number of 2D voxels; S5. Multi-task detection step: Detect 2D voxel features F 2d Perform multi-task detection processing, solve for the tree trunk detection box parameters and confidence level, and output the tree trunk detection results. M represents the number of tree trunks detected, and the 7-dimensional component includes the center position of the tree trunk detection box (c). x c y c z ), scale (l, w, h) and confidence level s.

2. The lightweight, sparse voxel-based real-time detection method for tree trunks using vehicle-mounted LiDAR in orchards, as described in claim 1, is characterized in that... Specifically, S2 is: S21. Perform non-isometric voxel partitioning on the original point cloud P to adapt to the anisotropic resolution features of LiDAR in the horizontal and vertical directions; set the reference size v for the horizontal XY axis and the vertical Z axis size to 2v; round down the normalized point coordinates to generate the corresponding voxel index. S22. Calculate the theoretical number of points that a single voxel can accommodate based on the LiDAR horizontal and vertical angular resolutions, and set the upper limit τ for voxel sampling. cap For dense voxels at close range, only the maximum number of measurement points are retained, while for sparse voxels at far range, all measurement points are retained, thus reducing the redundant computation of point clouds at close range. S23. Remove empty voxels without measurement points and retain only effective sparse voxels to participate in subsequent network operations, thereby reducing memory usage and overall computational overhead.

3. The lightweight, sparse voxel-based real-time detection method for tree trunks using vehicle-mounted LiDAR in orchards, as described in claim 1, is characterized in that... The S3 described above abandons the common 3-5 level downsampling design of networks, setting only 2 levels of downsampling to reduce the loss of fine-grained features in the trunk and significantly compress the number of network parameters. The input layer upscales the original voxel features to 16 channels through 3×3×3 sparse convolutions. Two sets of residual blocks are configured before and after each downsampling level. The residual blocks are composed of two layers of 3×3×3 sparse convolutions combined with BN, ReLU and residual mapping. The two downsampling levels increase the number of channels to 32 and 64 respectively. Finally, a 1×1×1 sparse convolution is used to upscale to 128 channels to output the 3D voxel features F. 3d .

4. A lightweight, sparse voxel-based real-time detection method for tree trunks using vehicle-mounted LiDAR in orchards, as described in claim 1, is characterized in that... Specifically, S5 is: S51: Features of 2D voxels The base confidence score s0, the localization quality score q, and the trunk bounding box parameters (c) are predicted using the classification confidence branch, the localization quality branch, and the bounding box branch, respectively. x c y c z (l, w, h); S52: An exponentially weighted fusion method is used to correct the base confidence level, reducing the confidence scores of candidate boxes with poor positioning accuracy. The correction formula is as follows: In the formula, β s The weights are used to balance the weight ratios of classification confidence and localization quality scores. S53: Set the confidence threshold τ s Low-confidence candidate boxes are filtered out, and overlapping detection boxes are eliminated using non-maximum suppression. The final output includes 3D tree trunk 3D detection boxes containing 3D center coordinates, length, width, and height scales, and corrected confidence.

Images