Method for accelerating voxel volume measurement through multi-sensor fusion

Abstract

The invention discloses a method for accelerating voxel volume measurement through multi-sensor fusion, and relates to the technical field of agricultural information, and the method comprises the following steps: S1, point cloud collection at different time points; s2, carrying out preprocessing and unified spatial reference on the fusion point clouds at different time points, constructing a time period three-dimensional reconstruction model, and obtaining an accurate volume variation through an external calibration method; s3, voxelization processing is carried out on the model, calculation is carried out under the condition that the resolution is reduced, and a coarse voxel volume change initial value of the time period is obtained; and S4, inputting the coarse voxel volume initial value and the multi-sensor feature difference value into the neural network model, and outputting a high-precision volume change result. According to the method for accelerating voxel volume measurement through multi-sensor fusion, the calculation complexity and memory consumption are remarkably reduced, meanwhile, the precision and real-time performance of volume change measurement are improved through multi-sensor fusion and network correction, and the method is suitable for dynamic volume monitoring in agriculture, storage and logistics management.

Description

Technical Field

[0001] This invention relates to the field of agricultural information technology, and in particular to a method for accelerating voxel volume measurement using multi-sensor fusion. Background Technology

[0002] In modern agricultural production, especially in intelligent variable-rate fertilization and precision agricultural management, rapid and accurate measurement of volume changes in storage media such as fertilizers and grains is of great significance. Taking intelligent variable-rate fertilization as an example, the dynamic monitoring of fertilizer volume in fertilizer bins or fertilizer applicator hoppers directly affects fertilizer application control, replenishment scheduling, and operational efficiency. Its measurement accuracy and real-time performance directly impact the uniformity of nutrient distribution in farmland and the level of intelligence in fertilization operations. Existing technologies commonly employ the following methods: One method is the indirect estimation method based on mechanical metering, such as estimating the output volume by the turntable speed and the material feeding area, and then inferring the remaining volume. This method is simple to implement, but it is prone to large errors in cases of uneven material accumulation, frictional slippage, and local blockage, making it difficult to accurately reflect the true volume change.

[0003] Second, there is the 3D measurement method based on sensor point cloud modeling. This method can use one or more LiDARs or depth cameras to collect and store spatial point clouds, and calculate the volume through mesh reconstruction or spatial integration. Single-sensor methods are not robust when there is severe occlusion or incomplete sampling, which can easily lead to measurement errors. Using multiple sensors can improve coverage, alleviate occlusion problems, and improve the stability of results, but at the same time, it brings the disadvantages of increased data volume and excessive computational cost.

[0004] Thirdly, there is the volume calculation method based on voxel partitioning. This method divides the 3D model into uniform voxel units and calculates volume changes by statistically analyzing voxel occupancy. Theoretically, this method can guarantee high accuracy. However, when the voxel size is small, while the calculation accuracy increases, memory usage and computational complexity rise sharply, and processing speed decreases significantly. When the voxel size is large, although the calculation can be accelerated, the accuracy is insufficient, making it difficult to meet the dual requirements of real-time performance and accuracy in dynamic monitoring scenarios.

[0005] Therefore, existing methods generally suffer from insufficient real-time performance, excessive computational complexity, or unstable measurement accuracy in variable fertilization and smart agriculture scenarios. There is an urgent need for a volume change measurement method that can reduce computational resource consumption while maintaining high accuracy and real-time performance, so as to provide technical support for fertilizer application control in smart variable fertilization and agricultural production automation. Summary of the Invention

[0006] The purpose of this invention is to provide a method for accelerating voxel volume measurement through multi-sensor fusion, which solves the problems of high computational complexity, large memory consumption and insufficient real-time performance in existing voxel-based volume measurement technologies. By using a coarse voxel rapid estimation and neural network correction mechanism, it enables rapid and accurate measurement of volume changes in storage spaces such as fertilizer bins and grain bins.

[0007] To achieve the above objectives, the present invention provides a method for multi-sensor fusion accelerated voxel volume measurement, comprising the following steps: S1. Point cloud acquisition at different time points: Point cloud data is acquired using two depth cameras or lidar cameras at different time points, and multi-sensor fusion is achieved at the same time point through calibration parameters. S2. Point cloud preprocessing and data calibration: preprocess the fused point clouds at different time points and unify the spatial reference, construct a three-dimensional reconstruction model for the time period, and obtain the accurate volume change through external calibration methods. S3. Rapid estimation of bold voxel volume: The model is voxelized and the calculation is performed under reduced resolution to obtain the initial value of the change in bold voxel volume during this time period. S4. Multi-sensor fusion network correction: The initial value of the coarse voxel volume and the difference between the features of the multiple sensors are input into the pre-trained neural network model to output high-precision volume change results.

[0008] Preferably, the calibration in S1 includes internal parameter calibration, external parameter calibration, and time synchronization.

[0009] Preferably, S2 includes the following steps: S201. Preprocessing within time points: Denoise, register and reconstruct triangular meshes for the fused point clouds at each time point to obtain the three-dimensional models at each time point; S202, Time Period Construction and Calibration: Unify the three-dimensional models of the two different times to the same spatial reference, construct the point cloud / model data of the time period, and obtain the accurate volume change of the time period through external actual measurement methods, while extracting the difference of multi-sensor change features.

[0010] Preferably, the preprocessing in S201 includes statistical outlier filtering, voxel downsampling, normal estimation, and mask segmentation, and then performs global initial matching and local fine matching in sequence; the 3D reconstruction preferably uses Poisson reconstruction, with Alpha-Shape or Ball-Pivoting as alternatives, and is combined with hole filling and smoothing.

[0011] Preferably, the external calibration in S202 is obtained through weighing conversion, volume scale, liquid level or water injection / discharge method; the multi-sensor characteristic difference includes coverage / occlusion rate, point density and normal variance, reflection intensity and distance distribution, sensor attitude jitter and environmental measurement.

[0012] Preferably, the voxelization process in S3 is performed by setting a downsampling factor. Coarse the voxel mesh by reducing the voxel edge length from Replace with Within the same spatial range, the number of voxels and computational complexity decrease approximately proportionally to the fine voxel scheme, based on surface area, column-by-column integration, or voxel filling methods. Under the condition of reduced resolution, the initial value of the coarse voxel volume change for that time period can be calculated quickly.

[0013] Preferably, S4 includes a training phase and an application phase. In the training phase, the precise volume change obtained from external calibration is used as the supervision label. The network is a lightweight multilayer perceptron or a convolutional network. The loss function adopts Huber / L2 and is combined with regularization terms and physical prior constraints.

[0014] Preferably, in the S4 application stage, after normalization processing, forward inference is performed to obtain high-precision volume change results, and the output uncertainty is used for quality control or online calibration.

[0015] Therefore, this invention employs the aforementioned multi-sensor fusion method for accelerating voxel volume measurement, by setting a downsampling factor. For coarse voxel estimation, the number of voxels and computational complexity increase with... The voxel count decreases by square or cubic order, effectively alleviating the resource consumption problem of high-precision voxel methods. By using multi-sensor feature difference and neural network correction, the error in coarse voxel estimation is compensated, achieving a balance between low complexity and high accuracy. Through the collaborative design of coarse voxel acceleration, multi-sensor fusion, and learning-based correction, in smart agriculture and variable fertilization scenarios, it can significantly reduce the computational complexity of voxels while maintaining high accuracy and real-time performance in dynamic volume change measurement.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for accelerating voxel volume measurement using multi-sensor fusion according to the present invention; Figure 2 This is a point cloud diagram of the time point / time period of the present invention, (a) is Merged point cloud, (b) is Merged point clouds, (c) is , Overlay point clouds; Figure 3 This is a comparison diagram of coarse / fine voxels in this invention. (a) shows the volume change of the voxels. (b) represents the volume change of a volume element. (c) represents the volume change of the volume element. (d) represents the volume change of the volume element. . Detailed Implementation

[0018] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0019] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] Example Please see Figures 1-3 This invention provides a method for accelerating voxel volume measurement using multi-sensor fusion, comprising the following steps: S1. Point cloud acquisition at different time points: Point cloud data of the target area are acquired using two depth cameras or LiDAR cameras at different time points, and the multi-sensor point clouds at the same time point are unified into the same coordinate system based on calibration parameters to obtain the fused point cloud at that time point.

[0021] Calibration includes internal and external parameter calibration as well as time synchronization; external parameter calibration can use a common calibration board or robust registration based on scene characteristics, and time synchronization can use hardware triggering or high-precision timestamps; IMU / odometer can be used to suppress jitter errors.

[0022] S2. Point cloud preprocessing and data calibration: Preprocess the fused point clouds at different time points to obtain the 3D reconstruction models at each time point; unify the 3D reconstruction models at different times to the same spatial reference to construct point cloud data for a time period, and calibrate the precise volume change of the time period through external actual measurement methods, while extracting the difference of multi-sensor change features.

[0023] S201. Preprocessing within each time point: Denoise, register, and reconstruct triangular meshes for the fused point clouds at each time point to obtain the 3D model at each time point.

[0024] S202, Time Period Construction and Calibration: Unify the three-dimensional models of the two different times to the same spatial reference, construct the point cloud / model data of the time period, and obtain the accurate volume change of the time period through external actual measurement methods, while extracting the difference of multi-sensor change features.

[0025] Preprocessing includes statistical outlier filtering, voxel downsampling, normal estimation, and mask segmentation, followed by global initial matching (e.g., feature + RANSAC) and local fine matching (e.g., point-to-surface ICP / color ICP). Voxel downsampling is used for noise reduction, and mask segmentation is used to remove non-target areas such as warehouse walls. 3D reconstruction preferably uses Poisson reconstruction, with Alpha-Shape or Ball-Pivoting as alternatives, combined with hole filling and smoothing. External calibration can be obtained through weighing conversion, volume scale, liquid level, or water injection / displacement methods. Multi-sensor feature differences can include coverage / occlusion rate, point density and normal variance, reflection intensity and distance distribution, sensor attitude jitter, and environmental measurements.

[0026] S3. Rapid estimation of coarse voxel volume: The triangular mesh model is voxelized and calculated under reduced resolution to obtain the initial value of the coarse voxel volume change during this time period.

[0027] Set the downsampling factor The voxel side length is changed from Replace with Within the same spatial range, the number of voxels and computational complexity are approximately equal compared to the fine voxel scheme. (Based on column-by-column integration / surface voxelization) or (Based on voxel filling) The ratio decreases, thereby quickly obtaining the initial value of the volume change of coarse voxels; coarse voxel calculation can be achieved by using the occupancy difference / column interval counting or TSDF difference method.

[0028] S4. Multi-sensor fusion network correction: The initial value of the coarse voxel volume and the difference between the multi-sensor change features are used as input. During the training phase, the precise volume change is used as the supervision label to train the neural network model. During the application phase, the trained model is used to output the high-precision volume change.

[0029] During the training phase, the precise volume change obtained from external calibration is used as the supervision label. The network can be a lightweight multilayer perceptron or a convolutional network. The loss function can be Huber / L2 with regularization terms and physical prior constraints. During the application phase, the input is normalized and then forward inference is performed to obtain high-precision volume change results. If necessary, the uncertainty can be output for quality control or online calibration.

[0030] The volume calculation result is the volume difference between two time points, and the absolute volume value at any time point can be output optionally.

[0031] The "time point" mentioned in this invention is denoted as (This can also be extended to multiple time points), a fused point cloud is obtained by acquiring and fusing data from multiple sensors at the same point in time; "time period" refers to... The world coordinate system is denoted as... The coordinate systems of each sensor are denoted as .

[0032] A method for accelerating voxel volume measurement using multi-sensor fusion includes the following steps: S1. Point cloud data collection at different time points.

[0033] At different time points Point clouds of the target area are acquired using two depth cameras or LiDAR sensors. Through intrinsic / extrinsic parameter calibration and time synchronization, the point clouds from multiple sensors at the same time point are transformed using extrinsic parameters. Unified to The data is then merged to obtain a fused point cloud at each time point. The following methods were employed: Intrinsic parameter calibration: calibration board; Extrinsic parameter calibration: calibration board; Time synchronization: hardware triggering or high-precision timestamp alignment; IMU / odometer for anti-shake.

[0034] S2, Point Cloud Preprocessing and Data Calibration, the object is... .

[0035] S201, Preprocessing within a given time point.

[0036] right Statistical outlier filtering and voxel downsampling are performed separately (only for noise reduction and does not affect the subsequent coarse voxel step). Normal estimation and mask segmentation (removing non-medium regions such as bin walls) are performed, followed by two-level registration: global initial registration: coarse pose is obtained based on features such as FPFH / SIFT + RANSAC; local fine registration: point-to-surface ICP / color ICP converges to the millimeter / centimeter level; subsequently, triangular mesh reconstruction is used to obtain the watertight model. (Poisson reconstruction is preferred; Alpha-Shape and Ball-Pivoting are alternatives, with hole filling / smoothing performed).

[0037] S202, Time Period Construction and Calibration.

[0038] Will Unify to the same spatial reference, Based on the CAD data of the warehouse, data for a specific time period is generated. Please see Figure 2 The precise volume change over a given period can be obtained through external measured weighing conversion, volume scale, liquid level / meter markings, or water injection / drainage methods. Simultaneously, multi-sensor variation feature difference vectors are extracted from the data at two time points. It may include, but is not limited to: sampling coverage and occlusion: visibility, overlap rate, and view distribution histogram; quality indicators: point density variation, normal variance, roughness, and residual histogram; sensors and environment: reflection intensity / confidence statistics, distance distribution, IMU jitter, temperature and humidity, flow sensor, and speed sensor.

[0039] S3, rapid estimation of bold element volume.

[0040] Will Voxelization to a uniform coarse resolution raster voxel side length To obtain the set of occupied voxels , Or corresponding TSDF field Please see Figure 3 .

[0041] Calculate initial values ​​of volume change at coarse resolution The two preferred implementations are as follows: Option A: Occupation difference / column interval counting (general 3D).

[0042] Surface voxelization: Based on BVH / uniform mesh, the intersection of triangular patches and voxels is calculated, and only near-surface voxels (sparse / hash voxels) are labeled.

[0043] Column-by-column scan: for each The intersection points between the surface and the data at two different time points are counted and sorted. The thickness of the interlayer is obtained by pairing odd and even points. .

[0044] Volume accumulation plus: .

[0045] Option B: TSDF Differential (Robust Parallel).

[0046] TSDF fusion: merge into Cut-off distance .

[0047] Zero intersection point extraction: Find the zero intersection point along a fixed column (gravity direction) or a column of pseudo-wall normals. zero intersection .

[0048] Volume accumulation plus: .

[0049] Explanation, complexity, and resource cost derivation.

[0050] Let the fine voxel resolution be The side length of the voxel is , surface area is The number of triangular pieces is .

[0051] downsampling factor At that time, the side length of the bold element The grid size is approximately ( ).

[0052] Based on voxel filling (traversing a 3D grid): The time / space principal terms of the fine voxels are .

[0053] The roughened principal term is approximately: ; Therefore, time and storage are approximately based on reduce.

[0054] Based on column-wise integration / surface voxelization (two-dimensional principal term); The cost of surface voxelization and intersection is approximately proportional to the number of voxels covering the surface area: ; The main items are counted column by column. coarsened to ; If the surface intersection index uses a BVH / uniform grid, the construct terms are approximately... It can be reused in adjacent batches and regarded as a one-time or secondary primary item.

[0055] Therefore, the overall geometric principal term time and storage overhead of column-by-column / surface routes vary with... reduce.

[0056] When the voxel side length is Expand to This invention achieves approximately [a certain result] in the voxel filling route. Level-based power consumption reduction, achieving approximately [amount missing] on per-column / surface routes. Level-based power consumption reduction; subsequently, learning-based error compensation is performed on the initial value of the volume change of coarse voxels through multi-sensor features and neural networks, achieving a balance between efficiency and accuracy.

[0057] S4, Multi-sensor fusion network correction.

[0058] by As input, output high-precision volume change. The training and reasoning are as follows: S401, Training Phase.

[0059] Construct normalized sample pairs A lightweight neural network is trained using regression loss. :

[0060] ; The preferred network is a multilayer perceptron (input dimension). ,depth Layer, width It can be either a one-dimensional lightweight convolution or a physical prior constraint (non-negativity, upper and lower bounds, monotonicity) as a soft constraint term.

[0061] S402, Application Phase.

[0062] Real-time computing and Data normalized and trained Output As a high-precision measure of volume change. Optionally, the uncertainty (MCdropout / deep integration) can be output as a confidence interval for online quality control.

[0063] Preferred and Deformation Scheme: Hierarchical / Adaptive Voxels: Enable smaller voxels near the interface or in high curvature / high uncertainty regions, while keeping coarse voxels in other regions to further reduce computational cost; Container prior: Based on the hull CAD / wall mesh, establish a "quasi-2.5D" column in the wall normal direction to reduce the effect of overhang; Online calibration: periodic fine-tuning with a small number of external measurement samples. (Low learning rate, freeze the previous layer) to suppress environment / material drift; Boundary handling: Integrate only in the region that is visible at both time points; apply morphological closing operation or TSDF regularization to thin layers / holes.

[0064] Parameter recommendations: In grain silo scenarios, the inner diameter of the silo... Surface undulations At that time, the reconstructed grid resolution can be chosen Bold element coefficient TSDF truncation Gravity is taken in each column direction. Network input includes Column coverage, boundary voxel ratio, normal variance, intensity / distance histogram, etc., are used as training loss parameters. Regular expressions.

[0065] Therefore, the present invention employs the above-mentioned method for accelerating voxel volume measurement through multi-sensor fusion, which has the following advantages: significantly improved computational efficiency by setting a downsampling factor. For coarse voxel estimation, the number of voxels and computational complexity increase with... The voxelization decreases by square or cubic order, effectively alleviating the resource consumption problem of high-precision voxel methods. Measurement accuracy is guaranteed; through multi-sensor feature difference and neural network correction, errors in coarse voxel estimation are compensated, achieving a balance between low complexity and high accuracy. It has strong applicability, and the method can be extended to different storage space scenarios, such as fertilizer warehouses and grain warehouses. It possesses real-time performance and robustness, enabling dynamic volume change monitoring in agriculture, warehousing, and logistics management. Implementation is simple and scalable; the method is compatible with multiple types of sensors and different voxelization implementation paths (fill / column-by-column / TSDF), possessing strong engineering feasibility and scalability. By combining multi-sensor fusion, coarse voxel acceleration, and network correction, it overcomes the bottleneck of traditional voxel computation efficiency and accuracy, significantly reducing computational complexity and memory overhead while maintaining high accuracy and real-time performance in volume change measurement, demonstrating significant application value.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method of multi-sensor fusion accelerated voxel volume measurement, characterized by, Comprising the following steps: S1, point cloud collection at different time points, point cloud data is collected at different time points by using two depth cameras or laser radar cameras, and multi-sensor fusion at the same time point is realized through calibration parameters; S2, point cloud preprocessing and data calibration, preprocessing and unified spatial reference of the fusion point cloud at different time points, construction of a three-dimensional reconstruction model in a time period, and obtaining accurate volume change through an external calibration method; S3, rough voxel volume rapid estimation, voxel processing of the model, and calculation under the condition of reducing resolution to obtain the rough voxel volume change initial value in the time period; S4, multi-sensor fusion network correction, inputting the rough voxel volume initial value and multi-sensor feature difference into a pre-trained neural network model to output a high-precision volume change result.

2. The method of claim 1, wherein: The calibration in S1 includes internal parameter calibration, external parameter calibration and time synchronization.

3. The method of claim 2, wherein, S2 includes the following steps: S201, preprocessing within the time point: denoising, registration and triangular mesh reconstruction of the fusion point cloud at each time point to obtain a three-dimensional model at each time point; S202, time period construction and calibration: unify the three-dimensional models at two different time points to the same spatial reference, construct the point cloud / model data in the time period, and obtain the accurate volume change in the time period through an external actual measurement method, while extracting the multi-sensor change feature difference.

4. The method of claim 3, wherein: The preprocessing in S201 includes statistical outlier filtering, voxel downsampling, normal estimation, mask segmentation, and sequentially performing global initial registration and local fine registration; three-dimensional reconstruction optimizes Poisson reconstruction, and Alpha-Shape or Ball-Pivoting is selected as an alternative and cooperates with hole filling and smoothing.

5. The method of claim 4, wherein: The external calibration in S202 is obtained by weighing conversion, volume scale, liquid level or drainage method; The multi-sensor feature difference includes coverage / occlusion rate, point density and normal variance, reflection intensity and distance distribution, sensor attitude jitter and environmental measurement.

6. The method of claim 5, wherein: The voxelization process in S3 sets a down-sampling factor The voxel grid is coarsened, and the voxel edge length is replaced by In the same spatial range, the number of voxels and the computational complexity are approximately reduced by the surface area-based or column-by-column integration method or voxel filling method. The coarse voxel volume change initial value of the time period is quickly calculated under the condition of reducing the resolution.​ 7. The method of claim 6, wherein: S4 includes a training stage and an application stage, the accurate volume change obtained by external calibration is used as a supervision label in the training stage, the network is a lightweight multilayer perceptron or convolutional network, and the loss function adopts Huber / L2 and cooperates with regular terms and physical prior constraints.

8. The method of claim 7, wherein: In the application stage of S4, the high-precision volume change result is obtained by forward inference after normalization processing, and the uncertainty is output for quality control or online calibration.

Images