A method for multi-sensor fusion accelerated voxel volume measurement

Abstract

The application discloses a kind of multi-sensor fusion accelerated voxel volume measurement method, it is related to agricultural information technology field, comprising the following steps: S1, different time point point cloud acquisition;S2, the fusion point cloud of different time points is preprocessed and unified spatial reference, constructs time period three-dimensional reconstruction model, and accurate volume change amount is obtained by external calibration method;S3, voxelization processing is carried out to model, and under the condition of reducing resolution, calculation is carried out, and the coarse voxel volume change initial value of this time period is obtained;S4, coarse voxel volume initial value and multi-sensor feature difference are input into neural network model, and high-precision volume change result is output.The application uses the above-mentioned multi-sensor fusion accelerated voxel volume measurement method, significantly reduces the complexity of calculation and memory consumption, and improves the accuracy and real-time performance of volume change measurement through multi-sensor fusion and network correction, suitable for dynamic volume monitoring in agriculture, warehousing and logistics management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural information technology, and in particular to a multi-sensor fusion accelerated voxel volume measurement method. BACKGROUND

[0002] In modern agricultural production, especially in intelligent variable rate fertilization and precision agricultural management, it is of great significance to quickly and accurately measure the volume changes of fertilizer, grain and other storage media. Taking intelligent variable rate fertilization as an example, the dynamic monitoring of the volume of fertilizer in the fertilizer warehouse or fertilizer tank is directly related to the control of fertilization amount, supply scheduling and operation efficiency. The measurement accuracy and real-time performance will directly affect the uniformity of nutrient distribution in farmland and the intelligent level of fertilization operation. In the prior art, the common methods mainly include the following categories:

[0003] First, the indirect estimation method based on mechanical measurement, for example, the amount of material is calculated by the rotation speed of the rotating disc and the discharge area, and then the remaining volume is inversely calculated. This method is simple to implement, but it is prone to large errors in the case of uneven material accumulation, friction sliding and local blockage, and it is difficult to accurately reflect the true volume change.

[0004] Second, the three-dimensional measurement method based on sensor point cloud modeling, which can use a single or multiple laser radars, depth cameras to collect storage space point clouds, and calculate the volume through grid reconstruction or spatial integration. The single sensor method has poor robustness when the coverage is severely blocked or the sampling is incomplete, which can easily lead to measurement deviation; the use of multiple sensors can improve the coverage range, alleviate the blocking problem and improve the result stability, but it also brings the defects of increased data volume and high computational overhead.

[0005] Third, the volume calculation method based on voxel division, which divides the three-dimensional model into uniform voxel units, and calculates the volume change by counting the voxel occupancy. This method can theoretically guarantee high accuracy, but when the voxel size is small, the calculation accuracy increases while the memory occupation and computational complexity increase sharply, and the processing speed decreases significantly; when the voxel size is large, the calculation can be accelerated, but the accuracy is insufficient, which is difficult to meet the dual requirements of real-time performance and accuracy in dynamic monitoring scenarios.

[0006] Therefore, the existing methods generally have the problems of insufficient real-time performance, high computational complexity or unstable measurement accuracy in the variable rate fertilization and intelligent agricultural scenarios, and there is an urgent need for a volume change measurement method that can reduce the consumption of computing resources while maintaining high accuracy and high real-time performance, to provide technical support for the control of fertilization amount and agricultural production automation in intelligent variable rate fertilization. SUMMARY

[0007] The application aims to provide a multi-sensor fusion accelerated voxel volume measurement method, solve the problems of high computational complexity, large memory occupation and insufficient real-time performance in the prior art based on voxel volume measurement, and realize fast and accurate measurement of the volume change of storage spaces such as fat warehouses and grain warehouses through a coarse voxel fast estimation and neural network correction mechanism.

[0008] To achieve the above-mentioned purpose, the application provides a multi-sensor fusion accelerated voxel volume measurement method, comprising the following steps:

[0009] S1, point cloud collection at different time points, point cloud data is collected at different time points by using two depth cameras or laser radar cameras, and multi-sensor fusion at the same time point is realized through calibration parameters;

[0010] S2, point cloud preprocessing and data calibration, the fusion point clouds at different time points are preprocessed and unified in space reference, a three-dimensional reconstruction model of the time period is constructed, and an accurate volume change is obtained through an external calibration method;

[0011] S3, coarse voxel volume fast estimation, voxelization processing is performed on the model, and calculation is performed under the condition of reducing resolution, to obtain a coarse voxel volume change initial value of the time period;

[0012] S4, multi-sensor fusion network correction, the coarse voxel volume initial value and the multi-sensor feature difference value are input into a pre-trained neural network model, and a high-precision volume change result is output.

[0013] Preferably, the calibration in S1 includes internal parameter calibration, external parameter calibration and time synchronization.

[0014] Preferably, S2 comprises the following steps:

[0015] S201, preprocessing at time points: denoising, registration and triangular mesh reconstruction are performed on the fusion point clouds at each time point, to obtain three-dimensional models at each time point;

[0016] S202, time period construction and calibration: the three-dimensional models at the two different time points are unified to the same space reference, the point cloud / model data of the time period is constructed, and the accurate volume change of the time period is obtained through an external actual measurement method, while the multi-sensor change feature difference value is extracted.

[0017] Preferably, the preprocessing in S201 includes statistical outlier filtering, voxel downsampling, normal estimation, mask segmentation, and global initial registration and local fine registration are sequentially performed; the three-dimensional reconstruction is preferably Poisson reconstruction, and the alternative Alpha-Shape or Ball-Pivoting is combined with hole filling and smoothing.

[0018] Preferably, the external calibration in S202 is obtained by weighing conversion, volumetric scale, liquid level, or drainage method; and the multi-sensor feature difference includes coverage / shading rate, point density and normal variance, reflection intensity and distance distribution, sensor posture jitter, and environmental measurement.

[0019] Preferably, the voxelization processing in S3 is performed by setting a down-sampling factor to replace to replace In the same spatial range, the number of voxels and the computational complexity are approximately reduced by a proportion based on surface area or column-by-column integral method or voxel filling method compared with the fine voxel scheme, and the coarse voxel volume change initial value of the time period is quickly calculated under the condition of reducing the resolution.

[0020] Preferably, S4 includes a training phase and an application phase, the accurate volume change quantity obtained by external calibration is used as a supervised label in the training phase, the network is a lightweight multilayer perceptron or convolutional network, and the loss function adopts Huber / L2 and cooperates with a regularization term and a physical prior constraint.

[0021] Preferably, the application phase in S4 is subjected to normalization processing and then forward inference is performed to obtain a high-precision volume change result, and the output uncertainty is used for quality control or online calibration.

[0022] Therefore, the method for multi-sensor fusion and accelerated voxel volume measurement is adopted, the down-sampling factor is set to perform coarse voxel estimation, and the number of voxels and the computational complexity are reduced by a square or cubic order, thereby effectively relieving the resource consumption problem of the high-precision voxel method; the error of the coarse voxel estimation is compensated through the multi-sensor feature difference and the neural network correction, and the low complexity and the high precision are realized.

[0023] The technical solutions of the present application will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0024] Fig. 1 The flowchart of the method for multi-sensor fusion and accelerated voxel volume measurement of the present application is shown in the figure.

[0025] Fig. 2 The point cloud diagram at a time point / time period point of the present application is shown in the figure, (a) is the fusion point cloud, (b) is the fusion point cloud, and (c) is ,​ superimposed point cloud;

[0026] Fig. 3 For the coarse / fine voxel contrast map of the present application, (a) is the volume change of the volume element , (b) is the volume change of the volume element , (c) is the volume change of the volume element , (d) is the volume change of the volume element . DETAILED DESCRIPTION

[0027] The technical solutions of the present application are further described below through the drawings and examples.

[0028] Unless otherwise defined, the technical terms or scientific terms used in the present application shall be understood as the usual meanings understood by those skilled in the art to which the present application belongs. The terms "first", "second" and the like used in the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connected" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] EMBODIMENT

[0030] Referring to Figs. 1-3 , the present application provides a method for multi-sensor fusion accelerated voxel volume measurement, comprising the following steps:

[0031] S1, point cloud acquisition at different time points, point cloud data of a target area is collected at different time points by using two depth cameras or laser radar cameras respectively, and multi-sensor point clouds at the same time point are unified to the same coordinate system based on calibration parameters to obtain the fusion point cloud at the time point.

[0032] Calibration includes internal and external parameter calibration and time synchronization; external parameter calibration can use common calibration board or robust registration based on scene features, time synchronization can use hardware trigger or high-precision timestamp; IMU / odometer can be used to suppress jitter error.

[0033] S2. Point cloud preprocessing and data calibration: Preprocess the fused point clouds at different time points to obtain the 3D reconstruction models at each time point; unify the 3D reconstruction models at different times to the same spatial reference to construct point cloud data for a time period, and calibrate the precise volume change of the time period through external actual measurement methods, while extracting the difference of multi-sensor change features.

[0034] S201. Preprocessing within each time point: Denoise, register, and reconstruct triangular meshes for the fused point clouds at each time point to obtain the 3D model at each time point.

[0035] S202, Time Period Construction and Calibration: Unify the three-dimensional models of the two different times to the same spatial reference, construct the point cloud / model data of the time period, and obtain the accurate volume change of the time period through external actual measurement methods, while extracting the difference of multi-sensor change features.

[0036] Preprocessing includes statistical outlier filtering, voxel downsampling, normal estimation, and mask segmentation, followed by global initial matching (e.g., feature + RANSAC) and local fine matching (e.g., point-to-surface ICP / color ICP). Voxel downsampling is used for noise reduction, and mask segmentation is used to remove non-target areas such as warehouse walls. 3D reconstruction preferably uses Poisson reconstruction, with Alpha-Shape or Ball-Pivoting as alternatives, combined with hole filling and smoothing. External calibration can be obtained through weighing conversion, volume scale, liquid level, or water injection / displacement methods. Multi-sensor feature differences can include coverage / occlusion rate, point density and normal variance, reflection intensity and distance distribution, sensor attitude jitter, and environmental measurements.

[0037] S3. Rapid estimation of coarse voxel volume: The triangular mesh model is voxelized and calculated under reduced resolution to obtain the initial value of the coarse voxel volume change during this time period.

[0038] Set the downsampling factor The voxel side length is changed from Replace with Within the same spatial range, the number of voxels and computational complexity are approximately equal compared to the fine voxel scheme. (Based on column-by-column integration / surface voxelization) or (Based on voxel filling) The ratio decreases, thereby quickly obtaining the initial value of the volume change of coarse voxels; coarse voxel calculation can be achieved by using the occupancy difference / column interval counting or TSDF difference method.

[0039] S4. Multi-sensor fusion network correction: The initial value of the coarse voxel volume and the difference between the multi-sensor change features are used as input. During the training phase, the precise volume change is used as the supervision label to train the neural network model. During the application phase, the trained model is used to output the high-precision volume change.

[0040] The accurate volume change amount obtained by external calibration is taken as a supervision label in the training stage, the network can be a lightweight multilayer perceptron or a convolutional network, and the loss function can adopt Huber / L2 and cooperate with a regular term and a physical prior constraint; in the application stage, the input is normalized and then forward inference is performed to obtain a high-precision volume change result, and if necessary, an uncertainty can be output for quality control or online calibration.

[0041] The volume calculation result is a volume difference between two time points, and an absolute volume value at any time point can be optionally output.

[0042] The "time point" in the application is denoted as (also applicable to multiple time points), a fused point cloud is obtained by acquiring and fusing multiple sensors at the same time point; the "time period" refers to . The world coordinate system is denoted as , and the coordinate system of each sensor is denoted as .

[0043] A multi-sensor fusion accelerated voxel volume measurement method, comprising the following steps:

[0044] S1, point cloud acquisition at different time points.

[0045] At different time points , two depth cameras or laser radars are used to collect target area point clouds. Through internal / external parameter calibration and time synchronization, the multi-sensor point clouds at the same time point are transformed by the external parameter and unified to and fused to obtain the fused point clouds at each time point . The internal parameter calibration uses a calibration board, the external parameter calibration uses a calibration board, the time synchronization uses hardware triggering or high-precision timestamp alignment, and the IMU / odometer is used for anti-shake.

[0046] S2, point cloud preprocessing and data calibration, the object is .

[0047] S201, preprocessing at a time point.

[0048] The is subjected to statistical outlier filtering, voxel downsampling (only for noise reduction, which does not affect the of the subsequent coarse voxel step), normal estimation and mask segmentation (to remove non-medium regions such as tank walls), and two-level registration is performed: global initial registration: based on FPFH / SIFT features + RANSAC to obtain a coarse pose; local fine registration: point-surface ICP / color ICP converges to millimeter / centimeter level; then a watertight model is obtained by triangular mesh reconstruction (Poisson reconstruction is preferred; Alpha-Shape and Ball-Pivoting are alternatives, with hole filling / smoothing performed).

[0049] S202, Time Period Construction and Calibration.

[0050] Will Unify to the same spatial reference, Based on the CAD data of the warehouse, data for a specific time period is generated. Please see Fig. 2 The precise volume change over a given period can be obtained through external measured weighing conversion, volume scale, liquid level / meter markings, or water injection / drainage methods. Simultaneously, multi-sensor variation feature difference vectors are extracted from the data at two time points. It may include, but is not limited to: sampling coverage and occlusion: visibility, overlap rate, and view distribution histogram; quality indicators: point density variation, normal variance, roughness, and residual histogram; sensors and environment: reflection intensity / confidence statistics, distance distribution, IMU jitter, temperature and humidity, flow sensor, and speed sensor.

[0051] S3, rapid estimation of bold element volume.

[0052] Will Voxelization to a uniform coarse resolution raster voxel side length To obtain the set of occupied voxels , Or corresponding TSDF field Please see Fig. 3 .

[0053] Calculate initial values ​​of volume change at coarse resolution The two preferred implementations are as follows:

[0054] Option A: Occupation difference / column interval counting (general 3D).

[0055] Surface voxelization: Based on BVH / uniform mesh, the intersection of triangular patches and voxels is calculated, and only near-surface voxels (sparse / hash voxels) are labeled.

[0056] Column-by-column scan: for each The intersection points between the surface and the data at two different time points are counted and sorted. The thickness of the interlayer is obtained by pairing odd and even points. .

[0057] Volume accumulation plus: .

[0058] Option B: TSDF Differential (Robust Parallel).

[0059] TSDF fusion: Fusion as , truncated distance .

[0060] Zero-crossing extraction: along fixed column (gravity direction) or wall-normal column, find zero-crossing of .

[0061] Volume accumulation: .

[0062] Explanation, complexity and resource overhead derivation.

[0063] Let the fine voxel resolution be , voxel edge length be , surface area be , and triangle number be .

[0064] When the down-sampling factor , the coarse voxel edge length , and the grid size is approximately .

[0065] Based on voxel filling (traverse 3D grid):

[0066] The fine voxel time / space main term is .

[0067] The coarse main term is approximately:

[0068] ;

[0069] Therefore, the time and storage are reduced by approximately .

[0070] Based on column-wise integration / surfelization (2D main term);

[0071] The cost of surfelization and intersection is approximately proportional to the number of surface area covered voxels:

[0072] ;

[0073] The column-wise statistic main term is , and the coarse is ;

[0074] If the surface intersection index uses BVH / Uniform grid, the construction term is approximately , which can be reused in adjacent batches and treated as one-time or sub-main term.

[0075] Therefore, the geometry main term time and storage overhead of column-wise / surface route are reduced by approximately .

[0076] When the voxel edge length is reduced by​ Expand to This invention achieves approximately [a certain result] in the voxel filling route. Level-based power consumption reduction, achieving approximately [amount missing] on per-column / surface routes. Level-based power consumption reduction; subsequently, learning-based error compensation is performed on the initial value of the volume change of coarse voxels through multi-sensor features and neural networks, achieving a balance between efficiency and accuracy.

[0077] S4, Multi-sensor fusion network correction.

[0078] by As input, output high-precision volume change. The training and reasoning are as follows:

[0079] S401, Training Phase.

[0080] Construct normalized sample pairs A lightweight neural network is trained using regression loss. :

[0081]

[0082] ;

[0083] The preferred network is a multilayer perceptron (input dimension) ,depth Layer, width It can be either a one-dimensional lightweight convolution or a physical prior constraint (non-negativity, upper and lower bounds, monotonicity) as a soft constraint term.

[0084] S402, Application Phase.

[0085] Real-time computing and Data normalized and trained Output As a high-precision measure of volume change. Optionally, the uncertainty (MCdropout / deep integration) can be output as a confidence interval for online quality control.

[0086] Preferred and Deformation Scheme: Hierarchical / Adaptive Voxels: Enable smaller voxels near the interface or in high curvature / high uncertainty regions, while keeping coarse voxels in other regions to further reduce computational cost;

[0087] Container prior: Based on the hull CAD / wall mesh, establish a "quasi-2.5D" column in the wall normal direction to reduce the effect of overhang;

[0088] Online calibration: periodic fine-tuning with a small number of external measurement samples. (Low learning rate, freeze the previous layer) to suppress environment / material drift;

[0089] Boundary handling: Integrate only in the region that is visible at both time points; apply morphological closing operation or TSDF regularization to thin layers / holes.

[0090] Parameter recommendations: In grain silo scenarios, the inner diameter of the silo... Surface undulations At that time, the reconstructed grid resolution can be chosen. Bold element coefficient TSDF truncation Gravity is taken in each column direction. Network input includes Column coverage, boundary voxel ratio, normal variance, intensity / distance histogram, etc., are used as training loss parameters. Regular expressions.

[0091] Therefore, the present invention employs the above-mentioned method for accelerating voxel volume measurement through multi-sensor fusion, which has the following advantages: significantly improved computational efficiency by setting a downsampling factor. For coarse voxel estimation, the number of voxels and computational complexity increase with... The voxelization decreases by square or cubic order, effectively alleviating the resource consumption problem of high-precision voxel methods. Measurement accuracy is guaranteed; through multi-sensor feature difference and neural network correction, errors in coarse voxel estimation are compensated, achieving a balance between low complexity and high accuracy. It has strong applicability; the method can be extended to different storage space scenarios, such as fertilizer warehouses and grain warehouses, possessing real-time performance and robustness, enabling dynamic volume change monitoring in agriculture, warehousing, and logistics management. Implementation is simple and scalable; the method is compatible with multiple types of sensors and different voxelization implementation paths (fill / column-by-column / TSDF), possessing strong engineering feasibility and scalability. By combining multi-sensor fusion, coarse voxel acceleration, and network correction, it overcomes the bottleneck of traditional voxel computation efficiency and accuracy, significantly reducing computational complexity and memory overhead while maintaining high accuracy and real-time performance in volume change measurement, demonstrating significant application value.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method of multi-sensor fusion accelerated voxel volume measurement, characterized by, Comprising the following steps: S1, point cloud collection at different time points, point cloud data is collected at different time points by using two depth cameras or laser radar cameras, and multi-sensor fusion at the same time point is realized through calibration parameters; S2, point cloud preprocessing and data calibration, preprocessing and unified spatial reference of the fusion point cloud at different time points, construction of a three-dimensional reconstruction model in a time period, and obtaining accurate volume change through an external calibration method; S3, rough voxel volume rapid estimation, voxel processing of the model, and calculation under the condition of reducing resolution to obtain the rough voxel volume change initial value in the time period; S4, multi-sensor fusion network correction, inputting the rough voxel volume initial value and multi-sensor feature difference into a pre-trained neural network model to output a high-precision volume change result; The voxelization process in S3 sets a down-sampling factor The voxel grid is coarsened, and the voxel edge length is changed from to In the same spatial range, the number of voxels and the computational complexity are approximately reduced by the ratio based on the surface area or the column-by-column integration method or the voxel filling method. The coarse voxel volume change initial value of the time period is quickly calculated under the condition of reducing the resolution.

2. The method of claim 1, wherein: The calibration in S1 includes internal parameter calibration, external parameter calibration and time synchronization.

3. The method of claim 2, wherein, S2 includes the following steps: S201, preprocessing within the time point: denoising, registration and triangular mesh reconstruction of the fusion point cloud at each time point to obtain a three-dimensional model at each time point; S202, time period construction and calibration: unify the three-dimensional models at two different time points to the same spatial reference, construct the point cloud / model data in the time period, and obtain the accurate volume change in the time period through an external actual measurement method, while extracting the multi-sensor change feature difference.

4. The method of claim 3, wherein: The preprocessing in S201 includes statistical outlier filtering, voxel downsampling, normal estimation, mask segmentation, and sequentially performing global initial registration and local fine registration; the three-dimensional reconstruction adopts Poisson reconstruction, and is assisted by hole filling and smoothing.

5. The method of claim 4, wherein: The three-dimensional reconstruction adopts Alpha-Shape or Ball-Pivoting and is assisted by hole filling and smoothing.

6. The method of claim 4, wherein: The external calibration in S202 is obtained by weighing conversion, volume scale, liquid level or drainage method; The multi-sensor feature difference includes coverage / occlusion rate, point density and normal variance, reflection intensity and distance distribution, sensor attitude jitter and environmental measurement.

7. The method of claim 6, wherein: S4 includes a training stage and an application stage, the accurate volume change obtained by external calibration is used as a supervision label in the training stage, the network is a lightweight multilayer perceptron or convolutional network, and the loss function adopts Huber / L2 and is assisted by regular term and physical prior constraint.

8. The method of claim 7, wherein: In the application stage of S4, the high-precision volume change result is obtained by forward inference after normalization processing, and the uncertainty is output for quality control or online calibration.

Images