Crop high-fidelity three-dimensional reconstruction method and system

Abstract

The invention provides a crop high-fidelity three-dimensional reconstruction method and system, and belongs to the technical field of three-dimensional reconstruction, and the method comprises the following steps: collecting multi-modal data of a target crop through multi-sensor fusion, including a visible light image, depth information and multispectral data; performing space-time alignment and preprocessing on the acquired data, and eliminating environmental noise and motion blur; constructing an initial three-dimensional point cloud model based on a deep learning network, and enhancing model detail fidelity through an adaptive optimization algorithm; and dynamically correcting the three-dimensional model in combination with the growth characteristics of crops to generate a high-precision and high-fidelity three-dimensional reconstruction result. Through multi-sensor fusion, deep learning modeling and dynamic optimization correction, the problems of low model fidelity and poor dynamic adaptability in the prior art are solved, geometric and physiological features of crops can be accurately restored, and high-precision data support is provided for intelligent agriculture.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a method and system for high-fidelity three-dimensional reconstruction of crops. Background Art

[0002] High-fidelity 3D reconstruction of crops involves collecting crop phenotypic data from multiple sensors and combining it with computer vision and deep learning algorithms to construct 3D digital models with high geometric accuracy, realistic texture detail, and dynamic growth characteristics. Its core goal is to accurately reproduce the morphological structure (such as leaf curvature and stem branching) and physiological state (such as chlorophyll distribution and disease characteristics) of crops, providing a data foundation for precision agriculture. 3D reconstruction technology is a key digital tool driving the development of precision agriculture. It can comprehensively visualize the spatial structure and morphological characteristics of plants, laying the foundation for phenotypic analysis, yield prediction, and breeding selection.

[0003] The existing crop 3D reconstruction technology has the following problems: Limitations of single-modal data: Relying on a single sensor (such as an RGB camera) leads to a lack of model details, especially insufficient accuracy under complex lighting or occlusion conditions; Poor dynamic adaptability: The static model fails to reflect the actual growth state because it does not take into account crop growth deformation and environmental interference; Low algorithm efficiency: Traditional point cloud processing methods require large amounts of computation and are unable to meet the needs of real-time farmland monitoring. Summary of the Invention

[0004] The purpose of this invention is to provide a high-fidelity 3D reconstruction method and system for crops, addressing existing technical issues. This method addresses low model fidelity caused by insufficient multi-source data fusion; interference from dynamic growth and environmental factors on 3D reconstruction; and the need to balance reconstruction efficiency and accuracy in complex scenarios. The method is suitable for scenarios such as crop phenotyping, precision agriculture management, and pest and disease monitoring.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A high-fidelity three-dimensional reconstruction method for crops, comprising the following steps: Collect multimodal data of target crops through multi-sensor fusion, including visible light images, depth information and multispectral data; Perform spatiotemporal alignment and preprocessing on the collected data to eliminate environmental noise and motion blur; Build an initial 3D point cloud model based on a deep learning network and enhance the model detail fidelity through an adaptive optimization algorithm; The 3D model is dynamically modified based on the growth characteristics of crops to generate high-precision and high-fidelity 3D reconstruction results.

[0006] Furthermore, the multi-sensor includes a high-resolution RGB camera, a laser radar (LiDAR) and a multi-spectral imager, and the multi-sensor layout adopts multi-perspective synchronous acquisition and control. Furthermore, the spatiotemporal alignment and preprocessing include: Inter-frame alignment of multi-sensor data based on feature point matching; Use lighting compensation algorithm to eliminate shadows and overexposed areas; Correct image blur caused by slight swaying of crops using a motion estimation model. Furthermore, the deep learning network is an improved 3D convolutional neural network (CNN), whose input is fused multimodal data and output is a three-dimensional point cloud with texture and geometric details. Furthermore, the adaptive optimization algorithm includes: Local refinement of point clouds based on surface curvature analysis; Introducing prior knowledge of crop morphology to perform topological optimization on structures such as leaves and stems; Generative Adversarial Networks (GANs) are used to enhance the realism of model surface textures. Furthermore, the dynamic correction includes: Adjust the model scale according to crop growth cycle parameters; Combine environmental sensor data (temperature, humidity, light intensity) to predict model deformation trends and update them.

[0007] A high-fidelity three-dimensional reconstruction system for crops, comprising: Multimodal data acquisition module, integrating RGB camera, lidar and multispectral imager; Data processing and fusion module, used for spatiotemporal alignment, noise elimination and multi-source data fusion; 3D modeling and optimization module, which generates high-fidelity models based on deep learning networks and adaptive algorithms; Dynamic update module, combining growth and environmental data to modify the 3D model in real time.

[0008] Furthermore, the system supports being mounted on drones or ground mobile platforms to achieve large-scale dynamic scanning of farmland.

[0009] Multimodal data fusion: combining visible light, depth, and multispectral data to comprehensively capture crop geometry and physiological characteristics; Deep Learning and Adaptive Optimization: Using an improved 3D Convolutional Neural Network (CNN) to generate an initial model, and enhancing details through curvature analysis, morphological priors, and generative adversarial networks (GANs); Dynamic model updating: Integrates growth parameters and environmental data to enable real-time model correction as crops grow.

[0010] The present invention has the following beneficial effects due to the adoption of the above technical solution: This invention uses multiple sensors to collaboratively improve data integrity and enhance the ability to restore details such as leaf texture and stem curvature; it uses an adaptive optimization algorithm to balance efficiency and accuracy, and is suitable for complex field environments; the dynamic correction mechanism makes the model highly consistent with the actual growth state, supporting long-term phenotypic tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 3D reconstruction process flow diagram of the present invention; Figure 2 This is a schematic diagram of the system provided by Example 2 of the present invention. DETAILED DESCRIPTION

[0012] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and by way of preferred embodiments. However, it should be noted that many of the details listed in this specification are merely provided to help the reader gain a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be practiced even without these specific details.

[0013] like Figure 1 As shown, a high-fidelity three-dimensional reconstruction method for crops includes the following steps: Step 1: Collect multimodal data of target crops through multi-sensor fusion, including visible light images, depth information, and multispectral data; The multi-sensor includes a high-resolution RGB camera, a lidar and a multi-spectral imager, and the multi-sensor layout adopts multi-perspective synchronous acquisition and control. Step 2: perform spatiotemporal alignment and preprocessing on the collected data to eliminate environmental noise and motion blur; The spatiotemporal alignment and preprocessing include: Inter-frame alignment of multi-sensor data based on feature point matching; Use lighting compensation algorithm to eliminate shadows and overexposed areas; Correct image blur caused by slight swaying of crops using a motion estimation model.

[0014] Step 3: Build an initial 3D point cloud model based on a deep learning network and enhance the model detail fidelity through an adaptive optimization algorithm; The deep learning network is an improved 3D convolutional neural network, whose input is fused multimodal data and output is a 3D point cloud with texture and geometric details; The adaptive optimization algorithm includes: Local refinement of point clouds based on surface curvature analysis; Introducing prior knowledge of crop morphology to perform topological optimization on structures such as leaves and stems; Enhance the realism of model surface texture using generative adversarial networks. Step 4: Dynamically modify the 3D model based on the growth characteristics of the crops to generate high-precision, high-fidelity 3D reconstruction results; The dynamic correction includes: Adjust the model scale according to crop growth cycle parameters; Combine environmental sensor data (temperature, humidity, light intensity) to predict model deformation trends and update them.

[0015] A high-fidelity three-dimensional reconstruction system for crops, comprising: Multimodal data acquisition module, integrating RGB camera, lidar and multispectral imager; Data processing and fusion module, used for spatiotemporal alignment, noise elimination and multi-source data fusion; 3D modeling and optimization module, which generates high-fidelity models based on deep learning networks and adaptive algorithms; Dynamic update module, combining growth and environmental data to modify the 3D model in real time; The system supports being mounted on drones or ground mobile platforms to achieve large-scale dynamic scanning of farmland.

[0016] Example 1 3D reconstruction of wheat plants A drone equipped with a multi-sensor module was used to scan the wheat field at low altitude, collecting RGB images, LiDAR point clouds, and near-infrared spectral data; The data processing module aligns multi-source data and eliminates blurring caused by wind-induced plant swaying; 3D CNN generates the initial point cloud model, refines the ear structure through curvature analysis, and GAN enhances the leaf texture; Combined with temperature and humidity sensor data, the plant growth for the next day is predicted and the model height parameters are updated.

[0017] Data collection and component configuration: The drone used is the DJI M300 RTK model, which is used to carry sensors and automatically scan routes; A high-resolution RGB camera, a Sony α7R IV (45MP, 35mm focal length), was used to capture leaf texture and disease details. The laser radar uses a lightweight LiDAR model, Hesai PandarXT-32 (ROI scanning mode, point cloud density >200 points / m²), to obtain the 3D point cloud of the stems; The multispectral imager used was the MicaSense Altum-PT model (5 bands: blue, green, red, red edge, and near infrared) to extract the chlorophyll distribution index; The temperature and humidity sensor data uses a temperature, humidity and light intensity integrated probe to monitor the growth microenvironment in real time.

[0018] Scan parameters: The scanning area is a 10m×10m wheat field in the heading period, the flight height is 5m, the speed is 1m / s, the overlap rate is 80%, and the single scanning time is 15 minutes.

[0019] Data processing flow and key algorithms: 1. Spatiotemporal alignment of multi-source data enter: RGB image sequence (300 images) LiDAR point cloud (approximately 2.5×10 6 point) Multispectral data (5-band reflectance matrix) Alignment method: Frame synchronization based on GPS timestamp aligned_data = synchronize( rgb_frames, lidar_points, multispectral_data, gps_timestamps, imu_orientation ) Output: Spatial registration error < 2cm.

[0020] 2. Dynamic deblurring optimization: Problem: Wind speed of 3.5m / s causes blurred plant swaying (PSNR=22.1dB) Solution: Motion estimation model (optical flow + LSTM trajectory prediction); Generative Adversarial Network (DeblurGAN-v2); Results: PSNR increased to 28.7dB after deblurring.

[0021] 3. 3D modeling and adaptive optimization Initial point cloud generation: Improve 3D CNN (structure see Table 1) to generate 1024×3 point cloud; Geometric refinement, local densification based on curvature analysis (radius 5mm), wheat ear void ratio increased from 31% to 4.2%; Texture enhancement, GAN network (input multispectral + RGB), leaf rust SSIM increased from 0.68 to 0.91.

[0022] Table 1 3D CNN network structure 4. Dynamic Growth Correction Input environmental data: temperature 28°C, humidity 65%, light intensity 1500μmol / m² / s; Prediction model: ΔH = 0.173 × T - 0.002 × RH + 0.0005 × Light Output: The predicted value of plant height growth in 24 hours is 2.41 cm (the measured value is 2.38±0.15 cm).

[0023] The verification results are shown in Table 2; Table 2 Verification results Example 2 Dynamic monitoring of fruit trees in orchards (taking citrus orchards as an example) The ground mobile platform moves along the rows of fruit trees, LiDAR obtains the three-dimensional outline of the tree trunk, and the multispectral camera captures the leaf reflectivity; After data fusion, the connection topology between fruits and branches is optimized based on morphological prior knowledge; The model is updated every 24 hours and combines historical growth data to predict fruit enlargement trends.

[0024] like Figure 2 As shown, key data verification is performed according to the system schematic diagram: 1. Repair of branch topology Input original point cloud: broken branches account for 18.7% Output optimization model: Based on morphological prior rules (main branch angle > 30° prohibits connecting fruits); The connection accuracy rate reached 98.6%.

[0025] 2. Visualization of Nutritional Stress Hyperspectral Data Mapping: Chlorophyll content gradient → pseudo-color texture; Accuracy of identifying nutrient-deficient areas: 92.8% (validation sample N=150 leaves).

[0026] 3. Fruit expansion prediction Training data: diameter series of the past 30 days (sampling interval 2h); Prediction model: Time Series Transformer; Result: The expansion curve R²=0.96 for the next 7 days.

[0027] Example 3 Greenhouse lettuce disease monitoring (precision plant protection scenario) End-to-end data link 1. Close range scanning Kinect v3 depth camera (effective distance 0.5-1m) Multispectral imaging plate (wavelength 680nm / 750nm dual channel) 2. Disease identification and location 3D leaf segmentation, data input: depth point cloud + RGB; data output: leaf spatial coordinates (error <1mm); Spectral feature extraction, data input: 680nm / 750nm reflectance ratio; data output: downy mildew confidence (0-1); Disease spread simulation, data input: historical infection area coordinates; data output: 72h diffusion heat map.

[0028] 3. Verify the results Disease detection accuracy: 93.4% (compared to manual labeling) Diffusion prediction error: The root mean square error (RMSE) of boundary displacement is 2.3 cm.

[0029] The technical advantages of the three-dimensional reconstruction method and system of the present invention are summarized in Table 3 below: Table 3 Data comparison between the present invention and the traditional solution The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A high-fidelity 3D reconstruction method for crops, characterized by: The following steps are involved: Collect multimodal data of target crops through multi-sensor fusion, including visible light images, depth information and multispectral data; Perform spatiotemporal alignment and preprocessing on the collected data to eliminate environmental noise and motion blur; Build an initial 3D point cloud model based on a deep learning network and enhance the model detail fidelity through an adaptive optimization algorithm; The 3D model is dynamically modified based on the growth characteristics of crops to generate high-precision and high-fidelity 3D reconstruction results.

2. The method for high-fidelity 3D reconstruction of crops according to claim 1, characterized in that: The multi-sensor includes a high-resolution RGB camera, a lidar and a multi-spectral imager, and the multi-sensor layout adopts multi-perspective synchronous acquisition and control.

3. The high-fidelity 3D reconstruction method for crops according to claim 1, characterized in that: The spatiotemporal alignment and preprocessing include: Inter-frame alignment of multi-sensor data based on feature point matching; Use lighting compensation algorithm to eliminate shadows and overexposed areas; Correct image blur caused by slight swaying of crops using a motion estimation model.

4. The method for high-fidelity 3D reconstruction of crops according to claim 1, characterized in that: The deep learning network is an improved 3D convolutional neural network, whose input is fused multimodal data and output is a three-dimensional point cloud with texture and geometric details.

5. The method for high-fidelity 3D reconstruction of crops according to claim 1, characterized in that: The adaptive optimization algorithm includes: Local refinement of point clouds based on surface curvature analysis; Introducing prior knowledge of crop morphology to perform topological optimization on structures such as leaves and stems; Enhance the realism of model surface texture using generative adversarial networks.

6. The method for high-fidelity 3D reconstruction of crops according to claim 1, characterized in that: The dynamic correction includes: Adjust the model scale according to crop growth cycle parameters; Combine environmental sensor data (temperature, humidity, light intensity) to predict model deformation trends and update them.

7. A high-fidelity 3D reconstruction system for crops according to any one of claims 1 to 6, characterized in that: include: Multimodal data acquisition module, integrating RGB camera, lidar and multispectral imager; Data processing and fusion module, used for spatiotemporal alignment, noise elimination and multi-source data fusion; 3D modeling and optimization module, which generates high-fidelity models based on deep learning networks and adaptive algorithms; Dynamic update module, combining growth and environmental data to modify the 3D model in real time.

8. The high-fidelity 3D crop reconstruction system according to claim 7, characterized in that: The system supports being mounted on drones or ground mobile platforms to achieve large-scale dynamic scanning of farmland.

Images