Plant phenotype platform three-dimensional imaging method and device based on multi-sensor fusion

Abstract

The invention provides a plant phenotype platform three-dimensional imaging method and device based on multi-sensor fusion, and the method comprises the steps: collecting the original three-dimensional point cloud data of a plant through a plurality of heterogeneous imaging sensors carried on a plant phenotype platform, and carrying out the time synchronization processing and space registration processing. And dynamically calculating the confidence coefficient weight of each data point according to the environment illumination condition, the measurement attribute of each data point in the registration point cloud set and the characteristics of the imaging sensor to which the data point belongs, and obtaining a point cloud set with a point-level confidence coefficient weight. Performing adaptive weighted fusion processing according to the spatial distribution of the point cloud set to generate a preliminary fusion point cloud model; and according to a quality index of the model and a real-time environment illumination condition, dynamically adjusting and calculating a weight parameter of a confidence coefficient weight, and finally outputting an optimized three-dimensional point cloud model after iterative optimization. The defect that a fusion result of an existing plant three-dimensional phenotype accurate acquisition scheme is insufficient in stability and precision in a complex and changeable field environment is overcome.

Description

Technical Field

[0001] This invention relates to the field of intelligent agriculture technology, and in particular to a three-dimensional imaging method and device for a plant phenotyping platform based on multi-sensor fusion. Background Technology

[0002] Precise acquisition of plant three-dimensional phenotypic data is one of the key technologies in modern smart agriculture and gene breeding. Currently, the acquisition of three-dimensional phenotypic information of crops in the field mainly relies on mobile platforms equipped with various three-dimensional sensors, such as track-based platforms and unmanned vehicle platforms. These platforms achieve non-destructive measurement of phenotypic parameters such as crop canopy structure, plant height, and canopy width through motion scanning.

[0003] However, existing methods for precise acquisition of plant 3D phenotypic data typically employ fixed weights or simple data stitching strategies, lacking the ability to perceive and respond to real-time sensor operating status and changes in the external environment. They cannot adaptively adjust the contribution of each sensor based on ambient light intensity, nor can they dynamically evaluate and optimize point cloud quality during the fusion process, resulting in insufficient stability and accuracy of the fusion results in complex and variable field environments. Summary of the Invention

[0004] This invention provides a three-dimensional imaging method and device for a plant phenotypic platform based on multi-sensor fusion, to address the shortcomings of existing precise acquisition schemes for plant three-dimensional phenotypic data, which suffer from insufficient stability and accuracy in complex and variable field environments. The technical solution proposed by this invention is as follows: In a first aspect, the present invention provides a three-dimensional imaging method for a plant phenotypic platform based on multi-sensor fusion, comprising: The original three-dimensional point cloud data of the plant is collected by a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform to obtain a multi-source asynchronous original point cloud set. The multi-source asynchronous original point cloud is subjected to time synchronization processing and spatial registration processing to obtain a registration point cloud under a unified spatiotemporal coordinate system; Based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, the confidence weight of each data point is dynamically calculated to obtain a point cloud with point-level confidence weights. For the point cloud set with point-level confidence weights, an adaptive weighted fusion process is performed based on its spatial distribution to generate a preliminary fused point cloud model. Based on the quality indicators of the preliminary fused point cloud model and the real-time ambient lighting conditions, the weight parameters for calculating the confidence weights are dynamically adjusted. After iterative optimization, the optimized 3D point cloud model is finally output.

[0005] Optionally, the time synchronization process is to assign a unified timestamp to the multi-source asynchronous original point cloud through hardware pulse signals and a unified timestamp server, thereby obtaining a time synchronization point cloud; The spatial registration process involves transforming the time synchronization point cloud to the same platform coordinate system based on a pre-calibrated sensor extrinsic parameter matrix, and then performing registration using an online point cloud registration algorithm to obtain the registration point cloud in the unified spatiotemporal coordinate system.

[0006] Optionally, the measurement attributes include measurement distance and reflectivity; the sensor characteristics include sensor type and reference weight parameters; and the ambient lighting conditions are obtained by real-time monitoring by an illuminance sensor.

[0007] Optionally, the multiple heterogeneous imaging sensors include a first type of lidar, a second type of lidar, and a time-of-flight depth camera; the dynamic calculation of the confidence weight for each data point includes: For data points from the first type of lidar and the second type of lidar, the confidence weight is determined at least based on the reference weight coefficient of the lidar to which it belongs, the measurement distance of the data point, the reflectivity of the data point, a distance attenuation coefficient, and a fusion closed-loop dynamic adjustment coefficient. For a data point from the time-of-flight depth camera, its confidence weight is determined based at least on the baseline weight coefficient of the time-of-flight depth camera, the pixel saturation of the data point, the reflectivity of the data point, the real-time ambient light intensity, and the illumination attenuation adjustment coefficient.

[0008] Optionally, the quality indicators include point cloud noise level, point cloud density, and registration error; the weighting parameters for dynamically adjusting the confidence weights include: Adjust one or more of the following: the baseline weighting coefficient, the distance attenuation coefficient, the fusion closed-loop dynamic adjustment coefficient, and the illumination attenuation adjustment coefficient.

[0009] Optionally, the step of adaptively weighting and fusing the point cloud set with point-level confidence weights according to its spatial distribution to generate a preliminary fused point cloud model includes: For multiple points in the point cloud set with point-level confidence weights whose spatial distance is less than a preset threshold, a weighted average fusion is performed based on their confidence weights to generate a fused point. Points in non-overlapping regions are retained. The initial fused point cloud model is obtained by filtering and denoising all fused points and the retained points.

[0010] Secondly, the present invention also provides a three-dimensional imaging device for a plant phenotyping platform based on multi-sensor fusion, comprising the following modules: The data acquisition module is used to acquire the original three-dimensional point cloud data of the plant through a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform, and obtain a multi-source asynchronous original point cloud set. The spatiotemporal alignment module is used to perform time synchronization and spatial registration processing on the multi-source asynchronous original point cloud to obtain a registration point cloud under a unified spatiotemporal coordinate system. The weight calculation module is used to dynamically calculate the confidence weight of each data point based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, so as to obtain a point cloud with point-level confidence weights. The point cloud fusion module is used to perform adaptive weighted fusion processing on the point cloud set with point-level confidence weights according to its spatial distribution to generate a preliminary fused point cloud model. The closed-loop optimization module is used to dynamically adjust the weight parameters for calculating the confidence weight based on the quality indicators of the preliminary fused point cloud model and the real-time ambient lighting conditions. After iterative optimization, the optimized 3D point cloud model is finally output.

[0011] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the three-dimensional imaging method for a plant phenotypic platform based on multi-sensor fusion as described in the first aspect above.

[0012] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional imaging method for a plant phenotypic platform based on multi-sensor fusion as described in the first aspect above.

[0013] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the three-dimensional imaging method for a plant phenotypic platform based on multi-sensor fusion as described in the first aspect above.

[0014] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The present invention provides a three-dimensional imaging method and apparatus for a plant phenotyping platform based on multi-sensor fusion. This method dynamically calculates the confidence weight of each data point based on ambient lighting conditions, the measurement attributes of each data point in the registered point cloud, and the characteristics of the imaging sensor to which the data point belongs, resulting in a point cloud with point-level confidence weights. This enables real-time perception of changes in the external environment (such as lighting conditions) and the characteristics of different sensors, assigning appropriate weights to each data point based on these factors, rather than using fixed weights. This enhances the perception and feedback capabilities regarding the real-time working status of sensors and changes in the external environment. When dynamically calculating the confidence weight of each data point, ambient lighting conditions are considered as an important factor. This allows for adaptive adjustment of the contribution of data collected by each sensor according to changes in ambient light intensity, ensuring that data from each sensor is utilized reasonably under different lighting conditions, thus improving the accuracy of the acquisition results. The point cloud with point-level confidence weights undergoes adaptive weighted fusion processing based on its spatial distribution to generate a preliminary fused point cloud model. This step considers the spatial distribution of the point cloud during the fusion process and improves the point cloud quality through adaptive weighting. Based on the quality indicators of the initial fused point cloud model and real-time ambient lighting conditions, the weight parameters for calculating the confidence weights are dynamically adjusted. After iterative optimization, the optimized 3D point cloud model is finally output. This method can continuously evaluate the quality of the initial fused point cloud model during the fusion process and dynamically adjust the weight parameters in conjunction with real-time ambient lighting conditions. Through iterative optimization, the point cloud quality is further improved, thus solving the problem that existing technologies cannot dynamically evaluate and optimize point cloud quality during the fusion process. This improves the stability and accuracy of the fusion results in complex and variable field environments.

[0015] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1This is a flowchart illustrating the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of the three-dimensional imaging device for the plant phenotyping platform based on multi-sensor fusion provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The three-dimensional imaging method for plant phenotyping platforms based on multi-sensor fusion provided by this invention refers to... Figure 1 As shown, it includes the following: S110. Collect the original three-dimensional point cloud data of the plant by using a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform to obtain a multi-source asynchronous original point cloud set.

[0023] Various heterogeneous imaging sensors can be mounted on plant phenotyping platforms (such as track platforms or unmanned vehicles). These sensors have different operating principles and performance characteristics. For example, various heterogeneous imaging sensors can include time-of-flight (ToF) depth cameras, structured light cameras based on triangulation principles, and 2D or 3D lidar based on laser scanning principles. Sensors based on different principles can acquire three-dimensional information about plants from different angles and in different ways. The fusion of multi-source data helps to improve the comprehensiveness and accuracy of the acquisition results.

[0024] During the movement of the plant phenotyping platform, these heterogeneous imaging sensors scan the plant, acquiring its raw 3D point cloud data. Each sensor operates independently according to its inherent sampling frequency, field of view, and coordinate system, acquiring the 3D spatial information of the plant within its field of view in parallel. Because the sensors are physically separate and their internal clocks and sampling periods are not synchronized, their output data streams are independent in both time and space, resulting in a multi-source asynchronous raw point cloud. This raw point cloud data contains the 3D coordinate information of various points on the plant surface, but the data is scattered and asynchronous at this point, making direct fusion analysis impossible.

[0025] S120. Perform time synchronization and spatial registration processing on the multi-source asynchronous original point cloud to obtain a registration point cloud under a unified spatiotemporal coordinate system.

[0026] Because the data acquired by multiple imaging sensors are not synchronized, time discrepancies can occur during subsequent fusion, affecting the accuracy of the fusion results. Therefore, time synchronization processing is required for multi-source asynchronous raw point clouds. Time synchronization methods can employ hardware synchronization and software synchronization. Hardware synchronization establishes a precise time synchronization signal between the imaging sensors, ensuring they begin acquiring data at the same time. Software synchronization, on the other hand, records the timestamp of each imaging sensor's data acquisition and then performs alignment and interpolation operations based on these timestamps during data processing, unifying data acquired at different times onto the same time reference.

[0027] Different imaging sensors have different positions and orientations in space, and the point cloud data they acquire reside in their respective local coordinate systems. To fuse data from different imaging sensors, they need to be transformed to a unified spatiotemporal coordinate system, i.e., spatial registration. Spatial registration includes two steps: coarse registration and fine registration. Coarse registration can roughly align the point clouds from different imaging sensors by manually selecting some obvious feature points or utilizing the initial position information of the imaging sensors. Fine registration uses algorithms such as the Iterative Closest Point (ICP) algorithm, which continuously iterates and optimizes to minimize the distance between corresponding points in different point clouds, thereby achieving accurate spatial registration and obtaining a registered point cloud set in a unified spatiotemporal coordinate system.

[0028] S130. Based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, dynamically calculate the confidence weight of each data point to obtain a point cloud with point-level confidence weights.

[0029] Ambient lighting conditions significantly affect the measurement accuracy of imaging sensors. For example, under strong light, the measurement data of some imaging sensors may be inaccurate; while in low-light environments, the signal-to-noise ratio of imaging sensors may decrease. Therefore, this invention assigns different weights to data points acquired under different lighting conditions based on real-time ambient lighting conditions. Data points acquired under suitable lighting conditions have higher weights, while data points acquired under poor lighting conditions have lower weights.

[0030] Each data point in the registration point cloud has its own measurement attributes, such as reflectance intensity and distance error. Reflectance intensity reflects the ability of an object's surface to reflect light; different plant parts or different materials may have different reflectance intensities. Distance error represents the distance deviation between the measured point and the actual object surface. Based on these measurement attributes, the reliability and accuracy of the data points can be evaluated. For example, data points with moderate reflectance intensity and small distance error have higher weights, while data points with abnormal reflectance intensity or large distance error have lower weights.

[0031] Different imaging sensors have different performance characteristics and error models. For example, some imaging sensors have high accuracy at close range but decrease accuracy at long range; some imaging sensors perform well in measuring objects of specific colors or materials. Therefore, it is necessary to assign appropriate weights to data points based on the characteristics of the imaging sensor to which they belong. By comprehensively analyzing ambient lighting conditions, measurement attributes of data points, and the characteristics of the imaging sensor to which the data points belong, the confidence weight of each data point is dynamically calculated, resulting in a point cloud with point-level confidence weights.

[0032] S140. For the point cloud set with point-level confidence weights, perform adaptive weighted fusion processing based on its spatial distribution to generate a preliminary fused point cloud model.

[0033] We analyze the spatial distribution of point clouds with point-level confidence weights. Point cloud density and distribution characteristics may differ across regions; for example, point cloud density may vary between the top and bottom of plant canopies. Analyzing spatial distribution allows us to understand the characteristics and reliability of point clouds in different regions.

[0034] Based on the spatial distribution of point clouds, an adaptive weighted fusion method is used to process point cloud sets with point-level confidence weights. In regions with high point cloud density, uniform distribution, and large weights, these data points are given greater fusion weights; while in regions with low point cloud density, sparse distribution, or small weights, the fusion weights are appropriately reduced. In this way, point cloud data from different imaging sensors are fused to generate a preliminary fused point cloud model. This model, to a certain extent, comprehensively considers the reliability and spatial distribution characteristics of different data points, improving the accuracy of the fusion results.

[0035] S150. Based on the quality indicators of the preliminary fused point cloud model and the real-time ambient lighting conditions, dynamically adjust the weight parameters for calculating the confidence weights. After iterative optimization, finally output the optimized 3D point cloud model.

[0036] The fusion effect is evaluated based on the quality metrics of the initial fused point cloud model. These metrics can include point cloud integrity, uniformity, and accuracy. Integrity can be assessed by comparing the coverage of the fused point cloud with that of the actual plant model; uniformity can be determined by analyzing the spatial distribution density of the point cloud; and accuracy can be measured by comparing it with high-precision reference data.

[0037] Based on the quality indicators of the initial fused point cloud model and the real-time ambient lighting conditions, the weight parameters for calculating the confidence weights are dynamically adjusted. If the quality of the initial fused point cloud model is not ideal and the ambient lighting conditions change, it is necessary to re-analyze the impact of ambient lighting conditions on imaging sensor measurements and data fusion, and adjust the weight parameters to ensure that data collected under different lighting conditions can be fused more reasonably.

[0038] By repeatedly performing steps S130-S150—namely, recalculating the confidence weights based on the new weight parameters, performing adaptive weighted fusion processing, evaluating the fusion quality, and adjusting the weight parameters—iterative optimization is achieved. After multiple iterations, the quality of the fusion result gradually improves, ultimately outputting an optimized 3D point cloud model. This model exhibits higher stability and accuracy, and can more accurately reflect the 3D phenotypic characteristics of plants.

[0039] The present invention provides a three-dimensional imaging method for plant phenotyping platforms based on multi-sensor fusion, which effectively solves the shortcomings of existing technologies in terms of stability and accuracy in complex and variable field environments. Specifically, by dynamically calculating the confidence weight of each data point based on ambient lighting conditions, the measurement attributes of each data point in the registered point cloud, and the characteristics of the imaging sensor to which the data point belongs, a point cloud with point-level confidence weights is obtained. This enables real-time perception of changes in the external environment (such as lighting conditions) and the characteristics of different sensors, and assigns appropriate weights to each data point based on these factors, rather than using fixed weights, thereby enhancing the perception and feedback capabilities of the real-time working status of sensors and changes in the external environment. When dynamically calculating the confidence weight of each data point, ambient lighting conditions are taken into account as an important factor. This allows for adaptive adjustment of the contribution of data collected by each sensor according to changes in ambient light intensity, so that data from each sensor can be rationally utilized under different lighting conditions, improving the accuracy of the acquisition results. The point cloud with point-level confidence weights is then subjected to adaptive weighted fusion processing based on its spatial distribution to generate a preliminary fused point cloud model. This step considers the spatial distribution of point clouds during the fusion process and uses an adaptive weighting method to fuse the point clouds, initially improving their quality. Based on the quality indicators of the initial fused point cloud model and real-time ambient lighting conditions, the weight parameters for calculating the confidence weights are dynamically adjusted. After iterative optimization, the final optimized 3D point cloud model is output. This method can continuously evaluate the quality of the initial fused point cloud model during the fusion process and dynamically adjust the weight parameters in conjunction with real-time ambient lighting conditions. Through iterative optimization, it further improves the point cloud quality, thus solving the problem that existing technologies cannot dynamically evaluate and optimize point cloud quality during the fusion process, and improving the stability and accuracy of the fusion results in complex and variable field environments.

[0040] In an optional embodiment, the time synchronization process in S120 above is to assign a unified timestamp to the multi-source asynchronous original point cloud through hardware pulse signals and a unified timestamp server, thereby obtaining a time synchronization point cloud.

[0041] On the plant phenotyping platform, a hardware pulse signal generator is configured for the various heterogeneous imaging sensors (such as lidar and time-of-flight depth cameras). This generator produces pulse signals (e.g., pulses per second) at a precise and fixed frequency, which are simultaneously sent to each imaging sensor. When an imaging sensor receives this pulse signal, it uses it as a reference point for the start time of data acquisition.

[0042] A unified timestamp server is set up, communicating with each imaging sensor via a network connection. When an imaging sensor begins acquiring data based on a hardware pulse signal, it simultaneously sends a request to the unified timestamp server. Upon receiving the request, the unified timestamp server records the current timestamp and returns it to the corresponding imaging sensor. The imaging sensor then associates and stores this unified timestamp with the raw point cloud data acquired in this session. In this way, each set of data in the multi-source asynchronous raw point cloud set is assigned a unified timestamp, forming a time-synchronized point cloud set.

[0043] Different imaging sensors, due to their varying operating mechanisms and startup times, acquire raw point cloud data at asynchronous times. This invention, through hardware pulse signals and a unified timestamp server, precisely aligns the data acquired by each imaging sensor onto a unified timeline, eliminating temporal discrepancies. This ensures accurate identification of data acquired by different imaging sensors at the same time when analyzing plant growth dynamics and movement states, avoiding data misalignment and misunderstandings caused by time asynchrony. The unified timestamp guarantees consistency of multi-source data across the time dimension. When constructing 3D plant models or performing other analyses, it ensures that data acquired by different imaging sensors correspond to the plant state at the same moment, thereby improving data reliability and usability and laying a solid foundation for subsequent spatial registration and fusion processing.

[0044] The spatial registration process described in S120 above is based on a pre-calibrated sensor extrinsic parameter matrix. The time synchronization point cloud is transformed to the same platform coordinate system, and then the online point cloud registration algorithm is used for registration to obtain the registration point cloud in the unified spatiotemporal coordinate system.

[0045] When installing each imaging sensor onto the plant phenotyping platform, specialized calibration tools and methods are used to precisely calibrate the sensor's extrinsic parameters. The extrinsic parameter matrix describes the rotational and translational relationship between the sensor coordinate system and the platform coordinate system. For example, by setting up a calibration object with a known position and orientation on the platform, data from the calibration object is collected using the imaging sensor. Combined with the actual geometric information of the calibration object, the sensor extrinsic parameter matrix is ​​calculated using mathematical algorithms (such as the least squares method). After obtaining the time-synchronized point cloud set, the point cloud data collected by each imaging sensor is transformed from its own sensor coordinate system to a unified platform coordinate system based on the pre-calibrated extrinsic parameter matrix corresponding to each imaging sensor. Assuming the lidar's sensor coordinate system is L, the platform coordinate system is P, and its extrinsic parameter matrix is ​​T... L-P For the coordinates of a point pL acquired by the lidar in the sensor coordinate system, the transformation formula p P =T L-P ×p LThis allows us to obtain the coordinates p of the point in the platform coordinate system. P .

[0046] Due to factors such as calibration errors and sensor measurement errors, even after transforming point cloud data to the same platform coordinate system through extrinsic parameter matrix transformation, point cloud data from different sensors may still not be perfectly aligned. Therefore, this invention uses an online point cloud registration algorithm to further optimize the registration effect. The online point cloud registration algorithm can adopt the ICP algorithm. Its basic idea is to find the nearest point in the other point set for each point in one point set, forming a set of corresponding point pairs; then, a rotation matrix and translation vector are calculated based on these corresponding point pairs to minimize the error between the two point sets; the calculated transformation is applied to one of the point sets, and the above process is repeated until the convergence condition is met. Through the online point cloud registration algorithm, the position and orientation of the point cloud data can be further fine-tuned to achieve more accurate alignment in space, ultimately obtaining a registered point cloud set in a unified spatiotemporal coordinate system.

[0047] This invention assigns a unified timestamp to all sensor data via hardware pulse signals and a unified timestamp server, achieving millisecond-level time alignment. This ensures that data from different sensors are snapshots of the scene at the same moment, providing a temporal consistency foundation for subsequent fusion and is key to reducing dynamic registration errors. Furthermore, based on a pre-calibrated sensor extrinsic parameter matrix, this invention initially transforms each point cloud to the same platform coordinate system; then, an online point cloud registration algorithm is used for fine-tuning, adjusting for minor misalignments caused by calibration errors and motion. Precise spatiotemporal synchronization and registration significantly reduce registration errors during multi-source data fusion, directly improving the measurement accuracy of phenotypic parameters such as plant height and crown width extracted from the fused point cloud. Finally, a registered point cloud set in a unified spatiotemporal coordinate system is obtained, with all point data located in the same reference system and time-aligned.

[0048] In an optional embodiment, the measurement attributes described in S130 above include measurement distance and reflectivity; the characteristics of the sensor include sensor type and reference weight parameters; and the ambient lighting conditions are obtained by real-time monitoring by an illuminance sensor.

[0049] For each data point, the measured distance is the straight-line distance between that data point and the imaging sensor that acquired it. This value is directly measured and output by the imaging sensor. For example, lidar calculates the time-of-flight of the laser beam to obtain the measured distance, and time-of-flight depth cameras calculate the phase difference of the light pulses to obtain the measured distance. These sensors record the measured distance information for each point as one of their measurement attributes while acquiring point cloud data. The measured distance is a key factor in evaluating the accuracy of sensor measurements. All imaging sensors have an optimal ranging range. Too close a distance may exceed the minimum range or increase errors due to optical distortion; too far a distance will cause signal attenuation, leading to a decrease in the signal-to-noise ratio and increased errors. Therefore, the confidence weight calculation algorithm attenuates the confidence of a data point based on its measured distance (e.g., using an exponential decay function). The further the distance deviates from the sensor's optimal range, the lower its weight. d represents the measurement distance. This is the distance attenuation coefficient.

[0050] Reflectivity is the ratio of the intensity of the echo signal received by an imaging sensor to the intensity of the transmitted signal. It reflects the reflective properties of the target surface material. Different plant parts, such as leaves, stems, and flowers, have different reflectivities due to their varying surface characteristics. The reflectivity is stored in conjunction with the measurement distance to form complete measurement attribute information.

[0051] Sensor type is prior knowledge, set during system initialization. It explicitly identifies which type of imaging sensor a given data point originates from (e.g., LiDAR or Time-of-Flight Depth Camera). The sensor type is used to select the corresponding confidence weight calculation formula. For example, identifying a data point as originating from LiDAR calls a confidence weight calculation formula based on measurement distance and reflectivity; identifying it as originating from a Time-of-Flight Depth Camera calls a confidence weight calculation formula based on saturation and ambient light intensity.

[0052] The baseline weight parameter is a configurable constant set during system design. It is an initial confidence value assigned based on performance evaluations of various imaging sensors under typical ideal conditions. For example, a high-precision lidar might be assigned a baseline weight of 0.9, while a time-of-flight depth camera, whose performance fluctuates significantly outdoors, might be assigned a baseline weight of 0.7.

[0053] Ambient lighting conditions are obtained in real time by an independent illuminance sensor. The illuminance sensor is mounted above the plant phenotyping platform and continuously measures ambient light intensity L(t) in lux. This data is provided in real time as an independent global variable to the confidence weight calculation formula. Ambient light intensity is a core dynamic factor adjusting the weights of optical sensors (especially time-of-flight depth cameras). The accuracy of time-of-flight depth cameras is highly susceptible to interference from strong ambient light, leading to depth calculation errors. Therefore, in the confidence weight calculation formula, ambient light intensity is treated as an attenuation term (e.g., using...). ), This is the light attenuation adjustment coefficient. is a natural constant. When an increase in ambient light intensity L(t) is detected, the weights of all data from the time-of-flight depth camera are automatically and significantly reduced, thereby minimizing the negative impact of unreliable data on the fusion results.

[0054] This invention corrects the measurement distance by considering reflectivity in the measurement attributes and combining it with sensor characteristics. Different reflectivities of plant surfaces can affect the measurement signal differently; for example, a highly reflective surface may cause excessive reflection intensity, leading to sensor measurement errors. By analyzing reflectivity data, the measurement distance can be corrected, thereby improving the accuracy of distance measurement. Furthermore, based on sensor type and reference weight parameters, during multi-sensor data fusion, the measurement distance data from different sensors can be utilized more rationally, further reducing errors and improving overall measurement accuracy.

[0055] Real-time monitoring of ambient light intensity provides crucial reference information for reflectance measurements. Ambient light intensity and spectral distribution affect the reflectance characteristics of plant surfaces, thus influencing the reflectance measurement results. Ambient light intensity acquired through illuminance sensors allows for illumination compensation and correction of reflectance measurements. For example, under strong light conditions, reflected light from the plant surface may be interfered with by ambient light, leading to an overestimation of reflectance. Correcting reflectance measurements using appropriate algorithms based on ambient light intensity enhances the reliability of reflectance measurements, enabling them to more accurately reflect the true characteristics of the plant surface.

[0056] In an optional embodiment, the present invention optimizes the deployment of multiple heterogeneous imaging sensors on a plant phenotyping platform to address the issues of sparse point clouds and blind spots inherent in single sensors. Specifically, the multiple heterogeneous imaging sensors include a first-type lidar, a second-type lidar, and a time-of-flight depth camera. The number of each type of lidar can be multiple, and the specific configuration can be determined according to actual needs.

[0057] The first type of lidar is a short-range, high-precision lidar with a short measurement range (0-10 meters) and high accuracy. This invention mounts it below and at an angle to the plant phenotyping platform. This layout fully utilizes its small blind zone, specifically designed to capture the fine structures of the plant base, the soil surface above the roots, and the lower canopy, which are difficult to scan with traditional platforms. This effectively compensates for the near-field blind zone limitations of mid-to-long-range sensors. The first type of lidar provides high-precision, moderate-density point clouds at medium to short ranges, serving as a primary source of phenotypic structures in the lower parts of plants.

[0058] The second type of lidar is a medium-to-long-range multi-line mechanical lidar with a long range (10-50 meters) and a large number of lines. This invention installs it at a high position around the plant phenotyping platform to form a multi-angle scan, covering the top and middle of the crop canopy and the overall structure of the distant field ridges. This layout aims to cover long-distance and top-of-crop views to obtain macroscopic morphological and spatial distribution information of the crop, but its point cloud is relatively sparse and has significant blind spots at close range.

[0059] The aforementioned first and second type of lidar can be selected from existing technologies that can meet the above range requirements; no specific limitations are made here.

[0060] The time-of-flight depth camera is based on the time-of-flight method and is installed directly below the plant phenotyping platform. It provides ultra-high density point clouds and can capture fine details such as leaf texture and edges.

[0061] Through the coordinated deployment and data complementarity of multiple lidar and time-of-flight depth cameras, this invention ensures comprehensive data acquisition from the outset, laying the hardware foundation for more complete and dense 3D plant structure acquisition. Each imaging sensor operates independently, outputting a multi-source asynchronous raw point cloud that is asynchronous in both time and space.

[0062] The dynamic calculation of the confidence weight for each data point as described in S130 above includes: S1301. For data points from the first type of lidar and the second type of lidar, the confidence weight is determined at least based on the reference weight coefficient of the lidar to which it belongs, the measurement distance of the data point, the reflectivity of the data point, a distance attenuation coefficient, and a fusion closed-loop dynamic adjustment coefficient.

[0063] For the i-th data point P from the j-th lidar i The confidence weight calculation formula is: in, Let P be the i-th data point of the j-th lidar. i The confidence weight is β, where β is the baseline weight coefficient for the lidar. jγ represents the baseline weighting coefficient for the j-th lidar. For example, short-range high-precision lidars, due to their stability, may be assigned a higher baseline weighting coefficient than medium- and long-range multi-line mechanical lidars. j Let d be the range attenuation coefficient of the j-th lidar. i The measured distance for the i-th data point is determined by the distance attenuation coefficient. To control its impact, an exponential decay function (such as...) is used in the calculation. For a data point from a short-range, high-precision radar, the closer the measured distance is to the 10-meter upper limit, the greater the weight attenuation. For a data point from a medium- to long-range, multi-line mechanical radar, the closer the measured distance is to 0 meters (close to its blind zone) or 50 meters (close to its limit), the greater the weight attenuation. norm,i Let be the normalized reflectivity of the i-th data point. C is the fusion closed-loop dynamic adjustment coefficient of the lidar. j(t) The fusion closed-loop dynamic adjustment coefficient of the j-th lidar at time t is used to fine-tune the overall reliability of this type of lidar in order to cope with slowly changing environmental factors such as rain, fog, and dust.

[0064] S1302. For data points from the time-of-flight depth camera, the confidence weight is determined at least based on the baseline weight coefficient of the time-of-flight depth camera, the pixel saturation of the data point, the reflectivity of the data point, the real-time ambient light intensity, and the illumination attenuation adjustment coefficient.

[0065] For the i-th data point P from the time-of-flight depth camera i The confidence weight calculation formula is: in, Let P be the i-th data point from the time-of-flight depth camera. i Confidence weights S is the baseline weighting coefficient, representing the basic reliability of the time-of-flight depth camera under ideal low-light conditions. i S represents the pixel saturation of the i-th data point. i The higher the value, the more likely the data point is to be invalid due to overexposure. The calculation uses (1... S i The higher the saturation, the closer this value is to 0, and the lower its weight is. i Let be the reflectance of the i-th data point. This represents the light attenuation adjustment coefficient. L(t) is the real-time ambient light intensity at time t, obtained in real-time by an illuminance sensor. The light attenuation adjustment coefficient... To control its impact, an exponential decay function is also used. The stronger the ambient light, the closer this value is to 0, resulting in a significant decrease in the weight of the entire data point. It can be dynamically adjusted to precisely control the intensity of ambient light effects.

[0066] In low-light environments such as early morning and evening, the L(t) value is small, and the confidence weight of the time-of-flight depth camera is low. The increased density of the radar point cloud allows it to fully compensate for the sparseness of the radar point cloud and capture rich details. Under strong midday sunlight, the L(t) value is large. The data is automatically suppressed to an extremely low level, relying almost entirely on the fusion of data from the two types of radars. This effectively avoids the fatal shortcoming of unreliable data from time-of-flight depth cameras under strong light, ensuring the stability of the output.

[0067] The fusion method of this invention is no longer a simple data stacking or averaging, but a data screening based on reliability. Data points with high confidence weights (i.e., high-quality data) dominate the fusion result, while the influence of data points with low confidence weights (i.e., low-quality data) is minimized. This directly results in a final 3D point cloud model with higher accuracy, less noise, and more realistic geometry. The confidence weights are no longer fixed values, but dynamically change with variables such as ambient light intensity L(t) and measurement distance d, providing robustness and all-weather operation capabilities.

[0068] In an optional embodiment, the above-described S140 process of adaptively weighting and fusing the point cloud set with point-level confidence weights according to its spatial distribution to generate a preliminary fused point cloud model includes: S1401. For multiple points in the point cloud set with point-level confidence weights whose spatial distance is less than a preset threshold, a weighted average fusion is performed based on their confidence weights to generate a fusion point; points in non-overlapping areas are retained.

[0069] The entire point cloud is traversed, and judgments are made based on spatial distance. For each point, all other points within its 3D spatial range that are less than a preset threshold ε are found. These points, which physically represent the same (or very close) spatial location, constitute a point cluster. The preset threshold ε is set according to the point cloud density and sensor accuracy, for example, from a few millimeters to a few centimeters.

[0070] For each cluster of points found, instead of a simple arithmetic average, a weighted average is calculated. The specific calculation is as follows: Assuming a cluster contains n data points, the i-th data point... The corresponding confidence weight is w i The fusion point P of this cluster of points fuse The calculation formula is: Confidence weight w iThe higher the confidence level of a point (i.e., the more reliable it is), the greater its impact on the final fusion point location. Points with very low confidence weights (i.e., unreliable points, noise points) have their influence on the final result significantly suppressed. If a point has a weight of 0, it has no effect on the fusion result at all. After the above calculations, each cluster of points is merged into a unique new fusion point P. fuse The merged point not only includes the merged coordinates, but also other attributes such as the merged reflectance, as well as its own new weights. These new weights can be the maximum, average, or sum of the confidence weights of all points in the point cluster.

[0071] For data points that have no neighbors in space (i.e., their distance from all other points is greater than a preset threshold ε), they are determined to be located in non-overlapping regions. These data points are areas covered by only a single sensor, such as sparsely cropped areas or blind spots of a particular imaging sensor's exclusive field of view. These data points are retained without any fusion calculations; retaining them ensures that the fused point cloud model does not lose important details.

[0072] S1402. Filter and denoise all fused points and retained points to obtain the preliminary fused point cloud model.

[0073] After the aforementioned fusion and retention operations, the resulting point cloud set may still contain some outlier noise points (e.g., points generated by occasional random errors that are not eliminated by weighted averaging because they have no other points around them) or minor non-uniformities caused by the fusion process. The point cloud set, composed of all fused points and retained points, is then filtered for noise reduction. Filtering algorithms can include statistical filtering, radius filtering, etc. Statistical filtering algorithms determine whether a point is a noise point based on the statistical characteristics (such as mean distance, standard deviation, etc.) of its surrounding neighborhood points. Radius filtering algorithms, on the other hand, set a radius and count the number of neighboring points within that radius for each point; if the number of neighboring points is less than a certain threshold, the point is considered a noise point.

[0074] The point cloud is filtered according to the selected filtering algorithm. For example, when using a statistical filtering algorithm, the average distance from each point's neighboring points to that point is first calculated. Then, assuming these average distances follow a Gaussian distribution, a distance threshold is set, and points with an average distance exceeding this threshold are identified as noise points and removed. This filtering and denoising process removes isolated noise points and some outliers from the point cloud, further improving the quality of the point cloud model. After these steps, the resulting point cloud is the preliminary fused point cloud model. This model integrates point cloud data from multiple imaging sensors. Through adaptive weighted fusion and filtering and denoising, it reduces noise and redundant information while retaining important features, providing a more accurate data foundation for subsequent analysis and processing (such as plant morphology reconstruction and physiological parameter extraction).

[0075] Point cloud data collected by different sensors may contain measurement errors. This invention employs adaptive weighted fusion processing, fusing data based on the confidence weight of each point. This ensures that information from more reliable points carries greater weight during the fusion process, thereby reducing the impact of measurement errors on the final point cloud model. In plant phenotypic analysis, point cloud data collected by LiDAR and depth cameras may exhibit biases due to differences in sensor characteristics. Weighted fusion can effectively correct these biases, improving the accuracy of the point cloud data. Point cloud data from different sensors may differ in spatial distribution and feature representation. Adaptive weighted fusion can coordinate and unify these differences, generating a consistent point cloud model. This is crucial for subsequent analysis and processing. For example, in reconstructing 3D plant models, consistent point cloud data ensures a more accurate and realistic reconstructed model. For overlapping points with a spatial distance less than a preset threshold, weighted average fusion can reduce redundant information while preserving details. For instance, in the canopy region of a plant, point cloud data from multiple sensors may overlap. Fine fusion can generate a denser and more accurate point cloud, better reflecting the structural characteristics of the canopy. By preserving points in non-overlapping regions, the fused point cloud model retains the unique features and boundary information of the original point cloud. In plant phenotypic analysis, these features may include leaf edges, stem bifurcation points, etc., thus accurately describing the morphology and structure of the plant.

[0076] Filtering and denoising effectively removes isolated noise points and outliers from point clouds. These noise points and outliers may be caused by sensor errors, environmental interference, or other factors. Removing them significantly improves the quality of point cloud data, making the point cloud model cleaner and more accurate. Clean point cloud data reduces errors and uncertainties in subsequent analysis and processing, enhancing the reliability of the model. For example, when extracting plant physiological parameters, accurate point cloud data ensures that the extracted parameters are more reliable, providing more valuable information for plant growth monitoring and variety evaluation.

[0077] In an optional embodiment, the quality indicators in S150 above include point cloud noise level, point cloud density, and registration error; the weighting parameters for dynamically adjusting the confidence weights include: S1501, Adjust the reference weight coefficients (α and β above) and the distance attenuation coefficient. The fusion closed-loop dynamic adjustment coefficient C and the illumination attenuation adjustment coefficient One or more of them.

[0078] The preliminary fused point cloud model is analyzed in real time, and three quality indicators are calculated: point cloud noise level, point cloud density, and registration error.

[0079] Point cloud noise level represents the proportion of outliers or interfering points in the data. A high point cloud noise level indicates the presence of many unreliable points. In this case, to reduce the impact of noise on the final result, relevant weighting parameters need to be adjusted. For example, the presence of noise reduces the overall reliability of the data, so the baseline weighting coefficient will be appropriately reduced; the distance attenuation coefficient may be adjusted based on the distribution of noise at different distances. If noise is more pronounced in distant areas, the distance attenuation coefficient in distant areas will be increased to reduce the impact of distant noise points; the fusion closed-loop dynamic adjustment coefficient will also respond to changes in noise level, quickly adjusting the confidence weight to suppress the effect of noise points; the illumination attenuation adjustment coefficient will be adjusted when there is noise caused by illumination. If strong light generates more noise, the illumination attenuation adjustment coefficient will be increased, and the weight of points affected by strong light will be reduced.

[0080] Point cloud density represents the spatial density of data distribution. In areas with low point cloud density, the reliability of the data is relatively poor. The baseline weight coefficient is re-evaluated based on the density, with lower baseline weights assigned to low-density areas. The distance attenuation coefficient is adjusted based on the relationship between point cloud density and measurement distance; for example, the distance attenuation coefficient is increased in low-density and distant areas. The fusion closed-loop dynamic adjustment coefficient dynamically adjusts the weights based on real-time changes in point cloud density to ensure reasonable calculation of confidence levels under different density conditions. The illumination attenuation adjustment coefficient plays a role when point cloud density is affected by illumination; if uneven illumination causes abnormal density in some areas, this coefficient is adjusted to balance the influence of illumination.

[0081] Registration error represents the deviation of point cloud data collected by different sensors during the registration process. A large registration error indicates inaccurate spatial alignment of the data. The reference weight coefficient decreases due to registration error, as inaccurate registration reduces the overall reliability of the data. The distance attenuation coefficient is adjusted based on the relationship between registration error and measurement distance; if the registration error is larger at long distances, the distance attenuation coefficient is increased. The fusion closed-loop dynamic adjustment coefficient quickly responds to changes in registration error and adjusts the weights accordingly. The illumination attenuation adjustment coefficient is adjusted when the registration error is affected by illumination; for example, if changes in illumination make registration difficult, this coefficient is adjusted to optimize weight calculation.

[0082] The baseline weighting coefficients are adjusted as follows: based on a comprehensive evaluation of point cloud noise levels, point cloud density, and registration errors, the baseline weighting coefficients for different imaging sensors or data sources are redefined. If the data from a particular imaging sensor performs well in terms of noise, density, and registration, its baseline weighting coefficient may be increased; conversely, it may be decreased.

[0083] The distance attenuation coefficient is adjusted as follows: The noise, density, and registration of the point cloud at different measurement distances are analyzed to establish a model relating the measurement distance to these quality indicators. Based on the model results, the distance attenuation coefficient is adjusted within different distance ranges. For example, in distance regions with high noise, low density, or large registration errors, the distance attenuation coefficient is increased.

[0084] The adjustment method for the fusion closed-loop dynamic adjustment coefficient is as follows: Based on real-time monitoring of quality indicator changes, the aforementioned fusion closed-loop dynamic adjustment coefficient is adjusted using an adaptive algorithm. When a quality indicator changes rapidly, the fusion closed-loop dynamic adjustment coefficient will increase or decrease accordingly to quickly adjust the confidence weight.

[0085] The illumination attenuation adjustment coefficient is adjusted as follows: Ambient light intensity is acquired in real-time using an illuminance sensor, and the performance of point cloud noise, density, and registration error under varying illumination conditions is considered to adjust the illumination attenuation adjustment coefficient. When changes in ambient light intensity lead to a decrease in data quality, the illumination attenuation adjustment coefficient is increased, and the weight of points affected by illumination is reduced.

[0086] After adjusting the weight parameters, recalculate the confidence weight for each data point according to the corresponding confidence weight calculation formula (i.e., the weight calculation formula for LiDAR and time-of-flight depth camera mentioned above). This ensures that the new confidence weight accurately reflects the reliability of the data point under the current quality indicators.

[0087] This invention dynamically adjusts weight parameters based on quality indicators, responding in real-time to changes in point cloud noise levels, point cloud density, and registration errors. This dynamic adjustment allows for a more rational allocation of weights among different data points during the fusion process. High-quality, reliable data points play a greater role in the fusion, while the impact of low-quality data points is reduced, significantly improving the accuracy of point cloud data fusion and generating a more accurate fused point cloud model. Accurate confidence weight calculation provides a more reliable data foundation for subsequent applications such as plant phenotypic analysis. In plant morphology reconstruction and physiological parameter extraction analyses, the optimized weight calculation results yield more accurate analytical results, providing more valuable references for plant growth monitoring and variety evaluation.

[0088] The following is a dynamic adjustment algorithm based on quality index weighting, used to adjust the baseline weight coefficient, distance attenuation coefficient, fusion closed-loop dynamic adjustment coefficient, and illumination attenuation adjustment coefficient. The algorithm includes the following steps: 1. Quantification of quality indicators: Statistical methods are used to calculate the average distance between each point in the point cloud and its neighboring points, and the proportion of noise points is determined by setting a threshold. For example, points whose average distance exceeds twice the standard deviation of the global average distance are identified as noise points, and the ratio of the number of noise points to the total number of points is used as the point cloud noise level index N, with a value range of [0,1].

[0089] The point cloud space is divided into regular grids, the number of points in each grid is counted, and the average point cloud density D is calculated, with the unit being points per cubic meter. Simultaneously, the uniformity of the density distribution is analyzed; the point cloud density uniformity index U is measured by calculating the standard deviation of each grid density from the average density.

[0090] For the registered point cloud, feature points are extracted and matched, and the distance deviation between corresponding feature points is calculated. The average distance deviation of all feature point pairs is used as the registration error index R, in meters.

[0091] 2. Weight parameter adjustment rules setting: The adjustment rule for the baseline weighting coefficient α of the time-of-flight depth camera is as follows: Set the baseline weighting coefficient adjustment factor f. α , calculate its value based on the quality indicators. When N <N good (N) good (This is a preset threshold representing a low noise level, such as 0.2), D>D good (D) good (where R is a preset density threshold, such as 100) <R good (R) good When the preset first registration error threshold is 0.05, f α =α inc (α)inc The preset first increase factor (e.g., 1.2) indicates good data quality and increases the baseline weight factor; when N>N bad (N) bad (For a preset high noise level threshold, such as 0.5) or R>R bad (R) bad When the preset second registration error threshold is 0.1, f α =α dec (α) dec The preset first reduction coefficient (e.g., 0.8) indicates poor data quality, thus reducing the baseline weight coefficient; otherwise, f α =1, keeping the baseline weighting coefficient unchanged. New baseline weighting coefficient α for the time-of-flight depth camera. new =α old ×f α α old This represents the baseline weighting coefficient for the time-of-flight depth camera before adjustment.

[0092] The adjustment rule for the baseline weighting coefficient β of the lidar is as follows: Similarly, when N <N good , D>D good And R <R good At that time, the triggering adjustment factor, namely α mentioned above, is activated. inc ,β new =β old ×α inc N>N bad Or R>R bad At that time, the trigger inhibition adjustment factor, namely α mentioned above, is activated. dec ,β new =β old ×α dec If the point cloud density uniformity index U > U thr (U) thr (Based on a set density uniformity threshold), areas with excessively low density are adjusted according to a preset compensation formula. The compensation formula is as follows: β new =β old × Where, β new β is the new benchmark weighting coefficient for lidar. old This represents the baseline weighting coefficient of the lidar before adjustment. (D) target For the target point cloud density, D actual This represents the actual point cloud density.

[0093] The distance attenuation coefficient γ is adjusted according to the distribution of point cloud density and registration error at different distances. The measurement distance is divided into three regions: near, medium, and far. For each region, the point cloud density D within that region is calculated.region And registration error R region Set the distance attenuation coefficient adjustment factor f. γ If in a certain region D region <D low (D) low (e.g., 50) and R region >R high (R) high If the preset high registration error threshold is 0.08, then f γ =γ inc (γ) inc (As a preset second increase factor, such as 1.5), this increases the distance attenuation factor in this region; if D region >D high (D) high (e.g., 150) and R region <R low (R) low If the preset low registration error threshold is 0.03, then f γ =γ dec (γ) dec The second reduction factor is set to 0.8 (preset), which reduces the distance attenuation factor; otherwise, f γ =1. New distance attenuation coefficient γ new =γ old ×f γ γ old This is the distance attenuation coefficient before adjustment.

[0094] The adjustment rule for the fusion closed-loop dynamic adjustment coefficient is as follows: Define the rate of change of quality indicators, such as the rate of change of point cloud noise level ΔN, the rate of change of point cloud density ΔD, and the rate of change of registration error ΔR. Set the adjustment factor f for the fusion closed-loop dynamic adjustment coefficient. cd When |ΔN|>ΔN thr (ΔN) thr This is a preset threshold for the rate of change of noise level, such as 0.1) or |ΔD|>ΔD thr (ΔD) thr (e.g., 20) or |ΔR|>ΔR thr (ΔR) thr When the preset registration error rate of change threshold is 0.02, f cd =1.3, increase the dynamic adjustment coefficient of the fusion closed loop to respond quickly to changes; otherwise f cd =1. New fusion closed-loop dynamic adjustment coefficient C new =C old ×f cd C old This represents the dynamic adjustment coefficient of the fusion closed loop before adjustment.

[0095] The adjustment rule for the illumination attenuation coefficient is as follows: Ambient light intensity L is obtained through an illuminance sensor, and the impact of changes in ambient light intensity on point cloud quality is analyzed. An adjustment factor f is set for the illumination attenuation coefficient. λ When changes in ambient light intensity cause the point cloud noise level N to increase by more than ΔN light (ΔN) light The preset threshold for noise increase due to illumination is 0.1) or the point cloud density D changes by more than (ΔD). light This is a preset threshold for density change due to illumination (e.g., 20%) or an increase in registration error R exceeding ΔR. light (ΔR) light When a threshold is added to the preset registration error caused by illumination (e.g., 0.02), f λ =λ inc (λ) inc (For the preset adjustment coefficient, such as 1.2), adjust the light attenuation adjustment coefficient; otherwise, f λ =1. New illumination attenuation adjustment coefficient λ new =λ old ×f λ . λ old This is the light attenuation adjustment coefficient before adjustment.

[0096] 3. Weight parameter update and confidence weight recalculation: After obtaining the new weight parameters according to the above adjustment rules, the confidence weight of each data point is recalculated according to the corresponding confidence weight calculation formula (i.e., the confidence weight calculation formula of the above lidar or time-of-flight depth camera).

[0097] This invention introduces an ambient light sensing closed-loop adjustment mechanism in S130 and S150, which monitors L(t) and dynamically adjusts it using an illuminance sensor. These parameters effectively alleviate the saturation and frame drop issues of the time-of-flight depth camera under strong light conditions, allowing it to leverage its high-density advantage in low light while intelligently suppressing its negative effects in strong light, thus ensuring the system's stable imaging capability in all weather conditions. Simultaneously, by adjusting the weight parameters of each sensor in real time (…),… ,β, , (C) In response to changes in environment and equipment status, the system becomes an adaptive organism rather than a fixed-parameter algorithm. This dynamic feedback mechanism significantly improves point cloud quality and the robustness of the entire system. In summary, this invention solves the spatial blind zone problem through optimized layout of heterogeneous imaging sensors, solves the data consistency problem through precise spatiotemporal registration, and solves the environmental adaptability problem in the temporal dimension through ambient light perception and dynamic weight adjustment. Ultimately, it successfully achieves blind-zone-free, high-precision, and highly robust 3D imaging of field plants, providing a reliable data acquisition solution for precision agriculture.

[0098] The following describes the three-dimensional imaging device for a plant phenotyping platform based on multi-sensor fusion provided by the present invention. The three-dimensional imaging device for a plant phenotyping platform based on multi-sensor fusion described below can be referred to in correspondence with the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion described above.

[0099] The three-dimensional imaging device for plant phenotyping platform based on multi-sensor fusion provided by this invention refers to... Figure 2 As shown, it includes: The data acquisition module 210 is used to acquire the original three-dimensional point cloud data of the plant through a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform, and obtain a multi-source asynchronous original point cloud set. The spatiotemporal alignment module 220 is used to perform time synchronization processing and spatial registration processing on the multi-source asynchronous original point cloud to obtain a registration point cloud under a unified spatiotemporal coordinate system. The weight calculation module 230 is used to dynamically calculate the confidence weight of each data point based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, so as to obtain a point cloud with point-level confidence weights. The point cloud fusion module 240 is used to perform adaptive weighted fusion processing on the point cloud set with point-level confidence weights according to its spatial distribution to generate a preliminary fused point cloud model. The closed-loop optimization module 250 is used to dynamically adjust the weight parameters for calculating the confidence weight based on the quality index of the preliminary fused point cloud model and the real-time ambient lighting conditions. After iterative optimization, the optimized 3D point cloud model is finally output.

[0100] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions from the memory 330 to execute a three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion.

[0101] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the three-dimensional imaging method for a plant phenotypic platform based on multi-sensor fusion provided by the above methods.

[0103] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion provided by the methods described above.

[0104] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional imaging method for plant phenotyping platforms based on multi-sensor fusion, characterized in that, include: The original three-dimensional point cloud data of the plant is collected by a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform to obtain a multi-source asynchronous original point cloud set. The multi-source asynchronous original point cloud is subjected to time synchronization processing and spatial registration processing to obtain a registration point cloud under a unified spatiotemporal coordinate system; Based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, the confidence weight of each data point is dynamically calculated to obtain a point cloud with point-level confidence weights. For the point cloud set with point-level confidence weights, an adaptive weighted fusion process is performed based on its spatial distribution to generate a preliminary fused point cloud model. Based on the quality indicators of the preliminary fused point cloud model and the real-time ambient lighting conditions, the weight parameters for calculating the confidence weights are dynamically adjusted. After iterative optimization, the optimized 3D point cloud model is finally output.

2. The three-dimensional imaging method for plant phenotyping platform based on multi-sensor fusion according to claim 1, characterized in that, The time synchronization process involves assigning a unified timestamp to the multi-source asynchronous original point cloud through hardware pulse signals and a unified timestamp server, thereby obtaining a time synchronization point cloud. The spatial registration process involves transforming the time synchronization point cloud to the same platform coordinate system based on a pre-calibrated sensor extrinsic parameter matrix, and then performing registration using an online point cloud registration algorithm to obtain the registration point cloud in the unified spatiotemporal coordinate system.

3. The three-dimensional imaging method for plant phenotyping platform based on multi-sensor fusion according to claim 1, characterized in that, The measured attributes include measurement distance and reflectivity; the sensor characteristics include sensor type and reference weight parameters; the ambient lighting conditions are obtained by real-time monitoring by an illuminance sensor.

4. The three-dimensional imaging method for plant phenotyping platform based on multi-sensor fusion according to claim 3, characterized in that, The various heterogeneous imaging sensors include a first type of lidar, a second type of lidar, and a time-of-flight depth camera; The dynamic calculation of the confidence weight for each data point includes: For data points from the first type of lidar and the second type of lidar, the confidence weight is determined at least based on the reference weight coefficient of the lidar to which it belongs, the measurement distance of the data point, the reflectivity of the data point, a distance attenuation coefficient, and a fusion closed-loop dynamic adjustment coefficient. For a data point from the time-of-flight depth camera, its confidence weight is determined based at least on the baseline weight coefficient of the time-of-flight depth camera, the pixel saturation of the data point, the reflectivity of the data point, the real-time ambient light intensity, and the illumination attenuation adjustment coefficient.

5. The three-dimensional imaging method for plant phenotyping platform based on multi-sensor fusion according to claim 4, characterized in that, The quality indicators include point cloud noise level, point cloud density, and registration error; the weighting parameters for dynamically adjusting the confidence weights include: Adjust one or more of the following: the baseline weighting coefficient, the distance attenuation coefficient, the fusion closed-loop dynamic adjustment coefficient, and the illumination attenuation adjustment coefficient.

6. The three-dimensional imaging method for plant phenotyping platform based on multi-sensor fusion according to claim 1, characterized in that, The point cloud set with point-level confidence weights is subjected to adaptive weighted fusion processing based on its spatial distribution to generate a preliminary fused point cloud model, including: For multiple points in the point cloud set with point-level confidence weights whose spatial distance is less than a preset threshold, a weighted average fusion is performed based on their confidence weights to generate a fused point. Points in non-overlapping regions are retained. The initial fused point cloud model is obtained by filtering and denoising all fused points and the retained points.

7. A three-dimensional imaging device for a plant phenotyping platform based on multi-sensor fusion, characterized in that, include: The data acquisition module is used to acquire the original three-dimensional point cloud data of the plant through a variety of heterogeneous imaging sensors mounted on the plant phenotyping platform, and obtain a multi-source asynchronous original point cloud set. The spatiotemporal alignment module is used to perform time synchronization and spatial registration processing on the multi-source asynchronous original point cloud to obtain a registration point cloud under a unified spatiotemporal coordinate system. The weight calculation module is used to dynamically calculate the confidence weight of each data point based on the ambient lighting conditions, the measurement attributes of each data point in the registration point cloud, and the characteristics of the imaging sensor to which the data point belongs, so as to obtain a point cloud with point-level confidence weights. The point cloud fusion module is used to perform adaptive weighted fusion processing on the point cloud set with point-level confidence weights according to its spatial distribution to generate a preliminary fused point cloud model. The closed-loop optimization module is used to dynamically adjust the weight parameters for calculating the confidence weight based on the quality indicators of the preliminary fused point cloud model and the real-time ambient lighting conditions. After iterative optimization, the optimized 3D point cloud model is finally output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional imaging method for a plant phenotyping platform based on multi-sensor fusion as described in any one of claims 1 to 6.

Images