Corn ear phenotype analysis method and system based on low-resolution dToF sensor

Abstract

The present application provides a corn ear phenotype analysis method and system based on a low-resolution dToF sensor. The method first acquires an RGB image and a depth image of a corn ear captured by a camera using a low-resolution dToF sensor, and performs target segmentation on the RGB image. Then, based on the target segmentation result and the depth image, a three-dimensional original point cloud of the ear is generated. Based on the three-dimensional original point cloud of the ear, the maximum longitudinal section of the ear geometry is determined. Then, the three-dimensional original point cloud of the ear is projected radially onto the plane where the maximum longitudinal section is located, to obtain a two-dimensional point cloud of the ear. Finally, based on the two-dimensional point cloud of the ear, the phenotype parameters of the ear are extracted. The present application greatly reduces the technical threshold and hardware cost of three-dimensional plant phenotype measurement while ensuring that the error of the phenotype parameters is within the acceptable range of agricultural production and scientific research, providing a feasible technical path for the popularization of low-cost portable measurement technology.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for analyzing the phenotypic characteristics of corn ears based on a low-resolution dToF sensor. Background Technology

[0002] Compared to LiDAR or professional depth cameras, the core advantage of image acquisition devices equipped with low-resolution dToF sensors (such as smartphones) lies in their ease of operation and portability, meeting the needs of rapid, non-contact field measurements. However, conventional consumer-grade TOF sensors have significant hardware performance shortcomings: their spatial resolution is very low (the iPhone 12 Pro Max has a resolution of only 49,152 pixels), and coupled with the characteristics of beam divergence, the instantaneous field of view (IFOV) of a single pixel covers a large physical area, easily causing severe spatial undersampling. This results in large areas of continuous depth signals being integrated into a single estimate, which is detrimental to the measurement of the undulating surface of the ear of fruit. More seriously, when the light spot simultaneously crosses the boundary between the ear of fruit and the background, the sensor integrates reflected photons from different distances, producing a mixed pixel phenomenon, manifested as incorrect intermediate depth values ​​and blurred edge transitions, ultimately appearing as a large number of abnormal offsets of edge points in the reconstructed point cloud.

[0003] Current mainstream solutions for processing low-quality depth data include directly using the original depth map for 3D calculations, and upsampling the depth map to the same size as the RGB image using nearest neighbor or bilinear interpolation before calculation. Spatial deviations caused by mixed pixels in the original depth map cannot be accurately removed by simple filtering to preserve the true shape of the point cloud. Directly calculating edge distances in 3D space introduces significant systematic errors. Furthermore, image-processing-based depth upsampling strategies, when faced with extreme spatial undersampling in mobile phone TOF (Time-of-Flight) sensors, not only fail to recover lost high-frequency edge details but also further blur boundary information. The point cloud generated from the original depth map is sparse and discrete. Nearest neighbor interpolation simply copies the depth values ​​of neighboring points, amplifying the original noise. While bilinear interpolation can fill in discrete sampling points into a continuous surface through neighborhood weighted averaging, it is essentially just a smoothing fill of low-resolution data and cannot eliminate the hardware-level errors inherent in the original depth measurement itself.

[0004] The above results indicate that the inherent undersampling problem caused by insufficient acquisition performance of the device itself cannot be fundamentally compensated for by traditional data augmentation methods. Summary of the Invention

[0005] The purpose of this invention is to address the spatial undersampling and mixed pixel error problems existing in the prior art by providing a method and system for corn ear phenotyping based on a low-resolution dToF sensor that can effectively solve the problem of ear size measurement.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor, comprising:

[0008] S1. Acquire RGB and depth images of corn ears captured by a camera using a low-resolution dToF sensor; perform target segmentation on the RGB images, where the segmentation target is the corn ear;

[0009] S2. Based on the target segmentation results and depth images, generate the original 3D point cloud of the ear of fruit.

[0010] S3. Determine the maximum longitudinal section of the ear's geometric shape based on the three-dimensional original point cloud of the ear;

[0011] S4. Project the original three-dimensional point cloud of the ear of fruit radially onto the plane containing the maximum longitudinal section to obtain the two-dimensional point cloud of the ear of fruit.

[0012] S5. Extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear, wherein the phenotypic parameters include ear length and ear width.

[0013] S3 includes:

[0014] S31. Perform point cloud filtering on the original three-dimensional point cloud of the ear of fruit to obtain the filtered three-dimensional point cloud of the ear of fruit, Q.

[0015] S32. Using the farthest point in point cloud Q as the base point and the camera plane angle as the reference, construct multiple planes parallel to the camera plane, with pitch and yaw angles within the range of (-A°, A°).

[0016] S33. Project the point cloud Q onto each of the constructed planes and calculate the corresponding projected area. Use the plane with the largest projected area as the largest longitudinal section in the geometric shape of the ear of fruit.

[0017] S5 includes: performing principal component analysis on the two-dimensional point cloud of the ear, extracting the maximum Euclidean distance in the direction of the first principal component as the ear length, and the maximum width perpendicular to the direction of the first principal component as the ear width.

[0018] The phenotypic parameters also include spikelet area;

[0019] The method for extracting the ear area includes: after performing principal component analysis on the two-dimensional point cloud of the ear, establishing a new coordinate system with the direction of the first principal component as the Y-axis and the perpendicular line of the direction of the first principal component in the plane of the two-dimensional ear point cloud as the X-axis, then dividing it into multiple slices along the Y-axis, and calculating the ear area A based on the following formula:

[0020] ;

[0021] In the above formula, , Let be the length and height of the i-th slice, respectively, and n be the number of slices.

[0022] S1 uses an improved YOLOv8 model to segment the RGB image;

[0023] The improved YOLOv8 model replaces the initial convolutional layer in the backbone network structure with a ConvNormLayer that integrates normalization operations, and adds an MSFBlock multi-scale fusion module to the shallow network of the backbone network.

[0024] The improved YOLOv8 model replaces the C2f module in the neck network structure with the RepC3 reparameterization module.

[0025] The S2 includes: determining the mask output by the identified target, extracting the depth coordinates corresponding to all points within the mask area from the depth image, and generating a three-dimensional original point cloud of the corn ear by combining the camera intrinsic parameters, wherein the mask area is the entire corn ear.

[0026] S4 includes: moving all points in the original three-dimensional point cloud of the ear of fruit along the line connecting it to the origin of the coordinate system to the plane where the maximum longitudinal section is located, and the resulting set of planar points is the two-dimensional point cloud of the ear of fruit.

[0027] The specific method of moving along the line connecting it to the origin is as follows: taking the camera optical center as the origin, each three-dimensional point in the original three-dimensional point cloud of the ear of fruit is translated along the radial line connecting the origin and the point to the plane where the maximum longitudinal section is located.

[0028] In a second aspect, the present invention provides a corn ear phenotypic analysis system based on a low-resolution dToF sensor, including a target segmentation module, a three-dimensional original point cloud generation module, a maximum longitudinal section determination module, a two-dimensional point cloud generation module, and a morphological parameter extraction module.

[0029] The target segmentation module is used to acquire RGB images and depth images of corn ears taken by a camera with a low-resolution dToF sensor, and to perform target segmentation on the RGB images, wherein the segmentation target is the corn ear;

[0030] The 3D original point cloud generation module is used to generate a 3D original point cloud of the ear of fruit based on the target segmentation results and the depth image.

[0031] The maximum longitudinal section determination module is used to determine the maximum longitudinal section of the ear's geometry based on the three-dimensional original point cloud of the ear.

[0032] The two-dimensional point cloud generation module is used to project the original three-dimensional point cloud of the ear of fruit radially onto the plane where the maximum longitudinal section is located, so as to obtain the two-dimensional point cloud of the ear of fruit.

[0033] The morphological parameter extraction module is used to extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear, and the phenotypic parameters include ear length and ear width.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. This invention discloses a method for phenotypic analysis of maize ears based on a low-resolution dToF sensor. The method first acquires RGB and depth images of maize ears captured by a camera using a low-resolution dToF sensor, and performs target segmentation on the RGB images. Then, based on the target segmentation results and the depth images, a three-dimensional original point cloud of the ear is generated. The maximum longitudinal section of the ear's geometric shape is determined based on the three-dimensional original point cloud. The three-dimensional original point cloud is then radially projected onto the plane containing the maximum longitudinal section to obtain a two-dimensional point cloud of the ear. Finally, phenotypic parameters of the ear are extracted based on the two-dimensional point cloud. This method proposes a point cloud plane convergence projection method, which corrects all offset point clouds and projects them onto the maximum cross-sectional plane of the ear, thereby transforming the complex three-dimensional point cloud fitting problem into a two-dimensional plane geometric analysis problem. Within a tolerable error range, it achieves a simplified representation of the complex three-dimensional morphology of the ear, effectively solving the problem of ear size measurement.

[0036] 2. This invention proposes an improved YOLOv8 model for target segmentation of RGB images using a low-resolution dToF sensor-based corn ear phenotypic analysis method. This model replaces the initial convolutional layers in the backbone network structure with a ConvNormLayer that integrates normalization operations, and adds an MSFBlock multi-scale fusion module to the shallow layers of the backbone network. This reduces the number of model parameters and computational load while maintaining the backbone's feature extraction capabilities. Furthermore, the RepC3 reparameterization module replaces the C2f module in the neck network structure. This module employs a multi-branch convolutional structure during training, using parallel stacking of 3×3 and 1×1 convolutions to enhance feature learning capabilities. Through the synergy of these two improvements, an optimal balance between accuracy and efficiency is achieved, maintaining strong detection capabilities while reducing the model's computational scale. Attached Figure Description

[0037] Figure 1 A schematic diagram of the overall framework for improving the YOLOv8 model.

[0038] Figure 2 This is a schematic diagram of the MSFBlock multi-scale fusion module.

[0039] Figure 3 This is a schematic diagram of the RepC3 reparameterization module.

[0040] Figure 4This is a schematic diagram of coordinate transformation in Example 1.

[0041] Figure 5 This is a schematic diagram of the overall framework of model G.

[0042] Figure 6 This is a schematic diagram of the AIFI attention module in model G.

[0043] Figure 7 This is a structural block diagram of the system described in this invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0045] This invention proposes a method for analyzing the phenotypic characteristics of corn ears based on a low-resolution dToF sensor. This method, based on the geometric principles of linear light propagation, employs a ray-plane convergence projection method to optimize coarse point cloud data acquired by a mobile device, effectively solving the challenge of size measurement. This ray-plane convergence projection method is not custom-developed for the iPhone 12 Pro Max's TOF sensor; its core logic is built upon the geometric principles of depth measurement, possessing cross-device and cross-scenario versatility. Spatial undersampling and mixed pixel errors are inherent limitations of all low-resolution depth sensors: whether it's various TOF modules in other brands' mobile phones, budget-friendly industrial depth cameras, or some lightweight LiDAR modules, as long as the instantaneous field of view of a single pixel is larger than the detail size of the measured target, edge depth confusion and point cloud offset along the light rays will inevitably occur.

[0046] Compared to traditional 3D measurement schemes that rely on high-precision hardware or multi-view stereo matching, this method only requires a single frame of RGB-D data to complete all calculations. It does not require additional calibration equipment or complex preprocessing procedures. While ensuring that the phenotypic parameter error is within the acceptable range for agricultural production and scientific research, it significantly reduces the technical threshold and hardware cost of 3D plant phenotypic measurement. It provides a feasible technical path for the popularization of low-cost portable measurement technology and also provides a new solution for the application of low-quality depth data in other fields.

[0047] In this invention, the resolution of the low-resolution dToF sensor can be as low as 256*192.

[0048] Point cloud filtering: If the sensor provides depth confidence function, depth confidence can be collected. Based on the depth confidence, low-confidence and medium-confidence depth points in the original 3D point cloud of the ear can be removed to obtain the filtered 3D point cloud of the ear Q. If the sensor does not provide depth confidence function, statistical filtering, radius filtering, small cluster point cloud removal and other methods can be used to filter the original 3D point cloud of the ear.

[0049] Example 1:

[0050] This embodiment uses an iPhone 12 Pro Max as a camera employing a low-resolution dToF sensor to implement the corn ear phenotypic analysis method based on a low-resolution dToF sensor. The specific steps are as follows:

[0051] 1. Using an iPhone 12 Pro Max, images of corn plants in the R5 stage (when the ears are morphologically mature and dry matter accumulation is about to cease, meeting the conditions for yield estimation) were taken. During image capture, all ears were within the frame, centered and facing upwards, avoiding extreme angles (where the ears are not fully visible) and strong sunlight. A total of 120 images of corn ears were collected; additionally, 800 RGB images were collected using the original camera for training the segmentation model.

[0052] The acquired image data includes RGB images (1920*1440), depth maps (256*192), and depth confidence images (256*192). The iOS-based mobile application "Field Maize Phenotypic Ground Data Acquisition Terminal," built on the Apple ARKit 6 framework, was used. This application extracts pixel-by-pixel depth values ​​using the Depth API and evaluates the confidence level of each pixel's depth value. The confidence level is generated based on the reliability of the depth measurement, and evaluation metrics include laser echo signal-to-noise ratio, inter-frame consistency of depth, spatial gradient change, and effective point cloud density. ARKit categorizes the confidence level into three levels: high confidence, medium confidence, and low confidence, corresponding to stable and reliable regions, generally reliable regions, and unreliable regions, respectively.

[0053] 2. Use X-AnyLabeling annotation software to annotate polygon bounding boxes. The annotation object is the entire ear outline, from the tip of the bract to the ear growth support node. Convert the labels to txt format supported by YOLO. The ear dataset is randomly divided into training set and validation set in a 7:3 ratio.

[0054] 3. The instance segmentation model for the fruit spike dataset was trained using an improved YOLOv8 model. The input image resolution was adjusted to 640*640 pixels, with 200 training epochs, a batch size of 8, and an initial learning rate of 0.001. A cosine annealing strategy was used to dynamically adjust the learning rate, resulting in a final learning rate decay coefficient of 0.01. Furthermore, a mixup technique (with a ratio of 0.15) was introduced to enhance the model's robustness to occlusion and complex backgrounds. An early stopping mechanism was implemented during training, with a patience value of 10 epochs. The entire training process was performed on a workstation equipped with an NVIDIA RTX 4060 Ti 8G graphics card.

[0055] The improved YOLOv8 model framework is as follows: Figure 1 As shown, this model systematically improves the network architecture of the YOLOv8 model, mainly in two aspects:

[0056] (1) To address the issues of channel redundancy and insufficient shallow feature extraction in the original YOLOv8 backbone network, the backbone network structure is simplified and optimized as follows:

[0057] The initial convolutional layers are replaced with ConvNormLayer that integrates normalization operations. An MSFBlock multi-scale fusion module is introduced into the shallow network to enhance information interaction between features at different scales. For the specific structure of the MSFBlock multi-scale fusion module, please refer to [link to documentation]. Figure 2 .

[0058] (2) To balance the model's feature representation capability and inference speed, the C2f module in the neck network structure is replaced with the RepC3 reparameterization module. This module employs a multi-branch convolutional structure during training, using parallel stacking of 3×3 and 1×1 convolutions to enhance feature learning capability. For the specific structure of the RepC3 reparameterization module, please refer to [link to RepC3 reparameterization module]. Figure 3 .

[0059] 4. The trained improved YOLOv8 model is used to segment the RGB image, and the segmentation target is corn ears.

[0060] 5. Determine the mask output by the identified target. The mask range is the entire corn ear. Extract the depth coordinates of all points within the mask range from the depth image. Combine the camera intrinsic parameters to generate the original 3D point cloud of the ear.

[0061] like Figure 4 As shown, this process is based on the inverse operation of the pinhole camera imaging model, using synchronously acquired depth information to recover the spatial coordinates of pixels. Since the size of the RGB image is 7.5 times that of the depth map, for each point on the RGB image, its coordinates are reduced to 1 / 7.5 and rounded down before the depth value z is read on the depth map. The three-dimensional spatial coordinates (x, y, z) are then calculated by combining the camera intrinsic parameters and the coordinates on the map. The conversion formula is as follows:

[0062]

[0063] In the above formula, u 、 v represents the x and y coordinates of the pixels on the graph, (c x c y ) represents the coordinates of the camera's principal point, f x f y This refers to the camera's focal length.

[0064] 6. Based on the three-dimensional original point cloud of the ear, determine the maximum longitudinal section of the ear's geometry, and project the three-dimensional original point cloud of the ear radially onto the plane containing the maximum longitudinal section to obtain the two-dimensional point cloud of the ear.

[0065] Based on the back-projection principle of the pinhole camera model, any pixel in a 2D image corresponds to a spatial ray passing through the origin of the optical center in the camera coordinate system. When the camera's intrinsic parameters are known, the direction vector of this ray is uniquely determined, and the 3D spatial coordinates of the pixel are determined solely by the depth value z. All drift points caused by depth errors are not randomly distributed in 3D space, but rather offset strictly along the direction of their corresponding pixel's spatial ray.

[0066] Based on this key discovery, theoretically, as long as the plane containing the real object point can be determined, any drifting point can be corrected to its correct position on that plane along the direction of its ray. However, corn ears are three-dimensional structures with continuous circular arc undulations, and their real surface is not a single plane. Directly performing high-precision fitting on the complete three-dimensional surface is not only computationally extremely complex but also severely interfered with by a large amount of noise in the original point cloud. In order to significantly simplify the problem while ensuring the accuracy of phenotypic parameter calculation, this invention proposes the maximum cross-section equivalence assumption based on the approximately cylindrical morphological characteristics of corn ears: the core phenotypic parameters such as the length, width, and projected area of ​​the ear are highly consistent with its maximum longitudinal cross-section in geometry. In other words, after projecting the three-dimensional point cloud of the ear radially onto the plane containing its maximum cross-section, the calculated two-dimensional geometric dimensions can accurately characterize the true three-dimensional phenotypic of the ear.

[0067] If the maximum longitudinal section of the ear can be found, all point clouds can be projected onto that plane along the ray. This transforms the complex 3D point cloud fitting problem into a simpler, more robust 2D geometric analysis problem without losing phenotypic information. However, this approach has a core contradiction: the correction of drift points depends on the accurate location of the maximum section, but the numerous drift points present in the original point cloud severely interfere with the search process for the maximum section. If the unprocessed original point cloud is used directly, incorrect depth values ​​will cause systematic deviations in the location of the maximum section, rendering subsequent point cloud correction completely ineffective.

[0068] Therefore, this embodiment adopts a step-by-step processing flow of "first filtering and positioning, then global correction", specifically including:

[0069] 6.1. Difference confidence levels are determined for the points in the original 3D point cloud of the ear of fruit. Unreliable depth points with low and medium confidence are removed from the original 3D point cloud of the ear of fruit, while high confidence depth points are retained to obtain the filtered 3D point cloud of the ear of fruit, Q.

[0070] 6.2. Using the farthest point in point cloud Q as the base point and the camera plane angle as the reference, construct multiple planes parallel to the camera plane, with pitch and yaw angles both within the range of (-20°, 20°).

[0071] Because the camera's shooting direction differs from the ear's growth direction during shooting, pitch and yaw angle corrections are necessary. To avoid the influence of single-point noise, this step uses the 95% quantile point P of the point cloud Q in the camera's Z direction as the cross-sectional plane positioning reference (i.e., the base point), instead of directly using the maximum depth point.

[0072] In this embodiment, within the range of (-20°, 20°), a fixed step size of 1° is used to traverse and sample, completing the pairwise combination of pitch angle and yaw angle in all dimensions, and generating all planes that may be the largest cross section of the ear.

[0073] 6.3 Project the point cloud Q onto each constructed plane and calculate the corresponding projected area. Use the plane with the largest projected area as the maximum longitudinal section in the ear's geometry, thus achieving maximum section positioning. Specifically,

[0074] The point cloud Q is vertically projected onto various planes, and principal component analysis is performed on the point cloud on the planes. A new coordinate system is established with the direction of the first principal component as the Y-axis and the perpendicular line to the Y-axis in the plane containing point cloud Q as the X-axis. The point cloud Q is then divided into a series of equidistant slices with minimal thickness along the Y-axis. Each slice can be approximated as having a length of x. i The height is y i The rectangle whose area is the product of the width of the slice along the X-axis and the thickness of the slice, S. i The sum of the areas of all valid slices gives the area S, which is the projected area of ​​the plane.

[0075] 6.4 Move all points in the original three-dimensional point cloud of the ear of fruit (including the filtered low and medium confidence points) along the line connecting it to the origin of the coordinate system to the plane containing the maximum longitudinal section. At this point, the geometric correction of the entire point cloud is completed, and the resulting planar point set is the two-dimensional point cloud of the ear of fruit.

[0076] In this step, the specific method of tracing the line connecting it to the origin is as follows: taking the camera optical center as the origin, each three-dimensional point in the original three-dimensional point cloud of the ear of fruit is translated along the radial line connecting the origin and the point to the plane where the maximum longitudinal section is located.

[0077] 7. Extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear. These phenotypic parameters include ear length, ear width, and ear area. The specific extraction process is as follows:

[0078] Principal component analysis (PCA) is performed on the two-dimensional point cloud of the ear of fruit. The direction of the first principal component is taken as the natural principal axis of the ear. The two farthest endpoints (points a and b) are searched along this direction; the Euclidean distance between these two points is the ear length. Subsequently, the two farthest points (points c and d) are searched along a direction perpendicular to the principal axis in the same plane; their Euclidean distance is the ear width. The calculation formula is as follows:

[0079]

[0080]

[0081] In the above formula, (x a y a , z a ), (x b y b , z b (x) are the coordinates of points a and b, respectively. c y c , z c ) 、(x d y d , z d ) are the coordinates of points c and d, respectively.

[0082] Considering the irregular shape and uneven edges of the ear of fruit, the traditional method of calculating the area by multiplying length and width is inaccurate. This step uses the concept of infinitesimal elements to calculate the area of ​​the ear. The specific method is as follows:

[0083] A new coordinate system is established with the direction of the first principal component as the Y-axis and the perpendicular line to the Y-axis in the plane containing point cloud Q as the X-axis. The system is then divided into a series of equidistant slices with minimal thickness along the Y-axis. Each slice can be approximated as having a length of x. i The height is y i The rectangle whose area is the product of the width of the slice along the X-axis and the thickness of the slice, A. i The area of ​​the ear is obtained by summing the areas of all valid slices.

[0084] The formula for calculating the area of ​​the ear is:

[0085] ;

[0086] In the above formula, , Let be the length and height of the i-th slice, respectively, and n be the number of slices.

[0087] To verify the effectiveness of the method described in this invention, the following comparative tests were conducted:

[0088] (1) To examine the performance of the improved YOLOv8 model described in this invention, three different optimization methods were considered: adding MSFBlock (replacing the initial convolutional layer in the backbone network structure with ConvNormLayer integrating normalization operations, and adding the MSFBlock multi-scale fusion module to the shallow network of the backbone network), replacing C2F with RepC3 (replacing the C2f module in the neck network structure with the RepC3 reparameterization module), and introducing the AIFI module (adding the AIFI attention module after the highest layer features of the neck network). The original YOLOv8 model and seven different improved YOLOv8 models obtained by combining the above three different optimization methods were trained for instance segmentation, and the precision (P), recall (R), average precision (AP) AP@50 and AP@50-95 were selected to evaluate the segmentation accuracy of these eight different models. The results are shown in Table 1:

[0089] Table 1 Training results of each segmentation model

[0090]

[0091] The overall framework of model G is as follows: Figure 5 As shown, the structure of the AIFI attention module is as follows: Figure 6 As shown.

[0092] As shown in Table 1, considering detection accuracy, inference efficiency, model complexity, and detection balance, model D (the improved YOLOv8 model used in this invention), which incorporates MSFBlock and RepC3, achieves the highest mAP50-95 value of 74.3%, a 9.2% improvement over the baseline model. It also boasts the lowest GFLOPs (7.9G), a 37.3% reduction compared to the baseline model, and the fastest inference speed. With only 3.13M parameters, slightly lower than the baseline model, it consumes less memory. Furthermore, its precision and recall are both 95.9%, achieving a perfect balance between the two, with false positives and false negatives kept to a minimum, demonstrating the most stable performance in practical applications. Other schemes employing only a single improvement or a combination of multiple improvements, while also improving some metrics, cannot achieve such a significant reduction in computational cost while maintaining ultra-high accuracy. These results demonstrate that the improved YOLOv8 model used in this invention exhibits the strongest positive synergistic effect, successfully achieving the optimal balance between accuracy and efficiency, reducing model computational scale while maintaining strong detection capabilities.

[0093] (2) Accuracy test of ear length and ear width detection

[0094] The error evaluation results of the ear length and ear width data obtained in Example 1 are shown in Table 2:

[0095] Table 2 Extraction performance of spikelet length and spikelet width

[0096]

[0097] Table 2 shows that the ear length prediction error is only 4.16%, and the ear width error is only 5.29%. This result demonstrates that the method described in this invention successfully solves the measurement error problem caused by insufficient mobile phone hardware performance. Even when using ordinary mobile phones, our algorithm can effectively overcome the limitations of low depth resolution and point cloud drift, achieving accurate measurement of maize ear size and providing a feasible solution for high-precision geometric inversion under low-cost hardware.

[0098] Example 2:

[0099] See Figure 7 A corn ear phenotypic analysis system based on a low-resolution dToF sensor includes a target segmentation module, a three-dimensional original point cloud generation module, a maximum longitudinal section determination module, a two-dimensional point cloud generation module, and a morphological parameter extraction module.

[0100] The target segmentation module is used to acquire RGB and depth images of corn ears captured by a camera using a low-resolution dToF sensor, and to perform target segmentation on the RGB images, wherein the segmented target is the corn ear. The specific execution method of this module is described in steps 1-4 of Example 1.

[0101] The 3D original point cloud generation module is used to generate a 3D original point cloud of the ear of fruit based on the target segmentation results and depth image. For the specific execution method of this module, please refer to step 5 in Example 1.

[0102] The maximum longitudinal section determination module is used to determine the maximum longitudinal section of the ear's geometry based on the original three-dimensional point cloud of the ear. For the specific execution method of this module, please refer to steps 6.1-6.3 in Example 1.

[0103] The two-dimensional point cloud generation module is used to project the original three-dimensional point cloud of the ear of fruit radially onto the plane containing the largest longitudinal section, thereby obtaining a two-dimensional point cloud of the ear of fruit. For the specific execution method of this module, please refer to step 6.4 in Example 1.

[0104] The morphological parameter extraction module is used to extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear. The phenotypic parameters include ear length, ear width, and ear area. For the specific execution method of this module, please refer to step 7 in Example 1.

Claims

1. A method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor, characterized in that: The method includes: S1. Acquire RGB and depth images of corn ears captured by a camera using a low-resolution dToF sensor; perform target segmentation on the RGB images, where the segmentation target is the corn ear; S2. Based on the target segmentation results and depth images, generate the original 3D point cloud of the ear of fruit. S3. Determine the maximum longitudinal section of the ear's geometric shape based on the three-dimensional original point cloud of the ear; S4. Project the original three-dimensional point cloud of the ear of fruit radially onto the plane containing the maximum longitudinal section to obtain the two-dimensional point cloud of the ear of fruit. S5. Extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear, wherein the phenotypic parameters include ear length and ear width.

2. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 1, characterized in that: S3 includes: S31. Perform point cloud filtering on the original three-dimensional point cloud of the ear of fruit to obtain the filtered three-dimensional point cloud of the ear of fruit, Q. S32. Using the farthest point in point cloud Q as the base point and the camera plane angle as the reference, construct multiple planes parallel to the camera plane, with pitch and yaw angles within the range of (-A°, A°). S33. Project the point cloud Q onto each of the constructed planes and calculate the corresponding projected area. Use the plane with the largest projected area as the largest longitudinal section in the geometric shape of the ear of fruit.

3. A method for analyzing the phenotypic characteristics of field crops based on a low-resolution dToF sensor according to claim 1 or 2, characterized in that: S5 includes: performing principal component analysis on the two-dimensional point cloud of the ear, extracting the maximum Euclidean distance in the direction of the first principal component as the ear length, and the maximum width perpendicular to the direction of the first principal component as the ear width.

4. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 1, characterized in that: The phenotypic parameters also include spikelet area; The method for extracting the ear area includes: after performing principal component analysis on the two-dimensional point cloud of the ear, establishing a new coordinate system with the direction of the first principal component as the Y-axis and the perpendicular line of the direction of the first principal component in the plane of the two-dimensional ear point cloud as the X-axis, then dividing it into multiple slices along the Y-axis, and calculating the ear area A based on the following formula: ； In the above formula, , Let be the length and height of the i-th slice, respectively, and n be the number of slices.

5. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 2, characterized in that: S1 uses an improved YOLOv8 model to segment the RGB image; The improved YOLOv8 model replaces the initial convolutional layer in the backbone network structure with a ConvNormLayer that integrates normalization operations, and adds an MSFBlock multi-scale fusion module to the shallow network of the backbone network.

6. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 5, characterized in that: The improved YOLOv8 model replaces the C2f module in the neck network structure with the RepC3 reparameterization module.

7. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 1, characterized in that: The S2 includes: determining the mask output by the identified target, extracting the depth coordinates corresponding to all points within the mask area from the depth image, and generating a three-dimensional original point cloud of the corn ear by combining the camera intrinsic parameters, wherein the mask area is the entire corn ear.

8. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 1, characterized in that: S4 includes: moving all points in the original three-dimensional point cloud of the ear of fruit along the line connecting it to the origin of the coordinate system to the plane where the maximum longitudinal section is located, and the resulting set of planar points is the two-dimensional point cloud of the ear of fruit.

9. The method for analyzing the phenotypic characteristics of maize ears based on a low-resolution dToF sensor according to claim 8, characterized in that: The specific method of moving along the line connecting it to the origin is as follows: taking the camera optical center as the origin, each three-dimensional point in the original three-dimensional point cloud of the ear of fruit is translated along the radial line connecting the origin and the point to the plane where the maximum longitudinal section is located.

10. A corn ear phenotypic analysis system based on a low-resolution dToF sensor, characterized in that: The system includes a target segmentation module, a 3D original point cloud generation module, a maximum longitudinal section determination module, a 2D point cloud generation module, and a morphological parameter extraction module. The target segmentation module is used to acquire RGB images and depth images of corn ears taken by a camera with a low-resolution dToF sensor, and to perform target segmentation on the RGB images, wherein the segmentation target is the corn ear; The 3D original point cloud generation module is used to generate a 3D original point cloud of the ear of fruit based on the target segmentation results and the depth image. The maximum longitudinal section determination module is used to determine the maximum longitudinal section of the ear's geometry based on the three-dimensional original point cloud of the ear. The two-dimensional point cloud generation module is used to project the original three-dimensional point cloud of the ear of fruit radially onto the plane where the maximum longitudinal section is located, so as to obtain the two-dimensional point cloud of the ear of fruit. The morphological parameter extraction module is used to extract phenotypic parameters of the ear based on the two-dimensional point cloud of the ear, and the phenotypic parameters include ear length and ear width.

Images