A hydroponic lettuce phenotyping segmentation method, system, medium, and device

Abstract

The application discloses a kind of hydroponic lettuce phenotype segmentation method, system, medium and equipment, it is related to plant phenotype identification field, including obtaining the multi-view RGB image of hydroponic lettuce plant, multi-view RGB image is separated and three-dimensional reconstruction is carried out to original plant point cloud;Original plant point cloud is input to pre-trained improved Mask3D model, and the instance point cloud of each leaf is obtained;Adaptive local secondary sheeting algorithm ALLT algorithm is used, and the instance point cloud of each leaf is projected to the half curved surface consistent with curvature along the local normal direction, to determine the leaf point cloud after sheeting;Based on the sheeting after leaf point cloud and whole plant point cloud, the phenotype parameters of leaf and the phenotype parameters of whole plant point cloud are obtained.

Description

Technical Field

[0001] This invention relates to the field of plant phenotypic recognition, and in particular to a method, system, medium, and device for segmenting the phenotypic characteristics of hydroponic lettuce. Background Technology

[0002] Reliable leaf-level phenotypic extraction remains a major bottleneck in hydroponic lettuce cultivation, primarily due to leaf shading, complex surface curvature, and background interference. High-precision plant phenotypic analysis is the cornerstone of modern precision agriculture and data-driven breeding.

[0003] In recent years, deep learning methods based on labeled data have made significant progress in the field of 3D plant organ segmentation, enabling fine semantic and instance separation of structurally complex crops. Among them, Mask3D, as the first end-to-end 3D instance segmentation model using Transformer, generates multi-scale point cloud features by employing a sparse feature backbone network to voxelize and extract hierarchical features from the input point cloud. It then uses a set of learnable initial instance queries, which are iteratively updated by the Transformer decoder, mapping each query to an instance mask. This method of querying the mask replaces the traditional manual voting and geometric post-processing process.

[0004] However, the initial instance queries in the Mask3D model are generated using random initialization or fixed preset methods, failing to fully utilize the global geometric distribution information inherent in the input point cloud itself. This limitation results in poor stability and spatial consistency of model instance prediction when processing plant point clouds with complex structures and varied morphologies, easily leading to false positives or false negatives, thus reducing the accuracy of hydroponic lettuce leaf instance segmentation. Summary of the Invention

[0005] This invention provides a method, system, medium, and device for phenotypic segmentation of hydroponic lettuce, to solve the aforementioned problems existing in the prior art, namely, how to improve the accuracy of phenotypic segmentation of hydroponic lettuce in the prior art. This invention provides a method for phenotypic segmentation of hydroponic lettuce, which includes: Acquire multi-view RGB images of hydroponic lettuce plants and generate original plant point clouds; The original plant point cloud is input into a pre-trained improved Mask3D model to obtain the instance point cloud of each leaf; the improved Mask3D model includes a feature backbone module and a decoding module; the feature backbone module includes a metric scale coordinate alignment (MSCA) module added before the backbone network of the original Mask3D model; the decoding module includes a structure adaptive query refinement (SAQR) module and a learnable linear layer; The metric scale alignment (MSCA) module maps the original plant point cloud to the origin, determining the translated point cloud. The backbone network performs hierarchical feature extraction on the translated point cloud to determine multi-scale point cloud features. The structure adaptive query refinement (SAQR) module extracts the statistical features of the radial distance and vertical coordinates of the translated point cloud as a global geometric descriptor. It then projects the global geometric descriptor into a structure bias vector through a learnable linear layer. The structure bias vector is fused with the initial instance query preset by the original Mask3D model to generate an instance query. The instance query is projected onto features with the same dimension as the multi-scale point cloud features to determine the projected query vector. The query vector is then multiplied by the feature vector of each point in the multi-scale point cloud features to determine the mean probability of each query. Queries with a mean probability greater than a preset threshold are used as instance point clouds for each leaf.

[0006] Optionally, the point cloud of each leaf instance is projected onto a semi-curved surface consistent with the curvature to determine the sliced ​​leaf point cloud; based on the sliced ​​leaf point cloud and the whole plant point cloud, the phenotypic parameters of the leaf and the phenotypic parameters of the whole plant point cloud are obtained; wherein, the step of projecting the point cloud of each leaf instance onto a semi-curved surface consistent with the curvature to determine the sliced ​​leaf point cloud specifically includes: For each point in the leaf point cloud, the KNN algorithm is used to search for its k nearest neighbors to construct a set of neighbors; Determine the local neighborhood covariance matrix of the set of neighboring points, perform eigenvalue decomposition on the local neighborhood covariance matrix, and determine the eigenvalues ​​after decomposition. , and Based on eigenvalues , and The normalized curvature is obtained in the following way: in, and These are the other two eigenvalues ​​corresponding to the changes in the principal components. It is the smallest eigenvalue. Indicates the first Normalized curvature at each point; By introducing a scaling factor and combining it with normalized curvature, the neighborhood search radius of each point is obtained in the following way. : in, It is a scaling factor. To prevent extremely small constants with a denominator of zero; Within the neighborhood search radius, the center point in the point cloud of any leaf is obtained using principal component analysis (PCA). The normal vector is determined by cross product, and the normal vector is determined by cross product. Two perpendicular tangent vectors and Construct a local coordinate system and place neighboring points Transform to a local coordinate system and use the least squares method to find the nearest neighbor points after the transformation. Fit the material to a local quadratic surface and repeat the above steps to determine the point cloud of the thinned leaf.

[0007] Optionally, the phenotypic parameters of the leaves include leaf length, leaf width, leaf area, leaf circumference, and leaf tilt angle; the phenotypic parameters of the whole plant point cloud include plant height, crown width, projected area, and plant volume.

[0008] Optionally, principal component analysis can be performed on the point cloud of the sliced ​​blade to establish a standardized three-dimensional coordinate system; In a standardized three-dimensional coordinate system, the leaf vein curve is extracted using Dijkstra's shortest path algorithm, and the Euclidean distances between adjacent three-dimensional discrete points on the leaf vein curve are accumulated and used as the leaf length. Extract the maximum and minimum coordinate values ​​of all points in the leaf point cloud after thinning, along the horizontal direction perpendicular to the main leaf vein, and use the difference between the maximum and minimum coordinate values ​​as the leaf width; Extract the principal axis vector of the first principal component in the standardized three-dimensional coordinate system, and use the angle between the principal axis vector and the obtained blade gravity direction as the blade tilt angle.

[0009] Optionally, the edge boundary points of the sliced ​​leaf point cloud are extracted and the edge boundary points are serialized and sorted to form a closed polygonal boundary line. The area of ​​the two-dimensional plane enclosed by the polygonal boundary line is taken as the leaf area. The adjacent boundary points of the sliced ​​leaf point cloud on the 2D projection plane are obtained, and the sum of the Euclidean distances between the adjacent boundary points is taken as the leaf perimeter.

[0010] Optionally, the plant height is defined as the vertical distance from the root to the highest point of the lettuce plant; and the diameter of the smallest circle enclosing the projection points of all leaves is defined as the crown width.

[0011] Optionally, before inputting the original plant point cloud into the pre-trained improved Mask3D model, the original plant point cloud is preprocessed, specifically including: point cloud clustering and separation, scale restoration, coordinate alignment, multi-stage denoising and sampling processing.

[0012] This invention provides a phenotypic segmentation system for hydroponic lettuce, realizing the above-mentioned phenotypic segmentation method for hydroponic lettuce.

[0013] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described hydroponic lettuce phenotypic segmentation method.

[0014] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described hydroponic lettuce phenotypic segmentation method.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention provides a phenotypic segmentation method for hydroponic lettuce. This method uses metric scale coordinate alignment (MSCA) technology to align the coordinates of the lettuce point cloud while maintaining its physical scale. The structure adaptive query precision SAQR module extracts the radial distance and vertical coordinate statistical features of the aligned point cloud to construct a global geometric descriptor. This descriptor is then projected into a structural bias vector through a learnable linear layer and fused with the initial instance query preset by Mask3D. This allows the instance query to carry the prior spatial structural knowledge that "lettuce leaves are radially distributed from the center outward" at the initial stage. The SAQR module then dynamically adjusts the query vector based on the interaction results between the instance query and the point cloud features, optimizing the spatial positioning and semantic expression capabilities of the instance query layer by layer. This makes the segmentation results more spatially reasonable, reduces false positives and false negatives, and thus improves the accuracy of leaf instance segmentation. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] Figure 1 A flowchart of a hydroponic lettuce phenotypic segmentation method provided in an embodiment of the present invention; Figure 2 This invention provides a technical roadmap for a hydroponic lettuce phenotypic segmentation method. Figure 3 This is a Mask3D architecture diagram of a 3D point cloud segmentation network provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of plant surface phenotypic extraction provided in an embodiment of the present invention; wherein, Figure 4 (a) The "minimum envelope sphere" algorithm used for CW extraction, with the minimum envelope circle of the plant marked in red; Figure 4 (b) is the Delaunay triangle partitioning algorithm used for PA extraction, where the area calculation is based on the remaining edges after filtering out the gray edges; Figure 4 (c) Algorithm for PV extraction; Figure 5This is an overall flowchart of the adaptive local secondary blade thinning algorithm provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of leaf-level phenotypic extraction provided in an embodiment of the present invention; wherein, Figure 6 (a) is a standardized three-dimensional coordinate system established through principal component analysis; Figure 6 (b) is a schematic diagram of LVC extraction, where green dots represent near endpoints and red dots represent far endpoints. The red path connecting two points represents the geodesic path points extending along the LVC; Figure 6 (c) is Isomap projecting 3D leaf point clouds onto a 2D plane; Figure 6 (d) is a schematic diagram of edge filtering in the leaf area extraction process; Figure 6 (e) is the leaf perimeter, calculated by accumulating the Euclidean distances between adjacent boundary points along the red line; Figure 6 (f) represents the boundary point that is projected back onto the 3D leaf surface;

[0018] Figure 7 This is a schematic diagram of the original RGB image and the binarized lettuce plant image provided in an embodiment of the present invention; wherein, Figure 7 (a) is the original RGB image of a single lettuce plant; Figure 7 (b) is the original RGB image of multiple lettuce plants; Figure 7 (c) is the binarized image of a single lettuce plant; Figure 7 (d) is a binarized image of multiple lettuce plants; Figure 8 This is a 3D point cloud of a lettuce plant provided in an embodiment of the present invention; Figure 8 (a) comes from a static dataset; Figure 8 (b) comes from a time series dataset; Figure 9 This is a segmentation result of lettuce plant leaves provided in an embodiment of the present invention, where different colors represent different leaf instances; wherein, Figure 9 (a) comes from a static dataset; Figure 9 (b) comes from a time-series dataset; Figure 10 Comparative analysis between predicted and measured phenotypic traits provided in embodiments of the present invention; wherein, Figure 10 (a) represents the height of the growth point; Figure 10 (b) represents the plant height; Figure 10 (c) represents leaf length; Figure 10 (d) represents the leaf width; Figure 10 (e) represents the leaf area; Figure 10 (f) represents the leaf circumference; Figure 11 Relevant heatmaps of phenotypic parameters provided in embodiments of the present invention; Figure 12 A comparative analysis of the measured and predicted values ​​of leaf-level phenotypic parameters with and without ALLT, provided for embodiments of the present invention; Figure 12 (a) is the leaf circumference; Figure 12 (b) represents the leaf area; Figure 12 (c) represents leaf width; Figure 13 The results of 3D reconstruction and instance segmentation of four leafy vegetables provided in the embodiments of the present invention; Figure 13 (a) is the reconstructed and processed plant point cloud; Figure 13 (b) shows the instance segmentation result, with colors used to distinguish different leaf instances; Figure 14 This paper presents a comparative analysis between the measured and predicted values ​​of the blade phenotypic parameters provided in the embodiments of the present invention. Figure 14 (a) to Figure 14 (d) represents the leaf length (LL), leaf width (LW), leaf area (LA), and leaf perimeter (LP) corresponding to LVs; Figure 14 (a) to Figure 14 (d) represents the leaf length (LL), leaf width (LW), leaf area (LA), and leaf perimeter (LP) corresponding to NLVs;

[0019] Figure 15 The multi-view imaging platform provided in this embodiment of the invention consists of four main units: illumination, imaging, motion, and gimbal; wherein, Figure 15 (a) is a schematic diagram of the imaging platform; Figure 15 (b) is a physical image of the imaging platform; Figure 15 (c) is a diagram of the main hardware configuration of the imaging platform; Figure 16 A schematic diagram of a computer device for the hydroponic lettuce phenotypic segmentation method provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0022] Figure 1 This is a flowchart of a hydroponic lettuce phenotypic segmentation method provided in an embodiment of the present invention, such as... Figure 1 As shown in the figure, this embodiment illustrates a method for phenotypic segmentation of hydroponic lettuce, including: S1: Obtain multi-view RGB images of hydroponic lettuce plants and generate original plant point clouds.

[0023] For example, the variety used in this embodiment is glass lettuce, chosen because its thin, curved, and often overlapping leaves make it a representative and challenging subject for 3D phenotypic analysis. Seeds were first evenly sown on pre-moistened sponges to ensure uniform moisture distribution during germination. After 72 hours, seedlings were transplanted root-down into prepared seedling sponges. The sponges were placed in seedling trays and then transferred to a growth chamber to promote uniform initial growth. Once the second true leaf emerged, seedlings with uniform morphology were selected and transplanted into a hydroponic tank containing 10 liters of nutrient solution. The nutrient solution had a pH of approximately 6.13 and an electrical conductivity (EC) of approximately 0.97 mS / cm. -1 Each hydroponic tank is equipped with an oxygen pump to maintain adequate dissolved oxygen levels. The temperature of the cultivation environment is maintained at 19±2℃, and the light intensity is adjusted to 250 μmol·m⁻². -2 ·s -1 The hydroponic cultivation period after seedling transplantation is 3 to 15 days to obtain representative 3D point cloud samples of seedlings.

[0024] For example, the Zhang Zhengyou calibration method can be used for camera calibration to obtain accurate intrinsic and extrinsic parameters. A total of 72 images are captured after each imaging cycle to ensure full angular coverage for reliable 3D reconstruction. The raw RGB images are automatically converted to the CIELab color space, and the color channels are binarized using threshold-based segmentation for foreground segmentation. The CIELab color space is well-suited for foreground segmentation in this invention due to its perceptual uniformity and ability to separate luminance (L channel) from color information (a and b channels). The binarized images are used as input data for multi-view 3D reconstruction, providing a reliable mask to limit the reconstruction process to the target plant area, which helps reduce background noise and minimize the influence of external elements. After image acquisition and preprocessing, the data is transmitted to a remote computing server for 3D reconstruction (Solimani et al., 2023). In the 3D reconstruction process, the background-filtered image was first used to generate a sparse point cloud using incremental SfM photogrammetry in the open-source framework Colmap, while the spatial structure of key feature points between views was reconstructed using previously calibrated camera parameters. Subsequently, the dense point cloud was reconstructed using the Patch-Matoh stereo algorithm in OpenMVS, transforming the sparse representation in Colmap into a high-resolution representation.

[0025] For example, the original 3D point cloud typically contains multiple lettuce plants, lacks a real-world scale and coordinate calibration, and contains noise generated during reconstruction (Lietal et al., 2022; Liu et al., 2023). To address this, this invention employs a density-based spatial clustering algorithm (DBSCAN) to first separate the multiple lettuce plant point clouds. A square calibration board is used as a reference to restore the real-world scale and correct its coordinate system. First, points belonging to the calibration board are extracted from the original point cloud based on its unique color characteristics. This effectively removes irrelevant points and accelerates the subsequent plane fitting process. The calibration plane is robustly estimated using the Random Sample Consensus (RANSAC) algorithm (Fischler and Bolles, 1981), ensuring the accuracy of the plane reconstruction.

[0026] Next, the Rodriguez formula was used to calculate a rotation matrix that aligned the plane normal with the Z-axis, thus flattening the ground plane. This rotation was applied to the entire point cloud data, positioning the calibration board at a Z-value of 0. The XY plane defined by the fitted calibration board was used as a reference, with the origin set at the center of the calibration board. The growth direction (positive Z-axis) was determined based on the primary spatial layout of the green points. The point cloud data was scaled to restore real-world dimensions. The scale and coordinates were corrected by measuring the maximum length of the square calibration board in the perpendicular XY direction, which was then aligned with the newly defined green points.

[0027] The reconstructed point cloud mainly contains edge noise, scattered noise, and residual calibration points. To address this, this invention develops a multi-stage denoising and sampling algorithm (MDSA), which comprises five consecutive steps designed to progressively remove noise and balance point distribution. First, a color-based filtering method is employed, using color thresholds to remove residual calibration points, planting points, and most edge noise. Subsequently, coarse radius filtering is performed to eliminate obvious outliers. Statistical OR (SOR) filtering is applied to further remove scattered noise. Next, voxel downsampling is performed on the plants to balance point density while maintaining the integrity of the overall plant structure. A fine radius filtering step is also implemented. This method aims to remove residual edge points and scattered noise. Finally, the farthest point sampling (FPS) algorithm is used to generate a uniformly distributed point set. All MDSA parameters were determined through pilot experiments on independent datasets to balance noise removal with morphological fidelity, thereby achieving reliable large-scale processing (see Table 1).

[0028] Table 1 Filtering and Sampling Parameters for Point Cloud Processing Color density threshold 75 Coarse radius filtering radius threshold 0.022 Coarse radius filtering minimum neighbor count 10 Statistical filtering neighbor count 50 Statistical standard deviation multiple 0.8 downsampling voxel size 0.03 Fine radius filtering radius threshold 0.035 Fine radius filtering minimum neighbor count 10 Number of sampling points at the farthest point 26000 For measuring plant height and growing point, the lettuce root base is used as a reference ground plane. To locate this point and set its Z-axis coordinate to zero, this invention performs a cylindrical search along the positive Z-axis, starting from the origin of the point cloud, identifying the lowest point within a specified XY radius as the root base. The point cloud is then translated to align this point with the coordinate origin, thus providing a consistent ground plane reference for subsequent measurements.

[0029] S2: Input the original plant point cloud into a pre-trained improved Mask3D model to obtain the instance point cloud of each leaf; the improved Mask3D model includes a feature backbone module and a decoding module; the feature backbone module includes a metric scale coordinate alignment (MSCA) module added before the backbone network of the original Mask3D model; the decoding module includes a structure adaptive query refinement (SAQR) module and a learnable linear layer; wherein, the metric scale coordinate alignment (MSCA) module maps the original plant point cloud to the origin, determining the translated point cloud; the backbone network performs hierarchical feature extraction on the translated point cloud to determine multi-scale point cloud features. The Structure Adaptive Query Refinement (SAQR) module extracts the statistical features of the radial distance and vertical coordinates of the translated point cloud as a global geometric descriptor. It then projects this global geometric descriptor into a structural bias vector through a learnable linear layer. This structural bias vector is fused with the initial instance query preset by the original Mask3D model to generate an instance query. The instance query is then projected onto features with the same dimension as the multi-scale point cloud features to determine the projected query vector. Finally, the query vector is multiplied by the feature vector of each point in the multi-scale point cloud features to determine the mean probability of each query. Query queries with a mean probability greater than a preset threshold are designated as instance point clouds for each leaf.

[0030] For example, this embodiment trains an improved Mask3D model by collecting a total of 706 point cloud samples of hydroponic lettuce (456 static samples and 250 time-series samples). The first stage generated 456 static point cloud samples of individual plants; the second stage monitored 25 plants for ten consecutive days, generating 250 time-series samples. All samples were manually segmented and labeled using CloudCompare software, and each leaf was assigned a unique instance label. The static dataset was divided into training, validation, and test sets, containing 330, 66, and 60 plants, respectively. Detailed manual measurements were performed on all 60 test set plants. Regarding the time-series dataset, 13, 6, and 6 plants were assigned to the training, validation, and test sets, respectively, generating 130, 60, and 60 time-series lettuce samples, as shown in Table 2.

[0031] Table 2. Dataset Information for Hydroponic Lettuce static dataset 456 330 66 60 Time series datasets 250 130 60 60 Total dataset 706 460 126 120 For example, leaf-level instance segmentation uses Mask3D (Schult et al., 2023), the first end-to-end 3D instance segmentation model using Transformers (Vaswani et al., 2017). It replaces manual voting and geometric post-processing with a simple "query-to-mask" approach for parallel semantic and mask prediction. Figure 3In this work, the original model centering step is replaced by converting the point cloud to a coordinate origin while maintaining its formal physical scale, thus providing a consistent spatial reference across samples. This adaptability allows the model to focus on the boundaries and shapes of individual leaves, thereby improving segmentation accuracy regardless of semantic categories.

[0032] like Figure 3 As shown, the metric scale coordinate alignment is done using the MSCA module ( Figure 3 The MSCA (Mean Search Address) is physically added at the boundary between the original point cloud input and the sparse backbone network (Feature Backbone), directly replacing the centralized preprocessing step of the original Mask3D model. The two sub-mechanisms of the Structure Adaptive Query Refinement (SAQR) module (structure bias injection mechanism and query refinement mechanism) are embedded in different core nodes of the Transformer decoder. The structure bias injection mechanism is located in the input stage of the Transformer decoder. It integrates the structure bias vector obtained by mapping the global geometric descriptor of the point cloud with the initial instance query preset by Mask3D, so that the instance query introduces the prior knowledge of plant spatial structure. The query refinement mechanism is located in the iterative update stage of the Transformer decoder. It dynamically adjusts the query vector through the interaction results between the instance query and the point cloud features, and optimizes the spatial localization ability and semantic expression ability of the instance query layer by layer, thereby improving the accuracy of leaf instance segmentation.

[0033] In terms of data association, the MSCA module splits the set of three-dimensional coordinate points aligned to the origin and retaining the actual physical scale into two. One path is fed into the sparse backbone network for hierarchical feature extraction, while the other path is directly input into the SAQR structure-aware query bias initialization submodule as a spatial prior. Figure 3 The initialization submodule uses the lossless coordinates output by MSCA to calculate the horizontal radial distance between the plant centroid and each point and extracts a compact plant-level geometric descriptor. After projecting the learnable linear layer into a structural bias vector, the element-wise summation node at the decoder entry point ( The query vector is added to and fused with the initial instance queries of the original model, so that the fused new query vector naturally carries the spatial structure guidance of the radial growth of lettuce, and is sent to the first-level query refinement module as the sole input source.

[0034] During the iterative refinement phase within the decoder, a query-by-query adaptive thresholding submodule is used. Figure 3 In The physical embedding is on the masked cross-attention computation path of each query refinement layer.

[0035] This submodule forms a closed-loop feedback relationship with the original model's masking module (MM): the intermediate probability mask image generated by the masking module after the dot product operation is directly input to this adaptive thresholding submodule, which dynamically calculates the mean probability corresponding to each query in the point dimension. After interval truncation, an adaptive threshold specific to this query is determined. Subsequently, this dedicated adaptive threshold The data is fed back to the mask cross-attention unit in real time, which dynamically filters the cross-attention weights and removes overlapping noise and background points below the threshold. This ensures that only high-confidence point features belonging to the leaf surface and edges participate in subsequent query updates, thereby greatly improving the original model's instance segmentation accuracy for small and thin leaves while maintaining the continuity of the algorithm.

[0036] For example, the sparse feature backbone first voxels the input lettuce point cloud, transforming continuous points into a discrete voxel grid suitable for sparse convolution processing. Hierarchical sparse convolution extracts multi-resolution features through progressive downsampling. Utilizing these multi-scale representations, the Transformer-based decoder progressively updates a constant set of instance queries. Each individual query initially appears as a multi-dimensional vector in some form to capture the morphology of candidate leaves.

[0037] In query-based 3D instance segmentation, instance queries typically do not explicitly consider the global geometry of the input lettuce point cloud during initialization, which can lead to unstable or spatially inconsistent instance predictions. To address this, this invention extracts a compact plant-level descriptor from the input to incorporate global structure priors into the decoding process.

[0038] Specifically, for the input plant point cloud First, calculate the centroid of the point cloud: Using the horizontal position of the centroid as a reference, calculate the horizontal radial distance of each point: Then, the radial distance of all points was calculated separately. and vertical coordinates Perform statistical analysis and calculate its mean and standard deviation: ， ， ， ， Finally, construct the global geometry descriptor [ , , , ] T.

[0039] This serves as a global geometric guide, enabling the model to encode the typical radial growth pattern of lettuce seedlings. The descriptor is projected into the query feature space through a learnable linear layer, generating a structure-aware bias term that is added to each instance query. By basing query conditions on globally geometric properties dependent on the samples, the decoder is guided toward structurally plausible instance assumptions. Across multiple decoding layers, queries aggregate spatially relevant features from voxels of different resolutions via a masked cross-attention mechanism and interact with each other through a self-attention mechanism to suppress duplicate predictions. Through iterative optimization, each query eventually converges to a discriminative representation corresponding to a single lettuce leaf.

[0040] The masking module generates the final instance mask. Each refined query is projected onto the same feature dimension as the high-resolution point feature map via a multilayer perceptron (MLP). The query embedding is then used to generate a similarity map through the dot product between points, which is then converted into a probability mask using a sigmoid activation function.

[0041] Furthermore, a query-based adaptive thresholding strategy is introduced to generate attention masks, aiming to improve performance when segmenting small and thin blades. For each query, the adaptive threshold is calculated as the average of the predicted mask probabilities across all points, generating a query-dependent threshold that reflects the overall confidence level of the predicted instances. This threshold is constrained within a predefined range to ensure numerical stability and prevent degradation. During cross-attention fine-tuning, points with mask probabilities below this adaptive threshold are excluded; points with higher confidence are retained.

[0042] Since the predicted sample set and the actual validation sample set are unordered, Mask3D uses the Hungarian algorithm during training to establish an optimal one-to-one correspondence. Matching cost. Defined as:

[0043] in, This represents the Dice loss between the predicted and the true mask. It is the binary cross-entropy loss used for mask prediction. This represents the cross-entropy loss used for instance semantic classification. (Coefficients) , and The contributions of each element are balanced, and in this invention, they are set as follows: and .

[0044] After determining the optimal allocation, aggregate all Auxiliary loss of each decoder layer : exist Layer mask loss Defined as: The entire training process lasted 400 epochs, with an initial learning rate of 1×10⁻⁶. -4 The training process was aided by the AdamW optimizer and a one-cycle scheduling strategy. Training was terminated when the validation performance stabilized. Experiments were conducted on an NVIDIA RTX 4090D (24GB) with a batch size of 16. To stabilize the training process and reduce overfitting, gradient pruning (maximum norm of 0.1) and weight decay (value of 5 × 10⁻⁶) were implemented. -4 ).

[0045] For example, plant height (PH) is calculated as the vertical distance from the root to the highest point of the lettuce plant: Crown width (CW) is calculated as the diameter of the smallest circle enclosing all the leaf projection points (e.g., Figure 4 As shown in (a), this invention uses the "minimum envelope sphere" algorithm to calculate this diameter:

[0046] in, These are the coordinates of each projection point on the XY plane. It is the center of the smallest enclosing circle.

[0047] The projected area (PA) is the region on the XY plane onto which a 3D plant is projected. The Delaunay triangulation algorithm is used to divide points into triangles (e.g., ...). Figure 4 As shown in (b). The area of ​​each triangle is calculated as follows:

[0048] The side length of each triangle is calculated as follows: Any triangle with a side length exceeding a specified threshold is discarded, and the total projected area is obtained by summing the areas of the remaining valid triangles.

[0049] Plant volume (PV) is the 3D space occupied by the entire plant. The alpha shape (concave shell) algorithm is used to calculate the volume (e.g., ...). Figure 4(as shown in (c)). For each blade instance, a watertight mesh is generated from its 3D point cloud by progressively adjusting the alpha parameter until a valid mesh is obtained. The volume of each blade is then calculated based on the mesh, and the PV is obtained by summing all the blade volumes.

[0050] For example, the original leaf point cloud obtained from multi-view reconstruction typically exhibits physical thickness and multi-layered structural features, stemming from a combination of intrinsic leaf volume and factors such as depth uncertainty and double surfaces generated during the reconstruction process (Whebell et al., 2021). However, for leaf-level phenotypic analysis, most morphological features are conceptually defined based on an idealized mid-leaf surface.

[0051] Therefore, the main challenge addressed in this invention is not surface denoising, but rather the restoration of a geometrically consistent neutral surface that most appropriately represents the neutral surface of thin biological organs. Traditional methods, such as projection methods based on principal component analysis (PCA) or moving least squares (MLS), typically assume that the local point distribution is symmetrical around the target surface and that the underlying geometry can be smoothly sampled. However, in MVS-reconstructed lettuce leaves, especially in images with high curvature regions, these assumptions are often violated; the point distribution near the midrib and leaf margin is often asymmetrical and locally layered.

[0052] To address this problem, this invention proposes an ALLT algorithm. This algorithm explicitly represents the reconstructed leaf point cloud as noise samples of a relatively ideal 2D surface embedded in 3D space. According to this representation, the variation along the local surface normal direction is primarily attributed to thickness-related errors, while the variation within the local tangential plane reflects the biologically significant leaf morphological structure. Therefore, the goal of ALLT is to iteratively project points along the locally estimated normal direction to a half-surface consistent with the curvature, while preserving the tangential geometric features crucial for accurate phenotypic measurements. Figure 5 All parameters were determined through pilot experiments on independent datasets and remained constant across all experiments to ensure reproducibility (Table 3).

[0053] Table 3 ALLT Algorithm Parameters KNN algorithm neighbor count 60 minimum radius 0.1 Maximum radius 1.0 Adaptive radius coefficient 0.01 Number of iterations 5 maximum angle 35° The local neighborhood of the target point is obtained using K-nearest neighbor search. The covariance matrix of this local neighborhood is calculated, and its minimum eigenvalue is obtained. This is used as a preliminary estimate of the curvature. Subsequently, this value is normalized based on the sum of all three eigenvalues ​​to obtain the normalized curvature. :

[0054] in, and These correspond to two other eigenvalues ​​that represent the changes in the direction of the principal components; It is the smallest eigenvalue; Indicates the first Normalized curvature at each point; To prevent extremely small constants with a denominator of zero.

[0055] For the center point Principal component analysis (PCA) is performed on its local neighborhood, and the eigenvector corresponding to the smallest eigenvalue of the covariance matrix is ​​selected as the normal vector. ; Introduce a with Non-parallel auxiliary vectors, through The cross product of the auxiliary vector and the first tangent vector is determined. Subsequently, using and The second tangent vector is determined by the quadratic cross product. ;by With the origin as the coordinate system, and mutually orthogonal vectors... Each as a local coordinate system axis, shaft and Axis, construct a local tangent space coordinate system. Connect neighboring points within the neighborhood. Relative to the center point Perform translation and centering, and then transform to the local coordinate system. :

[0056] Using the least squares method, the transformed nearest neighbors are fitted to a local quadratic surface, the equation of which is: in, to These are the surface fitting coefficients.

[0057] Obtain the origin in the local coordinate system. The projected height at that location is Then, the projection point is restored to the global 3D coordinate system to obtain the final projection point coordinates of the center point. :

[0058] This completes the projection contraction of a single target point onto an ideal neutral surface.

[0059] Based on the obtained local curvature Adaptively determine the neighborhood search radius for each center point in the next round. : in, The preset scaling factor; and the search radius. The cutoff is limited to the set effective range. within, that is ; To prevent extremely small constants with a denominator of zero.

[0060] The search radius is dynamically adjusted based on the local curvature, and the above process is repeated. Each iteration recalculates the point cloud normal vector to ensure the accuracy of the projection direction, thereby gradually eliminating point cloud thickness artifacts caused by 3D reconstruction and obtaining the final sheeted blade point cloud geometry.

[0061] Before applying the ALLT algorithm, growth points (undeveloped leaf buds) need to be processed separately to avoid distorting key morphological features. Due to their high structural density, complex geometry, and abrupt curvature changes, growth points are excluded from the pruning process, while their original point cloud data is preserved to maintain the integrity of the bud-level morphological structure, thus facilitating accurate phenotypic measurements.

[0062] First, the center point of each plant instance is calculated to accurately identify the growth point, and its horizontal distance relative to the Z-axis in the XY plane is calculated. Instances within a specified distance from the Z-axis are selected, and the angle between their first principal component and the Z-axis is determined using principal component analysis (PCA). Angle The calculation method is as follows:

[0063] in, It is the first principal component of the instance, referring to the eigenvector corresponding to the largest eigenvalue in the covariance matrix of the point cloud of that instance. Geometrically, it represents the principal axis direction in which the point cloud extends most significantly in three-dimensional space, that is, the "geometric central axis" or "major axis direction" of the instance. It is a unit vector along the Z-axis. The instances were retained as final growth point candidates, indicating that they had a near-vertical orientation, consistent with growth point characteristics.

[0064] The height of the growth point (HGP) can be calculated directly as follows: For example, the principal axis of each leaf is determined by performing principal component analysis (PCA) on its 3D point cloud data. The first principal component (PC1) is used as the leaf axis vector. And its z-coordinate components This is used to calculate the blade tilt angle (LIA). :

[0065] in, Representing vectors The Euclidean norm. The resulting angle. This represents the spatial attitude of the blade relative to the horizontal plane.

[0066] The leaf vein curve (LVC) is used to quantify the central axis of the leaf. The LVC is extracted via a geodesic path along the leaf centerline. First, the 3D leaf point cloud is normalized using principal component analysis (PCA) to establish a standardized coordinate system. Figure 6 (a)). Subsequently, the near and far ends of the blade are initially identified as the two points with the largest distance values ​​in the PC1 direction. Next, the blade base is further refined along the width direction (PC2) by selecting the midpoint to ensure the continuity of the geometry, and the Dijkstra shortest path algorithm is used to calculate the geodesic path from the near end to the far end of the blade, which represents the LVC ( Figure 6 (b)

[0067] The blade length (LL) is calculated as the sum of the Euclidean distances between consecutive points along the LVC. Let... Let represent the points distributed along the LVC, where and These represent the proximal and distal ends, respectively.

[0068] in, It represents the Euclidean distance between two adjacent points on the curve.

[0069] In a typical 3D lettuce leaf point cloud, its surface exhibits natural curvature, curling, and wrinkling characteristics. Therefore, directly measuring the area, perimeter, or width may deviate from the actual flat leaf state. Thus, this invention employs a nonlinear dimensionality reduction algorithm (Isomap) to project the 3D leaf point cloud onto a 2D plane. Figure 6 (c) While preserving both global and local geometry (Tenenbaum et al., 2000), after planarization, the main vein direction is defined as the reference axis (usually the X-axis). The leaf width (LW) is calculated along the vertical Y-axis.

[0070] Specifically, LW is defined as the maximum extension range of the leaf point cloud along the Y-axis. Leaf boundary points are arranged sequentially to form a closed polygon, with the last point connected to the first to form a closed structure. Excessively long sides are filtered out to reduce the impact of outliers. Figure 6 (d)). Leaf area (LA) is calculated using the polygon area formula:

[0071] in, It is the number of vertices. Used to close polygons.

[0072] The blade circumference (LP) is estimated by calculating the sum of the Euclidean distances between consecutive boundary points in the 2D planar blade representation. To ensure the polygon is a closed shape, the last point must be connected to the first point. Figure 6 (e):

[0073] in, It is the number of vertices, and Used to close polygons.

[0074] The blade circumference (LP) is estimated by calculating the sum of the Euclidean distances between consecutive boundary points in the 2D planar blade representation. To ensure the polygon is a closed shape, the last point must be connected to the first point. Figure 6 (f)

[0075] For example, the performance evaluation of the instance segmentation model uses several standard instance segmentation metrics, including mean precision (AP) and recall. These metrics are calculated as follows:

[0076] Here, TP, FP, and FN are the sets of true positives, false positives, and false negatives, respectively. At a fixed IoU threshold, mean precision (AP) is calculated as the area under the precision-recall curve, where precision and recall are obtained by varying the confidence threshold. The primary evaluation metric, AP, is the average of the AP values ​​at 10 different IoU thresholds (0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95).

[0077] This invention uses three common regression metrics: the coefficient of determination (R²). 2 The metrics are: Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). These metrics are calculated as follows:

[0078] in, It is the first The true value of each data point It is the first Predicted values ​​for each data point It is the average of the true values. That is the total number of points.

[0079] For example, the original multi-view images are generated by the imaging platform in single-plant or group-planting scenarios, respectively, as shown below. Figure 7 (a) and Figure 7 As shown in (b). After background segmentation, the image is binarized (as shown in the image). Figure 7 (c) and Figure 7 (as shown in (d)). Thanks to the perceptual consistency of the CIELab color space and its ability to decouple luminance information from color information, the target lettuce plant was effectively separated from the complex background, while its fine leaf structure was well preserved.

[0080] The background-filtered binarized image was used as input to the 3D reconstruction algorithm. Subsequent point cloud preprocessing (including clustering, coordinate alignment, scale recovery, and denoising) further standardized the data, thus largely eliminating reconstruction errors while preserving key morphological features. The resulting point cloud is of high quality and reliability (e.g., Figure 8 (a) and Figure 8 As shown in (b), this lays a solid foundation for subsequent leaf-level segmentation and phenotypic extraction.

[0081] For example, the performance of the improved Mask3D model in instance segmentation has been evaluated on a hydroponic lettuce test set. After simple post-processing (such as filtering out very small instances), typical segmentation results from a static dataset are shown. Figure 9 (a)). It also shows the segmentation results from the time-series dataset at different growth stages. Figure 9(b)). On both datasets, the improved Mask3D generated leaf-level instance masks with high accuracy, thus distinguishing individual lettuce leaves even in overlapping leaf regions. The improved Mask3D incorporates two main modifications for different stages. First, metric-scale coordinate alignment (MSCA) is used to map the input lettuce point cloud to the origin while maintaining its physical scale, ensuring geometric consistency among samples. Second, structure-aware query bias initialization and query-level adaptive threshold setting are integrated into a unified structure-adaptive query refinement (SAQR) algorithm, as both operations are performed at the instance query level and complement each other in improving query representation and decision confidence. MSCA and SAQR were evaluated individually and jointly in an ablation study (see Table 4).

[0082] Table 4. Instance segmentation results metrics for the Mask3D model Mask3D (Baseline) 91.8 96.8 97.6 96.2 97.5 + MSCA 93.8 98.1 98.5 97.1 98.1 + SAQR 92.9 97.4 97.9 97.9 98.6 Complete Model (This Invention) 94.5 98.6 98.8 98.1 98.9 Introducing MSCA significantly improved the AP value from 0.918 to 0.938, indicating a substantial improvement in stage recognition capability. SAQR also contributed to performance improvements, particularly in recall, increasing Recall50 from 0.963 to 0.979. When the two modules were integrated, the proposed complete model achieved an AP value of 0.945, while also improving recall to 0.981 (Recall50) and 0.989 (Recall25). These results demonstrate the complementary effects of MSCA and SAQR in improving segmentation accuracy and object coverage.

[0083] Phenotypic extraction was performed on 60 lettuce point cloud datasets from a static test set, and quantitative validation was conducted on six representative parameters, which were then compared with manual measurement results. The results show that there is consistency between the predicted and measured values ​​of all parameters, and all R... 2 The values ​​all exceeded 0.970 (e.g. Figure 10 (a) to Figure 10 (as shown in (f)). Leaf width (LW) and leaf area (LA) have the highest accuracy, with R0 being the highest for both. 2The values ​​were all 0.989. Accuracy was also high for plant height (PH, R² = 0.971), leaf length (LL, R² = 0.988), and growing point height (HGP, R² = 0.974). Prediction errors remained low for all traits; for example, the mean square error (MAE) for PH was 0.200 cm, and the root mean square error (RMSE) was 0.249 cm. Although the absolute error for leaf perimeter (LP) was relatively large (MAE = 0.678 cm, RMSE = 1.156 cm), these values ​​were acceptable given its large measurement range. The Pearson correlation matrix between fresh weight (FW) and the 11 automatically extracted phenotypic parameters is shown below. Figure 11 As shown, leaf width (FW) showed a strong correlation with traits related to size and area, particularly projected area (PA, r=0.98), total leaf width (TLW, r=0.97), total leaf area (TLA, r=0.97), total leaf perimeter (TLP, r=0.95), and total leaf length (TLL, r=0.93). This reflects the dominant contribution of cumulative leaf expansion to lettuce biomass accumulation. Crown width (CW) and plant volume (PV) were also highly correlated with FW (r=0.90), suggesting that canopy distribution and 3D occupancy provide complementary biomass information. The correlation between growing point height (HGP, r=0.62) and plant morphology index (PSI, r=0.56) was weaker, as these traits primarily describe plant structure rather than overall scale biomass.

[0084] For example, a test set of 121 leaves (consisting of relatively large areas and significant curvature) was constructed to rigorously evaluate ALLT. This design is based on two key rationales. First, these leaves represent extreme cases where inherent thickness and reconstruction artifacts are severe. If the ALLT algorithm demonstrates improvement when handling these challenging samples, it strongly suggests robust performance on simpler, flatter leaves. Second, by validating ALLT's performance on this difficult subset, it proves that the algorithm significantly enhances the robustness and accuracy of phenotypic extraction in complex, real-world environments, going beyond optimization for ideal conditions.

[0085] Ablation experiments on the blade test set confirmed the contribution of ALLT (Table 5).

[0086] Table 5. Performance comparison of predicting complex blades using and without the ALLT algorithm. LP 0.874 0.923 +0.049 LA 0.964 0.974 +0.010 LW 0.964 0.966 +0.002 MAE LP (cm) 1.465 1.098 -0.367 THE ( ) 0.750 0.647 -0.103 LW (cm) 0.133 0.123 -0.010 RMSE LP (cm) 1.803 1.409 -0.394 THE ( ) 1.057 0.907 -0.150 LW (cm) 0.172 0.166 -0.006 Positive values ​​indicate accuracy (R). 2 A higher value indicates an improvement, while a negative value indicates a reduction in estimation error (lower MAE and RMSE values).

[0087] Introducing ALLT consistently improves prediction performance across all three evaluated parameters (LP, LA, and LW), R 2 The MAE increased by 0.002 to 0.049, while the RMSE decreased by 0.010 to 0.367 and 0.006 to 0.394, respectively. For LL( Figure 12 (a) and LA ( Figure 12 For (b), a significant negative proportionality bias was observed in the untreated state, with the residuals gradually trending negative as the measured values ​​increased, indicating a tendency to gradually underestimate the values ​​on a larger scale. After applying ALLT, the residuals became more symmetrically distributed near zero, and the regression slope also decreased. For LW( Figure 12 Regarding (c), no significant systematic bias was observed in either state, with residual values ​​concentrated near zero and exhibiting a nearly horizontal trend; however, a slight reduction in residual dispersion was still observed after applying ALLT. In summary, these results indicate that ALLT not only improves geometric accuracy but also enables more stable and reliable calculation of blade-level phenotypic indices under complex blade conditions.

[0088] Additional experiments were conducted on four leafy vegetable crops to evaluate the generalization ability of the proposed 3D phenotypic analysis pipeline across different varieties. These experimental results were integrated into the original dataset for training. The crops were divided into two groups: leafy lettuce (LVs), including upright romaine lettuce and butter lettuce; and non-leaf lettuce (NLVs), including Chinese cabbage and baby bok choy. A total of 58 samples were collected, using the same imaging hardware and reconstruction algorithm as used for photographing glass lettuce. Of these, 22 samples were included in the training set, 14 samples were used in the validation set, and the remaining 22 samples were reserved as a separate test set for which manual measurements were performed. The final instance segmentation performance achieved a mean precision (AP) of 0.941. Figure 13 (a) and Figure 13 As shown in (b), despite significant differences in leaf morphology, size, curvature, and spatial arrangement among different species, the proposed pipeline method can still successfully distinguish individual leaf instances with clear boundaries. These results indicate that the learned geometric and structural representations are not overfitted to a particular variety, but rather capture general leaf-level characteristics.

[0089] Whether for LVs or NLVs, the predicted values ​​show a robust linear correlation with the actual manual measurement results (e.g. Figure 14 (a) to Figure 14 (as shown in (h)). Higher R 2The low values ​​of MAE and RMSE indicate that the proposed pipeline exhibits robust generalization performance and can be effectively applied to other leafy vegetable varieties. The predictive accuracy of LVs across all phenotypic traits is similar to that of glass lettuce. In contrast, NLVs show slightly larger prediction bias, especially in terms of LP. This is mainly attributed to the more complex leaf margin morphology of NLVs (such as baby bok choy) compared to the relatively smooth and regular leaf margins of LVs, which helps in more accurate perimeter estimation.

[0090] At the data acquisition level, the proposed method emphasizes phenotypic reliability rather than isolated algorithmic performance through the joint design of imaging hardware and data processing algorithms. A crucial factor affecting reconstruction accuracy is image acquisition quality. The imaging platform designed in this invention, combined with hardware-assisted background suppression technology, significantly reduces feature mismatches caused by the background, improving reconstruction consistency. Compared to traditional static camera setups, this design increases viewpoint diversity while maintaining stable imaging conditions, which is essential for generating dense and complete point cloud data for leafy crops.

[0091] At the data processing level, leaf instance segmentation determines whether 3D reconstruction results can be reliably converted into organ-level phenotypic features. In this invention, the Mask3D framework is not simply used as a general instance segmentation model, but is also adapted to better adapt to the geometric and tissue characteristics of plant-scale point clouds. By incorporating plant-scale spatial consistency and global structural cues into instance prediction, this segmentation method can maintain more reliable and accurate leaf boundaries. Simultaneously, allowing instance prediction to adapt to its own confidence distribution helps preserve small or thin leaves that are often suppressed in general settings.

[0092] Therefore, even in samples with different plant structures, leaf instances maintain spatial coherence and numerical stability. However, if the geometric representation of the leaf is biased, segmentation alone is insufficient for accurate phenotypic analysis. The reconstructed point cloud itself has thickness, which can affect subsequent planarization and the accurate calculation of leaf-level 3D phenotypic features. The proposed ALLT algorithm directly addresses this issue by converting thick leaf point clouds into neutral leaf surfaces. Importantly, these improvements are not limited to isolated features but encompass the entire set of structural parameters, thus supporting comprehensive phenotypic feature description using the proposed method. This is particularly crucial for hydroponic systems, where subtle morphological differences can reflect variations in nutrient supply, light conditions, or specific varietal growth strategies.

[0093] Beyond its methodological contributions, the main innovation of this study lies in its system-level co-design framework, in which hardware configuration and algorithm processing are jointly optimized. This framework no longer treats individual modules in isolation but incorporates each stage into a holistic consideration. The process design from image acquisition to phenotypic extraction fully considers downstream processing requirements. By systematically integrating each stage, the workflow ensures simultaneous achievement of data quality, geometric consistency, and algorithmic robustness, which is crucial for reliable and high-precision phenotypic analysis in agricultural applications (Tardieu et al., 2017; Detring et al., 2024; Hamidon and Ahamed, 2024). A strong Pearson correlation coefficient between fresh weight (FW) and extracted 3D phenotypic features further confirms the practical applicability of the proposed workflow for non-destructive biomass estimation. Figure 11 Compared to traditional 2D-based methods that typically suffer from systematic underestimation due to leaf curling and self-shading, the high-precision 3D reconstruction technology employed in this invention captures the complete spatial structure and accumulated leaf expansion—suitable for hydroponic lettuce seedlings. This accurately tracks changes in biomass accumulation over time without destructive sampling, providing a quantitative basis for phenotypic-driven breeding and precision nutrient management in agriculture. Future research could integrate various structural features into multivariate regression frameworks or deep learning-based estimators to further improve the accuracy of yield predictions at different growth stages.

[0094] Compared to other existing techniques that also focus on lettuce analysis, such as LettuceP3D (Ge et al., 2025), which reports an average R0... 2The value exceeds 0.950, and the AP value is 86.7% for instance segmentation. However, its workflow relies on commercial 3D reconstruction software to generate point cloud data without background segmentation, which may introduce additional noise and limit the flexibility of pipeline integration. In contrast, this invention combines a custom-designed active imaging platform with an open-source reconstruction pipeline covering all stages of the phenotypic analysis process. Compared to powerful general-purpose 3D reconstruction platforms, this invention achieves a shift from reconstruction to quantization. For example, the "Plant to Camera" platform built by Yang et al. (2024) integrates a Raspberry Pi camera and a motorized turntable, using U2-Net for background segmentation while also employing a modified SfM algorithm. It demonstrates high measurement relevance for overall parameters such as plant height and crown width; however, the final output of such platforms typically contains general point cloud data, while this invention focuses on downstream, task-specific processing to automatically extract precise leaf-level features. The resulting dataset contains both plant-scale and leaf-scale phenotypic parameters, which can be further used to construct time-varying growth curves, thus providing specialized quantitative solutions for precision agriculture decision-making. Traditional surface processing methods, such as Laplace smoothing and moving least squares (MLS), assume that the point samples cover a single continuous surface with high-frequency noise (Levin, 1998; Taubin, 1995). When applied to leaf point clouds with relative surface curvature and thickness defects, these methods often lead to layer blurring, geometric shrinkage, or surface distortion, thus affecting surface-dependent phenotypic measurements. Skeletonization methods can extract topological structure but discard most surface geometry information, and are therefore not suitable for surface-dependent trait analysis (Tagliasacchi et al., 2012).

[0095] In contrast, ALLT represents leaf point clouds as thin volumetric sheets and aims to recover geometrically consistent neutral surfaces. By applying an adaptive local quadratic fitting technique sensitive to curvature, points are projected onto this surface along local normal directions, effectively reducing thickness distortion while preserving local geometric features.

[0096] In this invention, Isomap is used to project 3D leaf point clouds onto a 2D plane while preserving both global and local geometry (Tenenbaum et al., 2000). In contrast, methods such as Uniform Manifold Approximation and Projection (UMAP) primarily focus on preserving local neighborhoods, which can lead to distortion of the overall leaf shape and affect the accurate measurement of leaf area and perimeter (Huang et al., 2022; Liu et al., 2025). Compared to commonly used color spaces (such as HSV), the a (green-red) and b (blue-yellow) channels of the Lab color space more stably represent chlorophyll and are less sensitive to changes in light intensity (Yu et al., 2022; Kumar and Miklavcic, 2018). This makes them more suitable for robust plant-background separation in controlled phenotypic observation environments.

[0097] For example, this invention proposes a hardware control system based on a Raspberry Pi to automate the imaging platform, such as... Figure 15 (a) and Figure 15 As shown in (b). The hardware architecture of this platform comprises four key units, as follows: Figure 15 As shown in (c), the motion unit precisely controls the rotation angle, speed, and range of the stepper motors, ensuring the stability and repeatability of the platform's motion during multi-view data acquisition. Each camera is mounted on a pair of dual servos, with the primary servo handling coarse positioning and the secondary servo for fine angle adjustment. This configuration ensures high-precision orientation and consistent camera repositioning. All servos are driven by a dedicated control board. Motion commands are transmitted from the Raspberry Pi via serial communication, enabling flexible and synchronized multi-camera control. The imaging unit within the platform consists of two Raspberry Pi HD cameras, one above the other. Both cameras are equipped with Sony IMX477R high-resolution CMOS sensors, which provide superior image clarity, high sensitivity, and low noise performance. The illumination system consists of three layers (upper, middle, and auxiliary layers) designed to ensure uniform illumination and minimize shadows. Illumination intensity is controlled by an STM32 microcontroller via PWM regulation. A white LED light source with a color temperature of 6500K and a CRI ≥ 95 is used as the emitter for diffused illumination, and is covered with sulfur paper to ensure that the illumination is always soft, uniform, and highly fidelity. In addition, the imaging chamber is covered with a black photographic cloth to further ensure the consistency of the imaging background.

[0098] The above describes one or more embodiments of the hydroponic lettuce phenotypic segmentation method provided in this specification. Based on the same idea, this specification also provides a corresponding hydroponic lettuce phenotypic segmentation system. Specific limitations regarding hydroponic lettuce phenotypic segmentation can be found in the limitations of the hydroponic lettuce phenotypic segmentation method described above, and will not be repeated here. Each module in the above-described hydroponic lettuce phenotypic segmentation system can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0099] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the hydroponic lettuce phenotypic segmentation method provided above.

[0100] The present invention also provides Figure 16 The schematic diagram of the computer device shown is as follows: Figure 16 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the hydroponic lettuce phenotypic segmentation method provided in the above embodiment.

[0101] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A method for phenotypic segmentation of hydroponically grown lettuce, characterized in that, include: Acquire multi-view RGB images of hydroponic lettuce plants and generate original plant point clouds; The original plant point cloud is input into a pre-trained improved Mask3D model to obtain the instance point cloud of each leaf; the improved Mask3D model includes a feature backbone module and a decoding module. The feature backbone module includes a metric scale coordinate alignment (MSCA) module added before the backbone network of the original Mask3D model; the decoding module includes a structure adaptive query refinement (SAQR) module and a learnable linear layer. The metric scale coordinate alignment (MSCA) module maps the original plant point cloud to the origin of the coordinate system to determine the translated point cloud; the backbone network performs hierarchical feature extraction on the translated point cloud to determine multi-scale point cloud features. The Structure Adaptive Query Refinement (SAQR) module extracts the statistical features of the radial distance and vertical coordinates of the translated point cloud as a global geometric descriptor. It then projects this global geometric descriptor into a structural bias vector through a learnable linear layer. This structural bias vector is fused with the initial instance query preset by the original Mask3D model to generate instance queries. These instance queries are then projected onto features with the same dimension as the multi-scale point cloud features to determine the projected query vector. Finally, the query vector is multiplied by the feature vector of each point in the multi-scale point cloud features to determine the mean probability of each query. Query queries with a mean probability greater than a preset threshold are designated as instance point clouds for each leaf.

2. The hydroponic lettuce phenotypic segmentation method as described in claim 1, characterized in that, Projecting the instance point cloud of each leaf onto a semi-curved surface consistent with its curvature to determine the sliced ​​leaf point cloud; based on the sliced ​​leaf point cloud and the whole plant point cloud, obtaining the phenotypic parameters of the leaves and the phenotypic parameters of the whole plant point cloud; wherein, the step of projecting the instance point cloud of each leaf onto a semi-curved surface consistent with its curvature to determine the sliced ​​leaf point cloud specifically includes: For each point in the leaf point cloud, the KNN algorithm is used to search for its k nearest neighbors to construct a set of neighbors; Determine the local neighborhood covariance matrix of the set of neighboring points, perform eigenvalue decomposition on the local neighborhood covariance matrix, and determine the eigenvalues ​​after decomposition. , and Based on eigenvalues , and The normalized curvature is obtained in the following way: in, and These are the other two eigenvalues ​​corresponding to the changes in the principal components. It is the smallest eigenvalue. Indicates the first Normalized curvature at each point; By introducing a scaling factor and combining it with normalized curvature, the neighborhood search radius of each point is obtained in the following way. : in, It is a scaling factor. To prevent extremely small constants with a denominator of zero; Within the neighborhood search radius, the center point in the point cloud of any leaf is obtained using the principal component analysis (PCA) method. The normal vector is determined by cross product, and the normal vector is determined by cross product. Two perpendicular tangent vectors and Construct a local coordinate system and place neighboring points Transform to a local coordinate system and use the least squares method to find the nearest neighbor points after the transformation. Fit the material to a local quadratic surface and repeat the above steps to determine the point cloud of the thinned leaf.

3. The hydroponic lettuce phenotypic segmentation method as described in claim 2, characterized in that, The phenotypic parameters of the leaves include leaf length, leaf width, leaf area, leaf perimeter, and leaf tilt angle; the phenotypic parameters of the whole plant point cloud include plant height, crown width, projected area, and plant volume.

4. The hydroponic lettuce phenotypic segmentation method as described in claim 3, characterized in that, Principal component analysis was performed on the point cloud of the thinned blade to establish a standardized three-dimensional coordinate system. Within a standardized three-dimensional coordinate system, the leaf vein curve is extracted using Dijkstra's shortest path algorithm, and the Euclidean distances between adjacent three-dimensional discrete points on the leaf vein curve are accumulated and used as the leaf length. Extract the maximum and minimum coordinate values ​​of all points in the leaf point cloud after thinning, along the horizontal direction perpendicular to the main leaf vein, and use the difference between the maximum and minimum coordinate values ​​as the leaf width; Extract the principal axis vector of the first principal component in the standardized three-dimensional coordinate system, and use the angle between the principal axis vector and the obtained blade gravity direction as the blade tilt angle.

5. The hydroponic lettuce phenotypic segmentation method as described in claim 3, characterized in that, Extract the edge boundary points of the sliced ​​leaf point cloud, and serialize and sort the edge boundary points to form a closed polygonal boundary line. The area of ​​the two-dimensional plane enclosed by the polygonal boundary line is taken as the leaf area. Obtain the adjacent boundary points of the sliced ​​leaf point cloud on the 2D projection plane, and take the sum of the Euclidean distances between the adjacent boundary points as the leaf perimeter.

6. The hydroponic lettuce phenotypic segmentation method as described in claim 3, characterized in that, The plant height is defined as the vertical distance from the root to the highest point of the lettuce plant; the crown width is defined as the diameter of the smallest circle enclosing the projection points of all leaves.

7. The hydroponic lettuce phenotypic segmentation method as described in claim 1, characterized in that, Before inputting the original plant point cloud into the pre-trained improved Mask3D model, the original plant point cloud is preprocessed, specifically including: point cloud clustering and separation, scale restoration, coordinate alignment, multi-stage denoising and sampling processing.

8. A hydroponic lettuce phenotypic segmentation system, characterized in that, The method for segmenting the phenotypic characteristics of hydroponically grown lettuce as described in any one of claims 1-7 is implemented.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the hydroponic lettuce phenotyping method according to any one of claims 1-7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hydroponic lettuce phenotypic segmentation method according to any one of claims 1-7.

Images