Soybean whole plant trait analysis method based on three-dimensional point cloud

Abstract

This invention relates to the field of soybean plant information analysis technology, and particularly to a method for analyzing the traits of the whole soybean plant based on three-dimensional point clouds. The method includes: employing a joint annotation method for soybean pod states based on three-dimensional point clouds to collect, annotate, and standardize the three-dimensional point clouds; constructing an improved SoftGroup multi-task learning model, where each module works collaboratively according to a preset logic, taking the three-dimensional point cloud of a single soybean plant as input, and outputting the target task results through point cloud feature extraction and instance clustering; performing multi-task joint training on the improved SoftGroup multi-task learning model, inputting the preprocessed three-dimensional point cloud of the soybean plant to be tested into the trained model, performing multi-task inference, refining geometric parameters through PCA, and counting the number of pods; removing the pod point cloud from the three-dimensional point cloud of a single soybean plant to obtain the main point cloud of the plant; extracting the plant skeleton from the main point cloud of the plant and obtaining the trait data of the plant skeleton.

Description

Technical Field

[0001] This invention relates to the field of soybean plant information analysis technology, and in particular to a method for analyzing the traits of whole soybean plants based on three-dimensional point clouds. Background Technology

[0002] Soybean plant traits (such as the number of pods, pod opening status, pod length / width / thickness, plant height, and number of branches) are core indicators for evaluating the quality of soybean varieties and predicting yield. Therefore, trait analysis of soybean plants is a crucial step in soybean breeding and cultivation research. Traditional trait analysis of soybean plants relies on manual peeling, counting, and visual judgment, which is not only time-consuming and labor-intensive but also prone to inaccurate data due to human subjective errors or mutual obstruction between pods and between pods and the main body of the plant. This makes it difficult to meet the needs of large-scale breeding and precision agriculture.

[0003] To address the drawbacks of manual soybean variety evaluation, the industry is gradually exploring the use of machine vision technology to assist in the evaluation process. Current technologies are mostly based on two-dimensional images: two-dimensional images of soybean plants are acquired using a camera, image segmentation algorithms are used to extract the pod region and the main body of the soybean plant, and feature extraction models are then used to identify pod bursting status, estimate pod size, and calculate the soybean plant's phenotypic data. However, two-dimensional images only reflect the planar projection information of the soybean plant and are easily affected by environmental factors such as changes in light, leaf occlusion, and shooting angle. For example, leaf occlusion can lead to incomplete segmentation of the pod region, and side lighting can cause stretching and deformation of pod size. These problems directly hinder the improvement of the accuracy of two-dimensional image-based variety evaluation, failing to meet the needs of high-precision breeding.

[0004] In recent years, three-dimensional point cloud technology has become an important direction for crop phenotypic detection because it can completely characterize the spatial structure of plants. Existing technologies have the following problems: (1) Three-dimensional point cloud acquisition mostly relies on single-view scanning or general registration algorithms, which easily leads to problems such as missing point clouds of soybean plants (especially single plants in the field) and large registration errors; (2) Point cloud labeling mostly adopts the method of labeling the pod state and the number of seeds separately, which requires manual maintenance of two types of label systems, resulting in low labeling efficiency (labeling a single plant takes more than 30 minutes) and poor label correlation; (3) The original point cloud has problems such as scale differences (caused by different plant sizes and scanning distances) and data redundancy. Directly inputting it into a deep learning model easily leads to low training efficiency and poor recognition generalization, making it difficult to achieve efficient and accurate joint recognition of pod traits.

[0005] In summary, current soybean plant trait analysis techniques still face problems such as low efficiency, poor accuracy, and weak multi-task collaborative capabilities. There is an urgent need for an automated and high-precision seed evaluation technology that can integrate the advantages of three-dimensional point clouds and achieve multi-trait collaborative detection to meet the actual needs of modern soybean breeding research. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a method for analyzing the traits of the entire soybean plant based on three-dimensional point clouds.

[0007] The purpose of this invention is to provide a method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds, which specifically includes the following steps:

[0008] S1. A joint annotation method for soybean pod state based on 3D point cloud was adopted to complete the acquisition, annotation and standardization preprocessing of soybean plant 3D point cloud, and the data was divided into training set and test set;

[0009] S2. Construct an improved SoftGroup multi-task learning model, including a pod-perception attention backbone network, a SoftGroup grouping module, a feature aggregation module, a multi-task learning head, and a PCA refinement module; each module works collaboratively according to a preset logic, taking a single soybean 3D point cloud as input, and outputting the target task result through point cloud feature extraction and instance clustering;

[0010] S3. Based on the training set and test set divided in step S1, perform multi-task joint training on the improved SoftGroup multi-task learning model, configure the loss function weight coefficients and training hyperparameters, and set an early stopping strategy.

[0011] S4. Input the three-dimensional point cloud of the soybean plant to be tested into the trained model after the preprocessing in step S1, perform multi-task inference, refine the geometric parameters through PCA, count the number of pods, and finally output the test results.

[0012] S5. Based on the pod identification results obtained in step S4, remove the pod point cloud from the three-dimensional point cloud of a single soybean plant to obtain the main point cloud of the plant; extract the plant skeleton from the main point cloud of the plant and obtain the phenotypic data of the plant skeleton.

[0013] Preferably, step S1 includes the following sub-steps:

[0014] S11. Perform multi-view scanning on the target soybean plant, simultaneously acquire depth and color images from each view, and place markers on the soybean plant for spatial reference.

[0015] S12. Spatial alignment of depth image data, unifying all depth image data to the same global coordinate system;

[0016] S13. Based on the registered depth image data and color image data, an initial color three-dimensional point cloud of soybean plants is generated. The initial color three-dimensional point cloud is then subjected to post-processing of noise removal, data simplification and surface smoothing to obtain an optimized three-dimensional point cloud model.

[0017] S14. Set up structured binary encoding rules, label the number of pods and the pod bursting state according to the structured binary encoding rules, clean up redundant or erroneous labels, and verify the labeling quality by instance segmentation IoU, number of pods and pod bursting state labeling accuracy.

[0018] S15. Point cloud standardization: The labeled point cloud data that has passed the S14 verification is resampled and spatially normalized, and the coordinates are mapped to the [-1,1] interval;

[0019] S16. Data partitioning: Divide the standardized point cloud into training and testing sets proportionally to maintain a balanced ratio of popped and non-popped points and a balanced distribution of point numbers.

[0020] Preferably, in step S11, a three-dimensional scanning device is used to scan the target soybean plant from multiple preset scanning stations, and depth images and color images from that perspective are acquired simultaneously at each scanning station; 4-6 marker points are set on a single soybean plant, and the marker points are respectively pasted on the base, different branches in the middle and the top main stem of the soybean plant, and the marker points do not cover the pod area.

[0021] Step S12 specifically includes the following sub-steps:

[0022] S121. Coarse registration: Based on marker points and geometric feature points extracted from depth images or appearance feature points extracted from color images, feature matching is performed between different images, and the rigid body transformation matrix between adjacent viewpoint images is estimated to achieve preliminary alignment.

[0023] S122. Fine registration and global optimization: The iterative nearest point algorithm is used to refine the coarse registration results; the bundle adjustment method is used to globally optimize the rigid body transformation matrix of all scanning stations to minimize the cumulative error, and finally unify all depth images to the same global coordinate system.

[0024] Step S13 specifically includes the following sub-steps:

[0025] S131. For each registered depth image, the pixel coordinates are calculated based on the camera intrinsic parameters using the principle of coordinate back projection. u , v ) and the depth value is converted to a 3D point in the global coordinate system. X , Y , Z The data is then processed and fused with the (R,G,B) color information of the corresponding color image to synthesize a complete initial color 3D point cloud.

[0026] S132. For the reconstructed initial color 3D point cloud, apply statistical filtering or radius filtering to remove outlier noise points; use voxelized mesh downsampling method to simplify the data while maintaining the model shape; use moving least squares method to smooth the point cloud surface to obtain the 3D point cloud data of soybean plants.

[0027] Preferably, step S14 further includes constructing a correspondence table according to structured binary encoding rules; the correspondence table includes the association information of decimal label, binary representation, number of grains and pod bursting state;

[0028] The encoding rule is as follows: the least significant bit of the binary number represents the pod-exploding state, 0 = no pod-exploding, 1 = pod-exploding; the index of the position of 1 in the second and above bits from the right represents the number of pods.

[0029] Based on the encoding rules and the corresponding relationship table, the optimized 3D point cloud model is segmented and labeled to distinguish between three semantic categories: popped pods, non-popped pods, and background. The number of grains in each pod is labeled, and a binary code conforming to the encoding rules is assigned to each pod instance.

[0030] Step S15 specifically includes the following sub-steps:

[0031] S151. Resampling: Set the number of target points N; use a random sampling algorithm to unify the point cloud scale. When the number of points exceeds N, downsampling is performed. When the number of points is less than N, the original data is retained. The sampling process is synchronously associated with spatial coordinates, normal vectors and label data.

[0032] S152. Normalization: Calculate the geometric center point of the point cloud and translate the entire point cloud to a reference system with the center point as the origin; then, through proportional scaling, map all coordinates to the [-1,1] interval to eliminate scale differences.

[0033] S153. Format Conversion and Storage: Convert the processed spatial coordinates, normal vectors, semantic tags, and instance tags into tensor format, combine and encapsulate them, and then serialize them into binary files for storage, with each plant as a unit.

[0034] Preferably, the pod-sensing attention backbone network performs downsampling and feature upsampling sequentially on the input 3D point cloud of a single soybean plant through a 3-layer Set Abstraction module. After each Set Abstraction module outputs features, a convolutional block attention module is embedded. First, the channel attention submodule performs average pooling and max pooling on the intermediate point cloud feature map, and calculates the channel weights through a shared MLP. Then, the spatial attention submodule performs spatial pooling and spatial weight calculation on the feature map. The dual weights are multiplied element-wise and normalized by the sigmoid activation function, and then the original features are weighted and fused to enhance the feature response of the pod region. Finally, the 3-layer Feature Abstraction module performs upsampling on the encoded high-dimensional features, and combines skip connections to supplement shallow spatial details, gradually restoring the original spatial resolution of the point cloud.

[0035] The SoftGroup grouping module first obtains feature association information through dual-branch prediction, then adopts the Top-downGrouping clustering strategy, combined with the feature association information, to aggregate spatially adjacent points with consistent semantic categories into preliminary pod instance clusters; finally, it uses ScoreNet to score the confidence of the preliminary instance clusters, filters high-confidence instance clusters and removes redundant instances, and outputs accurate pod instance proposals.

[0036] The feature aggregation module takes each pod instance proposal point set output by the SoftGroup grouping module as input, and maps the instance point set features through an MLP network. Each layer uses the ReLU activation function to introduce non-linearity, and uniformly transforms the high-dimensional features of instance point sets of different sizes into 32-dimensional fixed-dimensional instance feature vectors, providing standardized input for the multi-task learning head.

[0037] The multi-task learning head includes a parallel geometric parameter regression head and a pod classification head. The geometric parameter regression head first extracts features from the instance feature vector using an MLP[128,64,32], and then outputs the original geometric parameters through a fully connected layer. The pod classification head first enhances the classification features using an MLP[128,64,32], then performs feature mapping through a fully connected layer, and finally completes probability normalization through a Softmax layer, outputting the "non-pod" and "pod" category probabilities for each pod instance.

[0038] The PCA refinement module performs calibration and optimization on the original geometric parameters; and performs geometric feature decomposition and precise calibration on the point cloud of the pod instance based on principal component analysis.

[0039] Preferably, in the pod-perception attention backbone network, each Set Abstraction module first filters key sampling points by sampling the farthest point, then divides the local neighborhood point set with the sampling point as the center by ball query, and finally performs feature mapping on the neighborhood point set through MLP to improve the feature dimension.

[0040] The SoftGroup grouping module has two branches: a semantic prediction branch and an offset prediction branch. The semantic prediction branch is used to output the semantic scores of each point in the three-dimensional point cloud of a single soybean plant, corresponding to the three categories of "non-exploded pod", "exploded pod", and "background". The offset prediction branch is used to output the X / Y / Z offset of each point relative to the center of its respective pod instance, so as to characterize the positional relationship between the point cloud and the center of the corresponding pod instance in three-dimensional space.

[0041] In the feature aggregation module, the MLP network dimension is set to [128, 64, 32];

[0042] The original geometric parameters output by the geometric parameter regression head are K×3, where K is the number of instances and 3 corresponds to pod length, aspect ratio, and pod thickness.

[0043] The processing flow of the PCA refinement module is as follows: First, extract the point cloud of a single pod instance output by the SoftGroup grouping module, calculate the mean coordinate of the point cloud and translate it to a local coordinate system with the mean as the origin; then, construct a 3×3 covariance matrix and perform eigenvalue decomposition on it to obtain three eigenvalues ​​and corresponding eigenvectors; using the correlation between the eigenvalues ​​and the actual size of the pod, calculate the refined size in the principal axis direction; then, fuse the refined size with the original geometric parameters output by the multi-task learning head according to preset weights to output the final accurate three-dimensional size of the pod.

[0044] Preferably, in step S3, the weighting coefficients α, β, γ, δ of the loss function are set. The values ​​range from (0,1);

[0045] The multi-task loss function is a weighted sum of the losses of each sub-task, and the sub-task loss includes the pod classification loss. L cls Geometric parameter regression loss L reg Semantic prediction loss L sem Instance rating loss L score and group loss L group The formula is as follows:

[0046] ;

[0047] Pod Explosion Classification Loss Lcls The Focal Loss function is used to address the class imbalance problem in pod classification tasks and improve the classification accuracy of the minority classes; the expression is:

[0048] ;

[0049] In the formula, It is the probability that the model predicts the true class. γ It is an adjustable focusing parameter. It is a weighting factor used to balance the number of positive and negative samples;

[0050] Geometric parameter regression loss L reg The Smooth L1 Loss function is used to optimize the regression accuracy of geometric parameters and reduce the impact of outliers on loss calculation; the expression is:

[0051] ;

[0052] In the formula, and These are the true value and the predicted value of the i-th sample, respectively. β It is a threshold parameter.

[0053] Preferably, in step S4, multi-task reasoning includes: inputting the preprocessed point cloud into the model and simultaneously outputting the pod instance mask, the probability of the pod burst state, and the original geometric parameters;

[0054] The test results include the instance ID, instance mask, pod bursting status, number of pods, and parameters for pod length, pod width, and pod thickness for each pod.

[0055] Preferably, in step S5, the plant skeleton is extracted from the main point cloud of the plant, and the phenotypic data of the plant skeleton are obtained, specifically including:

[0056] The PC-Skeletor algorithm was applied to the main point cloud of the plant to generate the original topological skeleton.

[0057] Treating the topological framework as a graph, a graphical method is used to extract the main stem path of the plant from the topological framework.

[0058] Subtract all nodes and edges on the main stem path from the topological skeleton, and the remaining part is the branch skeleton;

[0059] Calculate the height of the main plant body based on the main stem path;

[0060] A series of normal cutting planes are generated along the main stem path. For each normal cutting plane, the RANSAC algorithm is used to fit each normal cutting plane to a circle. The diameter of each fitted circle is the stem thickness of the plant body at the corresponding normal cutting plane.

[0061] Each connection point between the branch skeleton and the main stem skeleton is defined as a node, and the total number of nodes is the number of main stem nodes of the plant.

[0062] In the branch skeleton connected to the main stem skeleton, the branch skeleton with a length greater than the set branch length threshold is defined as a valid branch, and the number of valid branches is the number of branches of the main body of the plant.

[0063] For each effective branch, take a segment of the skeleton vector near the connection point of the main stem skeleton, and take a segment of the skeleton vector of the main stem skeleton at that connection point. Calculate the branching angle between the two skeleton vectors, calculate the average value of all branching angles, and obtain the overall opening degree of the plant body.

[0064] Preferably, the main stem path of the plant body is extracted from the topological skeleton using a graphical method, including:

[0065] The main stem path is obtained by solving the following formula:

[0066] The main stem path is obtained by solving the following formula:

[0067] ;

[0068] in, Indicates the main stem path. Represents the topological skeleton from the root node The set of paths to all leaf nodes. Let e ​​represent the weight of edge e, and P represent the point cloud in the topological skeleton.

[0069] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0070] (1) Based on three-dimensional data, the whole soybean plant is analyzed. Compared with two-dimensional data, it has a stronger spatial expression ability, can capture plant structural features more comprehensively and accurately, and improves the accuracy of the analysis data.

[0071] (2) Using a deep learning model for point cloud segmentation to segment soybean plants and pods can effectively learn the local and global structural features in the point cloud, and achieve efficient and precise segmentation of soybean plants and pods.

[0072] (3) Through examples, individual phenotypic analysis can be performed on each pod, providing a data foundation for subsequent pod screening, superior breeding, etc.

[0073] (4) Extracting the skeleton of the plant and removing invalid point cloud noise can more effectively analyze the characteristics of the main trunk and lateral branches.

[0074] (5) The entire technical solution has a high degree of automation, high accuracy and high stability, which solves the inefficiency of previous manual analysis and has strong generalization. Attached Figure Description

[0075] Figure 1 This is a flowchart of a soybean whole-plant trait analysis method based on three-dimensional point cloud provided in an embodiment of the present invention.

[0076] Figure 2 This is a network framework diagram of the improved SoftGroup multi-task learning model provided according to an embodiment of the present invention.

[0077] Figure 3 This is a flowchart of the PC-Skeletor algorithm provided according to an embodiment of the present invention. Detailed Implementation

[0078] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0080] This invention provides a method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds, specifically including the following steps:

[0081] S1. A joint annotation method for soybean pod state based on 3D point cloud was adopted to complete the acquisition, annotation, and standardization preprocessing of soybean plant 3D point cloud. The specific operations are as follows:

[0082] S11. Multi-view data acquisition: Using a 3D scanner, the target soybean plant is scanned from multiple preset scanning stations; at each scanning station, the 3D scanner simultaneously acquires a depth image and a color image from that viewpoint (depth data and color data are acquired synchronously to ensure pixel-level spatial alignment); the depth image records the distance information of each pixel in the scene relative to the scanner, and the color image provides the corresponding texture color information; during the scanning process, marker points are placed on the plant or its supporting device as a common reference for subsequent registration.

[0083] In some embodiments, the device selection and parameter settings are as follows: Select a 3D scanner (model: FAROFocusS70), set the scanning accuracy to 0.1mm, the dot pitch to 0.5mm, and the scanning distance range to 1.5~3m; Simultaneously enable the color image acquisition function, set the color image resolution to 2048×1536 pixels, and the frame rate to 30fps to ensure pixel-level alignment between the depth image and the color image;

[0084] Use circular reflective markers with a diameter of 5mm (e.g., PET reflective film, reflectivity ≥85%). Place 4 markers on a single plant: 1 at the base of the plant (10cm from the ground on the stem), 2 on different branches in the middle (30-40cm from the base, avoiding the pods), and 1 on the top main stem (5cm from the top). The center of the marker must be flat and attached to the stem to avoid feature extraction errors caused by wrinkles.

[0085] Six scanning stations were evenly distributed in a 360° pattern on the horizontal plane, with the center of the soybean plant as the center (the angle between adjacent stations was 60°), and each station was 1.8m away from the center of the plant. For the distribution of pods in the vertical direction of the plant, each station was scanned in two layers (the lower layer scan height was 0-40cm, and the upper layer scan height was 40-80cm) to ensure that the top and bottom pods were completely covered. The scanning time for each station was 15s. During the scanning process, the scanner was kept still. If the plant shook slightly (amplitude ≤2cm), the station was scanned again once, and the overlap of the two scans was used for subsequent processing.

[0086] S12. Multi-source data registration: Spatial alignment of the acquired multi-view, independent coordinate system depth images; specifically including the following sub-steps:

[0087] S121. Coarse registration: Based on marker points and geometric or appearance feature points extracted from depth or color images, feature matching is performed between different images, and the rigid body transformation matrix between adjacent viewpoint images is estimated to achieve preliminary alignment.

[0088] In some embodiments, the compute_convex_hull convex hull function of the Open3D library is used to extract geometric feature points such as pod vertices and stem bifurcation points in the depth image (20-30 feature points are extracted from each depth image); the Canny edge detection algorithm (threshold: low threshold 50, high threshold 150) is used to extract appearance feature points in the color image (30-40 texture edge points are extracted from each color image); the extracted geometric feature points and appearance feature points are merged into a joint feature point set;

[0089] The marker matching method is as follows: Identify reflective markers (connected regions with grayscale values ​​≥ 240) in each depth image and obtain the three-dimensional coordinates of the four markers; using the coordinates of the markers at the first scanning station as a reference, calculate the Euclidean distance between the markers at the other five scanning stations and the reference marker, filter matching pairs with a distance error ≤ 0.3 mm, and preliminarily determine the positional relationship between adjacent stations;

[0090] The rigid transformation matrix estimation method is as follows: Based on the joint feature point set and the marker point matching results, the Random Sample Consensus (RANSAC) algorithm (1000 iterations, inlier threshold 0.2mm) is used to estimate the rigid transformation matrix (including translation vector and rotation matrix) of the depth images of adjacent stations, and complete the coarse registration. At this time, the error of the overlapping area of ​​the point cloud of adjacent stations is controlled within 1-1.5mm.

[0091] S122. Fine registration and global optimization: The iterative nearest point algorithm is used to refine the coarse registration results; the cluster adjustment method is used to globally optimize the rigid body transformation matrix of all scanning stations to minimize the cumulative error, and finally unify all depth images to the same global coordinate system.

[0092] In some embodiments, the fine-tuning method of the Iterative Closest Point (ICP) algorithm is as follows: taking the coarsely registered point cloud as input, the ICP algorithm is used for fine registration; the iteration termination condition is set: the number of iterations reaches 50 or the average distance error between two adjacent iterations is ≤0.05mm; in each iteration, 1000 random points in the overlapping area are selected as sampling points, the distance error of the corresponding point pairs is calculated, and the rigid body transformation matrix is ​​updated by the least squares method, so that the overlap error of the point clouds of adjacent stations is reduced to within 0.3mm;

[0093] The global optimization method of the bundle adjustment method is as follows: The point clouds of the 6 stations and their corresponding rigid body transformation matrices are input into the bundle adjustment algorithm. The optimization objective is to minimize the sum of squared cumulative distance errors of the point clouds of all stations. The constraint is the consistency of the transformation matrices of adjacent stations (the error between the product of adjacent matrices and the directly calculated cross-station matrix is ​​≤0.02mm). The optimization problem is solved by the Levenberg-Marquardt algorithm, and the globally optimal transformation matrix is ​​output. The depth images of all stations are unified to the same global coordinate system (the origin of the coordinate system is the center of the scanner of the first scanning station).

[0094] S13. Point cloud generation and post-processing: Generate the final three-dimensional point cloud model based on the registered data; specifically, generate the initial color three-dimensional point cloud of soybean plants based on the registered depth image data and color image data, and perform post-processing operations such as noise removal, data simplification and surface smoothing on the initial color three-dimensional point cloud to obtain the optimized three-dimensional point cloud model.

[0095] Specifically, it includes the following sub-steps:

[0096] S131. Color 3D Point Cloud Reconstruction: For each registered depth image, the pixel coordinates are reconstructed using the coordinate back projection principle and based on the camera intrinsic parameters. u , v ) and the depth value is converted to a 3D point in the global coordinate system. X , Y , Z The data is then processed and fused with the (R,G,B) color information of the corresponding color image to synthesize a complete initial color 3D point cloud.

[0097] In some embodiments, the (R, G, B) values ​​(ranging from 0 to 255) of the corresponding color image are read and compared with the three-dimensional spatial points ( X , Y , Z The points are associated one by one to generate an initial colored 3D point cloud of a single plant (approximately 80,000-100,000 points), which is then stored as a .pcd file.

[0098] S132. Post-process the reconstructed initial colored 3D point cloud to obtain the 3D point cloud data of soybean plants; this includes: applying statistical filtering or radius filtering to remove outlier noise points; using a voxelized mesh downsampling method to simplify the data while maintaining the model shape; and using algorithms such as moving least squares to smooth the point cloud surface. After the above processing, the 3D point cloud data of soybean plants can be obtained.

[0099] In some embodiments, the outlier noise removal method is as follows: a statistical filtering algorithm is used, the number of neighboring points is set to 10, the average distance between each point and its neighboring points is calculated, and points with a distance greater than "average distance + 2 times standard deviation" are identified as outliers (mostly dust and light spots in the scanning environment). After removal, the number of points in the point cloud is retained to about 90%; if there is local dense noise (such as reflective interference points on the surface of the stem), radius filtering is added (points with a search radius of 5mm and a number of neighboring points ≤ 3 are removed).

[0100] The voxel downsampling method is as follows: Set the voxel side length to 1mm, replace all points in the same voxel with the voxel center coordinates, and take the average (R,G,B) of all points in the voxel as the color value of the center coordinates, so that the number of point clouds is reduced to 20,000-25,000, reducing the amount of data while maintaining the pod morphological characteristics (such as pod length and curvature).

[0101] The surface smoothing method is as follows: using the moving least squares method, setting the neighborhood radius to 5mm and the polynomial order to 2, the surface of the downsampled point cloud is fitted to correct the small protrusions on the point cloud surface (such as the burrs on the surface of the bean pod caused by scanning noise), so that the surface roughness of the point cloud is ≤0.1mm, and finally the optimized color 3D point cloud is obtained.

[0102] S14. Joint Labeling and Quality Verification of Pod Traits: Establish structured binary encoding rules and construct a correspondence table; based on the binary encoding rules and the correspondence table, perform instance segmentation and labeling on the optimized 3D point cloud model, simultaneously completing label cleaning and quality verification to ensure the accuracy and completeness of the labeled data; specific operations are as follows:

[0103] S141. Encoding rules and correspondence table construction: Set structured binary encoding rules, where the least significant bit of the binary number represents the pod bursting state (0 = no pod bursting, 1 = pod bursting), and the index of the position of 1 in the second and above bits from the right represents the number of pods; construct a correspondence table based on this rule (as shown in Table 1), which includes the association information of decimal label, binary representation, number of pods, pod bursting state, and semantic meaning;

[0104] Table 1 Binary Encoding Correspondence Table

[0105]

[0106] S142. Instance Segmentation and Labeling: Open CloudCompare software, import the 3D point cloud file optimized in step S13, and enable the instance segmentation and labeling function; use the point cloud selection tool (rectangular selection + manual fine-tuning) to select instance objects within a single plant one by one: if the instance object is a "split pod", select all its point clouds and label it as "exploded pod"; if the instance object is a "closed pod", select all its point clouds and label it as "non-exploded pod"; if the instance object is a stem, branch, or leaf, select all its point clouds and label it uniformly as "background"; during the labeling process, assign a binary code that conforms to the S141 encoding rules to each pod instance to ensure that the code uniquely corresponds to the number of pods and the pod-exploded state; if the same pod is segmented into multiple point cloud clusters, use the software instance merging function to integrate them into a single instance; if different pod point clouds are stuck together, separate them using the point cloud cutting tool (based on a distance threshold of 2mm) to ensure the independence of each instance object.

[0107] S143. Label Cleaning: Targeted cleaning of the labeled raw data, specifically including:

[0108] Tag cleaning: Remove redundant tags (such as tags that are repeatedly labeled in the same instance) and incorrect tags (such as tags that mistakenly label the background as a bean pod);

[0109] Point cloud cleaning: merge broken point cloud segments of the same instance, separate incorrectly connected point clouds of different instances, and remove extremely small point cloud clusters with ≤50 points (such point clouds do not have characteristic representation significance).

[0110] S144. Quality Verification: Verify the annotation effect through quantitative indicators, calculating the instance segmentation intersection-over-union ratio (IoU), particle count annotation accuracy, and pod bursting state annotation accuracy of the annotated data; details are as follows:

[0111] Instance segmentation IoU: Calculate the intersection-union ratio (IoU) between the labeled pod instances and the real pod regions, requiring IoU ≥ 90%;

[0112] Particle count accuracy: The labeled particle count is compared with the actual particle count count after manual peeling, and the accuracy rate is required to be ≥95%;

[0113] Accuracy of labeling the pod breakage status: The accuracy rate of labeling the pod breakage status is required to be ≥96% compared with the actual status as determined by human visual inspection.

[0114] Samples that do not meet the above indicators are returned to S142 for re-labeling and cleaning until all samples meet the quality requirements.

[0115] S15. Point Cloud Standardization Processing: The labeled point cloud data that has passed the verification in S14 is resampled and spatially normalized to unify the data size and scale, providing standardized input for deep learning models; specifically including the following sub-steps:

[0116] S151. Resampling: To meet the requirement of consistent input data scale for deep learning models, the point cloud of each plant after integration is resampled: a predefined target number of points N is set, and a random sampling algorithm is used to adjust the scale of the point cloud data; for point clouds with a total number of points exceeding the target number of points N, random downsampling is performed to uniformly reduce the amount of data; for point clouds with a total number of points less than the target number of points N, all original data points are retained; the sampling process is applied synchronously to spatial coordinates, normal vectors, and all label data to ensure that the correspondence between various types of data remains unchanged after sampling, and to maintain the integrity of point cloud morphological features and annotation structure while unifying the data scale;

[0117] In a specific embodiment, the number of target points is set to N=22000 (to adapt to the input requirements of the improved SoftGroup model), and the point cloud scale is adjusted using a "pod region priority" random sampling algorithm:

[0118] For the bean pod point cloud region: retain the original point cloud at a 1:1 scale, and prioritize the retention of key morphological feature points such as bean pod vertices and edges;

[0119] For the background point cloud region: if the total number of points exceeds 22,000, the background point cloud is downsampled uniformly; if the total number of points is less than 22,000, all original point clouds are retained.

[0120] During the sampling process, spatial coordinates, normal vectors, semantic labels, instance labels, binary codes, and other data are synchronously associated to ensure that the correspondence between various types of data does not shift after sampling.

[0121] S152. Normalization: Spatial coordinate normalization is performed on the resampled point cloud data to eliminate spatial scale uncertainties introduced by differences in measurement distance, angle, or plant size. This process first calculates the geometric center point of all spatial coordinates of the point cloud and then translates the entire point cloud to a reference frame with this center point as the origin, i.e.: P center = P raw -C;

[0122] In the formula, P center The coordinates of the centered point cloud; P raw The coordinates of the original point cloud are N×3; C is the geometric center point; the expression for C is as follows:

[0123] ;

[0124] Calculate the absolute maximum value of each coordinate axis component of the point cloud in the translated coordinate system, and use this as the scaling factor. All spatial coordinates are scaled proportionally, ultimately normalizing the spatial distribution range of the entire point cloud to a standardized numerical range, resulting in the normalized point cloud coordinates. All of its coordinate values ​​are within the range [-1, 1].

[0125] S153. Format Conversion and Storage: Convert the normalized point cloud data into a format suitable for deep learning frameworks: Convert the spatial coordinate data representing the spatial location of the point cloud, the normal vector data representing the surface geometric characteristics, the semantic label data representing the object category, and the instance label data representing the uniqueness of the instance into the corresponding tensor formats respectively; combine and encapsulate each tensor according to the preset structure, and serialize them into binary data files (such as pth format) for storage, with each plant as the unit, to provide direct input for the training and inference of the SoftGroup model in the future.

[0126] S16. Data Partitioning: To ensure the generalization ability of model training and the effectiveness of testing, the standardized point cloud data stored in S15 is scientifically partitioned as follows:

[0127] S161. Sample selection: Standardized point cloud data of several potted soybean samples were selected. The samples covered multiple mainstream soybean varieties and were all in the pod-setting stage (R3-R6). The ratio of broken pods to non-broken pods in the samples was balanced, and 1-4 pods were evenly distributed to ensure the diversity and representativeness of the samples and meet the data distribution requirements for model training.

[0128] S162. Splitting rule: The selected samples are divided into training and test sets in an 8:2 ratio:

[0129] Training set: accounting for 80%, containing balanced samples from various varieties, used to improve the parameter training and optimization of the SoftGroup model;

[0130] Test set: accounting for 20%, with no overlap with the training set, containing representative samples of each variety, used for independent validation of model performance;

[0131] During the partitioning process, the class distribution of samples must be kept balanced (the ratio of popped pods to non-popped pods and the distribution of particle numbers must be consistent with the overall sample) to avoid poor model training performance due to data bias.

[0132] In some embodiments, the sample selection is as follows: standardized point cloud data of 100 potted soybean samples are selected, covering 5 mainstream soybean varieties (20 plants for each variety), with the ratio of broken pods to non-broken pods being 3:7, and 1-4 pods each accounting for about 25%, to ensure the rationality of the sample distribution; after corresponding division, the training set contains 80 samples (16 plants for each variety), and the test set contains 20 samples (4 plants for each variety), to further ensure the reliability of model training and validation.

[0133] S2. Improved SoftGroup Multi-Task Learning Model Construction: To achieve integrated processing of pod instance segmentation, pod classification, and geometric parameter estimation from a single soybean 3D point cloud, an improved SoftGroup multi-task learning model is constructed. This model takes a single soybean 3D point cloud as input (input format: , where is the number of points, 3 corresponds to X / Y / Z 3D coordinates), and integrates five core modules: a pod perception attention backbone network, a SoftGroup grouping module, a feature aggregation module, a multi-task learning head, and a PCA refinement module. Each module works collaboratively according to a preset logic to complete the entire process from point cloud feature extraction and instance clustering to the output of the target task results. The specific structure is as follows:

[0134] The pod-aware attention backbone network (feature extraction module) adopts a hierarchical architecture of "Set Abstraction + CBAM" to enhance the feature response of the pod region, suppress background interference, and achieve efficient encoding and decoding of point cloud features.

[0135] A three-layer Set Abstraction module (L1-L3) is used to downsample and upsize the input point cloud. Each layer follows the standard process of "sampling-grouping-feature extraction" to gradually reduce the number of points and increase the feature dimension, thereby realizing the transformation from the original coordinate information to high-dimensional semantic features.

[0136] After the output of each Set Abstraction module, a Convolutional Block Attention (CBAM) module is integrated, which calculates attention weights along both the channel and spatial dimensions: the channel attention submodule performs "average pooling + max pooling" operations on the intermediate point cloud feature map (dimensions C×H×W, where C is the number of channels), and calculates channel weights through a shared MLP (hidden layer dimensions set to [64,32]); the spatial attention submodule performs spatial average pooling and spatial max pooling operations on the feature map, and calculates spatial weights through a 7×7 convolution; finally, the dual weights are applied to the original features through element-wise multiplication, and the attention weights are normalized by the sigmoid activation function to output the enhanced features;

[0137] The intermediate point cloud feature map format is (N is the number of points, C is the number of channels);

[0138] Channel attention The calculation formula is as follows:

[0139] ;

[0140] Spatial attention The calculation formula is as follows:

[0141] ;

[0142] In the above formula, σ This represents the Sigmoid activation function (used to normalize weights to the 0-1 range). MLP This represents a multilayer perceptron (used for non-linear mapping of pooled features). AvgPool ( F ) indicates channel-dimensional average pooling of the feature map F; MaxPool ( F ) indicates that the feature map F is subjected to channel-dimensional max pooling; W 0、 W 1 represents the shared weight matrix of the MLP (parameters of the two-layer MLP). This represents the result of channel average pooling of feature map F; This represents the result of channel max pooling of feature map F; This represents a convolution operation with a size of 7×7;

[0143] The final weighted output features are: ;

[0144] A three-layer Feature Abstraction module (L2-L0) is used to upsample the encoded high-dimensional features, gradually restoring the spatial resolution of the point cloud, and providing feature support with both semantic information and spatial details for the subsequent grouping module.

[0145] The SoftGroup grouping module (the core module for instance segmentation) uses the output features after feature decoding to achieve accurate clustering and proposal generation of pod instances through a dual-branch prediction and top-down clustering strategy.

[0146] The dual-branch prediction consists of a semantic prediction branch and an offset prediction branch. The semantic prediction branch outputs three semantic scores for each point through a fully connected layer (corresponding to "non-pod", "pod", and "background" respectively), with an output format of N×C. The offset prediction branch outputs the offset of each point through a fully connected layer (with a dimension of 3, corresponding to the relative offset in the X, Y, and Z directions), with an output format of N×3, representing the positional relationship of the point to the center of its instance.

[0147] Top-down grouping clustering combines semantic prediction scores and offset results, and adopts a grouping strategy with a distance threshold of 2mm to aggregate spatially adjacent points with consistent semantic categories into preliminary instance clusters, thereby achieving separation between pod instances and background, as well as between different pod instances;

[0148] The instance proposal optimization uses ScoreNet to score the confidence of the initial instance clusters, filters out the set of instance proposals with high confidence, removes redundant instances, and outputs accurate pod instance proposals, providing a target instance basis for subsequent multi-task processing.

[0149] The feature aggregation module performs feature aggregation on the proposal point set of each pod instance output by the SoftGroup grouping module to generate a fixed-dimensional instance feature vector. The feature aggregation is implemented using an MLP network with dimensions set to [128, 64, 32]. Each layer uses the ReLU activation function. Through hierarchical mapping, the high-dimensional features of the instance point set are transformed into a unified-dimensional aggregated feature vector, providing standardized input for the multi-task learning head.

[0150] The multi-task learning head (classification and regression module), including a geometric parameter regression head and a pod classification head, takes the fixed-dimensional instance feature vector output by the feature aggregation module as input and uses a parallel branching structure to simultaneously complete pod state classification and pod geometric parameter regression, realizing multi-task collaborative inference.

[0151] The specific structure of the geometric parameter regression head is as follows: First, the instance feature vectors are extracted and mapped using a high-dimensional MLP [128,64,32]. Then, feature compression and target mapping are completed through a fully connected layer FC [32,3] (with ReLU activation function). Finally, the original geometric parameters K×3 are output (K is the number of instances, and 3 corresponds to pod length, aspect ratio, and pod thickness). The geometric parameter regression head is responsible for regressing the key size parameters of the pod. To conform to the prior physical structure of the pod (i.e., the pod width must not be greater than the pod length), it does not independently regress the length, width, and height, but rather regresses the pod length. L p (Estimated length along the main axis of the pod), aspect ratio R (The ratio of the secondary axis length to the primary axis length, with a value range of (0,1]), pod thickness T p (Estimated thickness along the shortest axis of the pod), the final pod width is calculated; this regression uses Smooth L1 Loss as the loss function, the formula is:

[0152] ;

[0153] In the formula, and These are the true value and the predicted value of the i-th sample, respectively. β It is a threshold parameter;

[0154] The specific structure of the pod-bursting classification head is as follows: First, the instance feature vectors are enhanced with MLP[128,64,32] for classification features; then, feature mapping is performed through a fully connected layer FC[32,3]; finally, probability normalization is completed through a Softmax layer, outputting the pod-bursting state probability (corresponding to the probabilities of "non-pod-bursting" and "pod-bursting" categories, respectively); Focal Loss is used as the loss function to solve the class imbalance problem that may exist in binary classification. The loss function formula is:

[0155] ;

[0156] In the formula, It is the probability that the model predicts the true class. γ It is an adjustable focusing parameter. It is a weighting factor used to balance the number of positive and negative samples.

[0157] The PCA refinement module (geometric parameter optimization module) is used to correct the original geometric parameter deviations output by the multi-task learning head. Based on the principle of principal component analysis, it performs geometric feature decomposition and precise calibration on the point cloud of the pod instance. The specific process is as follows:

[0158] Instance point cloud centering: Extract the instance point cloud of a single pod (formatted as N×3, representing the number of points in the instance) output by the SoftGroup grouping module, calculate the average coordinates of the point cloud on the X / Y / Z axes, and normalize the entire instance point cloud to a local coordinate system with the average as the origin by coordinate translation, thus eliminating the interference of position offset on size calculation;

[0159] Covariance Matrix Calculation: Based on the centered point cloud coordinate data, a 3×3 covariance matrix is ​​constructed. This matrix represents the discrete distribution characteristics of the point cloud in three-dimensional space and the correlation between each coordinate axis.

[0160] Eigen Decomposition: The constructed covariance matrix is ​​decomposed into eigenvalues ​​to obtain three eigenvalues ​​and corresponding unit eigenvectors, where the directions of the eigenvectors correspond to the three principal axes of the pod (long axis, width axis, and thickness axis).

[0161] Refined Dimensions: By utilizing the correlation between eigenvalues ​​and the actual size of the pod, refined dimensions are calculated in the three principal axis directions to obtain the precise dimensions of the pod in the length, width, and thickness directions;

[0162] Fusion with Regression: The refined dimensions calculated by PCA are weighted and fused with the original geometric parameters output by the multi-task learning head to output the final pod's 3D geometric parameters, balancing the inference efficiency of the regression model with the geometric accuracy advantages of PCA.

[0163] Specifically, for each pod instance point cloud First, decentralize the computation and calculate the centroid of the point cloud. Then the point cloud is translated to the origin. Then calculate the covariance matrix. Perform eigenvalue decomposition on the covariance matrix: , among which is Eigenvalues These are the corresponding eigenvectors. Finally, the dimensions of the pod in the three principal axes are proportional to the square root of the eigenvalues ​​for refined size calculation.

[0164] Through the collaborative design of the above modules, the improved SoftGroup multi-task learning model can directly output the instance mask, pod bursting status, pod number, and refined pod length / width / thickness parameters of each pod from the three-dimensional point cloud of a single soybean plant, meeting the application requirements of automated and high-precision soybean pod testing.

[0165] S3. Based on the training set and test set divided in step S1, perform multi-task joint training on the improved SoftGroup multi-task learning model, configure the loss function weight coefficients and training hyperparameters, and set an early stopping strategy.

[0166] Set the loss weight coefficients α, β, γ, δ. The values ​​range from (0,1); in a specific embodiment, α =0.3、 β =0.4、 γ =0.1、 δ =0.1、 =0.1;

[0167] Multi-task loss function: To achieve collaborative optimization of multiple tasks such as instance segmentation, pod classification, and geometric parameter regression, a weighted fusion multi-task loss function is designed to simultaneously constrain the training errors of each task and ensure a balanced improvement in the overall model performance; the total loss is the weighted sum of the losses of each sub-task, as shown in the following formula:

[0168] ;

[0169] In the formula, L cls For the pod-breaking classification loss, the Focal Loss function is adopted to solve the class imbalance problem in the pod-breaking classification task and improve the classification accuracy of the minority class (pod-breaking); L reg For geometric parameter regression loss, the Smooth L1Loss function is used to optimize the regression accuracy of geometric parameters (pod length, pod width, pod thickness) and reduce the impact of outliers on loss calculation. L sem For semantic prediction loss, the cross-entropy loss function is adopted to constrain the consistency between the three semantic scores output by the semantic prediction branch and the true category labels; L score For instance scoring loss, the MSE (mean squared error) loss function is adopted to optimize the confidence scoring accuracy of ScoreNet for instance proposals; L group For grouping loss, the DiceLoss function is used to ensure the overlap between the instance clusters output by the SoftGroup grouping module and the real instance masks; α, β, γ, δ, The loss weight coefficient has a value range of (0,1).

[0170] The hyperparameter settings are as follows: Batch size is set to 8 (to adapt to GPU memory capacity), initial learning rate is 0.001, and a cosine annealing learning rate decay strategy (T) is used. max=100, meaning a learning rate cycle is completed every 100 rounds), the number of training iterations is 100 rounds, and an early stopping strategy is set: when the total loss of the validation set does not decrease for 10 consecutive rounds, training is terminated early to avoid overfitting;

[0171] In some embodiments, the early stopping strategy is as follows: the total loss value of the test set is used as the early stopping criterion, and the total loss of the test set is calculated after each round of training; if the total loss of the test set does not decrease for 10 consecutive rounds (i.e., the loss value of the next round is greater than or equal to the loss value of the previous round), the early stopping mechanism is triggered to terminate the training process in advance; at the same time, the model parameters at the time of early stopping are saved as the final training model to avoid overfitting due to excessive iteration and to ensure the generalization performance of the model on the test set;

[0172] In some embodiments, to further improve training efficiency, mixed precision training (FP16) can be used, which can increase the training speed by about 40% while ensuring that the model accuracy does not decrease significantly.

[0173] S4. Input the 3D point cloud of the soybean plant to be tested into the trained model after the same preprocessing as in step S1, perform multi-task inference, refine the geometric parameters through PCA, count the number of pods, and output the test results, including the instance ID, instance mask, pod bursting state, number of pods, and parameters of pod length, pod width, and pod thickness for each pod.

[0174] Specifically, multi-task inference includes: inputting the preprocessed point cloud into the model and simultaneously outputting pod instance masks (distinguishing the spatial region of each pod), pod bursting state probabilities (the probability values ​​of "non-bursted" and "bursted" for each pod), and original geometric parameters (pod length). L p Aspect Ratio R Thick pods T p );

[0175] In some embodiments, the threshold for determining the pod-bursting state is set to 0.5: when the probability of "bursting pod" is ≥0.5, the pod is determined to be "bursting pod"; otherwise, it is determined to be "non-bursting pod". This threshold has been verified by the test set to enable the accuracy of pod-bursting state recognition to reach more than 95%.

[0176] Pod count: The number of pod instance masks output by the statistical model is the number of pods per soybean plant; if there are duplicate instance masks (due to point cloud adhesion), duplicate instances are merged by a distance threshold (2mm) to ensure that the accuracy of pod count is ≥98%;

[0177] The output includes: (1) Individual plant identifier: variety, growth period, and sampling time of the soybean to be tested; (2) Pod details: instance ID, instance mask (spatial coordinate range), pod breakage status (non-breakage / breakage), and number of seeds for each pod (parsed by binary encoding in step S14); (3) Trait parameters: number of pods per plant, and pod length, pod width, and pod thickness (unit: mm) for each pod.

[0178] In some embodiments, the accuracy indicators of the method for seed evaluation are: measurement error of pod length, pod width, and pod thickness ≤ 0.3 mm; accuracy rate of pod bursting state identification ≥ 95%; accuracy rate of pod count statistics ≥ 98%; and accuracy rate of grain count identification ≥ 92%.

[0179] S5. Based on the pod identification results obtained in step S4, remove the pod point cloud from the three-dimensional point cloud of a single soybean plant to obtain the main point cloud of the plant; extract the plant skeleton from the main point cloud of the plant and obtain the phenotypic data of the plant skeleton.

[0180] In some embodiments, in step S5, the plant skeleton is extracted from the main point cloud of the plant, and the phenotypic data of the plant skeleton are obtained. This process specifically includes:

[0181] S51. Apply the PC-Skeletor algorithm to the main point cloud of the plant to generate the original topological skeleton; specifically, apply the PC-Skeletor algorithm to the main point cloud of the plant to generate the original topological skeleton. The skeleton It consists of a series of nodes and connecting edges, capturing the overall topological structure of the plant. The core idea of ​​the PC-Skeletor algorithm is to shrink the dense point cloud into a central line that represents its topological structure. See also Figure 3 The algorithm flow is as follows:

[0182] Point cloud shrinkage initialization: Using the original plant main point cloud P (N points) as input, a K-nearest neighbor graph is constructed to establish neighborhood relationships between points. Based on these neighborhood relationships, a Laplacian matrix L is constructed. This matrix, along with the Laplacian operator, causes the center of each point's neighbor points to shift, resulting in a shrinkage energy function that gradually shrinks the point cloud.

[0183] ;

[0184] Among them, E - l represents the contraction energy function, and X represents the point cloud coordinate matrix;

[0185] To maintain the overall shape of the point cloud during the shrinkage process, a shrinkage constraint is added. This constraint ensures that the shrinkage occurs inward along the normal vector direction, preventing shape distortion. Specifically:

[0186] ;

[0187] Among them, E - c represents the contraction constraint term. X 0 represents the original point cloud coordinates (i.e., the original main point cloud P of the plant), and W represents the weight matrix;

[0188] Based on the contraction energy function E - l and contraction constraint term E - c, construct the complete energy function E as follows:

[0189] ;

[0190] in, α and β The hyperparameter representing the weights of the two energy terms;

[0191] By solving the linear system of the following equation, the position of each point cloud after shrinkage can be obtained: ;

[0192] After multiple iterations, the original dense point cloud P of the plant body is shrunk into a sparse and disordered skeleton point set S that represents its topological structure.

[0193] For the obtained skeleton point set S, the minimum spanning tree algorithm is used to construct the initial connectivity graph, and then the connectivity graph is optimized according to geometric rules to obtain the topological skeleton. .

[0194] S52. Treat the topological framework as a graph and use a graphical method to extract the main stem path of the plant from the topological framework. Specifically, treat the topological framework as a graph. S - raw Consider the graph G(V,E), where V represents the set of nodes and E represents the set of edges. The main stem corresponds to the longest node in graph G(V,E) that is closest to the root node (defined as the skeleton node closest to the lowest point in the point cloud). The shortest path to a certain terminal leaf node; the main stem path is obtained by solving the following formula:

[0195] ;

[0196] in, Indicates the main stem path. Represents the topological skeleton from the root node The set of paths to all leaf nodes. Let P represent the weight of edge e, and let P represent the point cloud in the topological skeleton. In this embodiment of the invention, the weights... Defined as a function of the point cloud density of the line segment represented by edge e in graph G(V,E), to obtain a thicker and more continuous main stem path.

[0197] S53. From the topological skeleton Subtract the main stem path All nodes and edges on the graph, and the remaining part is the branch skeleton.

[0198] S54. Based on the obtained main stem path and branching framework, obtain the phenotypic data of the plant framework. The phenotypic data of the plant framework include plant height, stem diameter, number of main stem nodes, number of branches, and branching angle. The corresponding process for obtaining the phenotypic data of the plant framework includes:

[0199] Plant height: Calculate the height of the main body of the plant based on the main stem path, specifically the height along the main stem path. Top of the node With the root node The Euclidean distance between them is the plant height. H ,Right now ;

[0200] Stem thickness: A series of N normal cutting planes are generated along the main stem path. For each normal cutting plane, the RANSAC algorithm is used to fit a circle to each normal cutting plane. The diameter of each fitted circle is... This refers to the stem diameter of the plant body at the corresponding normal cutting plane; in this embodiment of the invention, the average of all diameters is taken. The thickness of the stem of the entire plant is called the stem diameter. ;

[0201] Number of main stem nodes: Each connection point between the branch framework and the main stem framework is defined as a node, and the total number of nodes is the number of main stem nodes of the plant.

[0202] Branch count and branching angle: Among the branch skeletons connected to the main stem skeleton, those with a length greater than a set branch length threshold are defined as valid branches. The number of valid branches is the number of branches in the main body of the plant. For each valid branch, a segment of the skeleton vector near the connection point with the main stem skeleton is taken. And take a segment of the skeleton vector of the main stem skeleton at the connection point. Calculate the branch angle between the two skeleton vectors. Calculate the average value of all branch angles to obtain the overall opening degree of the plant body.

[0203] The key technical points of this invention are: 1. Three-dimensional point cloud joint annotation and standardized preprocessing technology: Designing structured binary encoding rules to associate and encode the two core traits of pod number and pod bursting state, realizing the representation of multiple traits by a single label; Combining the "rectangle selection + manual fine-tuning" annotation strategy and quantitative quality verification indicators (IoU≥90%, pod number accuracy≥95%), the problem of fragmented multi-trait information and uncontrollable annotation accuracy in traditional annotation is solved; At the same time, by adopting "pod region priority" resampling, spatial normalization (coordinate mapping to [-1,1]) and tensor format encapsulation, a standardized point cloud input dataset adapted to deep learning is constructed, eliminating scale interference caused by differences in scanning distance and plant size; 2. By improving SoftGroup The multi-task learning model architecture integrates a pod-aware attention backbone network, a SoftGroup grouping module, a feature aggregation module, a multi-task learning head, and a PCA refinement module. The pod-aware attention backbone network combines SetAbstraction and CBAM architectures to enhance the feature response of the pod region and suppress background interference. The multi-task learning head achieves collaborative inference of pod state classification and pod geometric parameter regression through parallel geometric parameter regression head and pod classification head. The PCA refinement module calibrates and optimizes the regression parameters based on principal component analysis, and the weighted multi-task loss function achieves balanced convergence of each task. Finally, an end-to-end automated examination technology chain is constructed to achieve efficient processing of the entire process from 3D point cloud acquisition to examination result output. Through a complete technical chain (without the need for manual intervention in intermediate links), the direct output of soybean plant three-dimensional point cloud to evaluation indicators (number of pods, pod bursting state, pod length / width / thickness) is realized, thus constructing an automated and high-precision evaluation technology system; 3. This invention combines the PC-Skeletor algorithm with the graphical method to shrink the point cloud of the plant's main stem, obtaining a thicker and more continuous main stem path, laying a stable data foundation for the subsequent accurate acquisition of plant traits.

[0204] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0205] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds, characterized in that: Specifically, the steps include the following: S1. A joint annotation method for soybean pod status based on 3D point cloud was adopted to complete the acquisition, annotation and standardization preprocessing of soybean plant 3D point cloud, and the data was divided into training set and test set; the annotation included: S14. Set structured binary encoding rules, and annotate the number of pods and the pod breakage status according to the structured binary encoding rules. After cleaning redundant or erroneous annotations, the annotation quality was verified by the accuracy of instance segmentation IoU, number of pods and pod breakage status annotation; S2. Construct an improved SoftGroup multi-task learning model, including a pod-perception attention backbone network, a SoftGroup grouping module, a feature aggregation module, a multi-task learning head, and a PCA refinement module; each module works collaboratively according to a preset logic, taking a single soybean 3D point cloud as input, and outputting the target task result through point cloud feature extraction and instance clustering; The pod-sensing attention backbone network performs downsampling and feature upscaling on the input 3D point cloud of a single soybean plant through a 3-layer Set Abstraction module. After each Set Abstraction module outputs features, a convolutional block attention module is embedded. First, the channel attention submodule performs average pooling and max pooling on the intermediate point cloud feature map, and calculates the channel weights through a shared MLP. Then, the spatial attention submodule performs spatial pooling and spatial weight calculation on the feature map. The dual weights are multiplied element-wise and normalized by the sigmoid activation function, and then the original features are weighted and fused to enhance the feature response of the pod region. Finally, the 3-layer Feature Abstraction module performs upsampling on the encoded high-dimensional features, and combines skip connections to supplement shallow spatial details, gradually restoring the original spatial resolution of the point cloud. S3. Based on the training set and test set divided in step S1, perform multi-task joint training on the improved SoftGroup multi-task learning model, configure the loss function weight coefficients and training hyperparameters, and set an early stopping strategy. S4. The 3D point cloud of the soybean plant to be tested, after preprocessing in step S1, is input into the trained model. Multi-task inference is performed, and geometric parameters are corrected through PCA refinement. The number of pods is counted, and the test results are finally output. The multi-task inference includes: inputting the preprocessed point cloud into the model and simultaneously outputting the pod instance mask, the probability of pod bursting state, and the original geometric parameters. The test results include the instance ID, instance mask, pod bursting state, number of pods, and parameters of pod length, pod width, and pod thickness for each pod. S5. Based on the pod identification results obtained in step S4, remove the pod point cloud from the three-dimensional point cloud of a single soybean plant to obtain the main point cloud of the plant; extract the plant skeleton from the main point cloud of the plant and obtain the phenotypic data of the plant skeleton.

2. The method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds according to claim 1, characterized in that: Step S1 further includes the following sub-steps: S11. Perform multi-view scanning on the target soybean plant, simultaneously acquire depth and color images from each view, and place markers on the soybean plant for spatial reference. S12. Spatial alignment of depth image data, unifying all depth image data to the same global coordinate system; S13. Based on the registered depth image data and color image data, an initial color three-dimensional point cloud of soybean plants is generated. The initial color three-dimensional point cloud is then subjected to post-processing of noise removal, data simplification and surface smoothing to obtain an optimized three-dimensional point cloud model. S15. Point cloud standardization: The labeled point cloud data that has passed the S14 verification is resampled and spatially normalized, and the coordinates are mapped to the [-1,1] interval; S16. Data partitioning: Divide the standardized point cloud into training and testing sets proportionally to maintain a balanced ratio of popped and non-popped points and a balanced distribution of point numbers.

3. The method for analyzing the whole soybean plant traits based on three-dimensional point clouds according to claim 2, characterized in that: In step S11, a three-dimensional scanning device is used to scan the target soybean plant from multiple preset scanning stations. At each scanning station, a depth image and a color image from that perspective are acquired simultaneously. Four to six marker points are set on a single soybean plant. The marker points are respectively pasted on the base, different branches in the middle and the top main stem of the soybean plant, and the marker points do not cover the pod area. Step S12 specifically includes the following sub-steps: S121. Coarse registration: Based on marker points and geometric feature points extracted from depth images or appearance feature points extracted from color images, feature matching is performed between different images, and the rigid body transformation matrix between adjacent viewpoint images is estimated to achieve preliminary alignment. S122. Fine registration and global optimization: The iterative nearest point algorithm is used to refine the coarse registration results; the bundle adjustment method is used to globally optimize the rigid body transformation matrix of all scanning stations to minimize the cumulative error, and finally unify all depth images to the same global coordinate system. Step S13 specifically includes the following sub-steps: S131. For each registered depth image, the pixel coordinates are calculated based on the camera intrinsic parameters using the principle of coordinate back projection. u , v ) and the depth value is converted to a 3D point in the global coordinate system. X , Y , Z The data is then processed and fused with the (R,G,B) color information of the corresponding color image to synthesize a complete initial color 3D point cloud. S132. For the reconstructed initial color 3D point cloud, apply statistical filtering or radius filtering to remove outlier noise points; use voxelized mesh downsampling method to simplify the data while maintaining the model shape; use moving least squares method to smooth the point cloud surface to obtain the 3D point cloud data of soybean plants.

4. The method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds according to claim 2, characterized in that: Step S14 further includes constructing a correspondence table according to structured binary encoding rules; the correspondence table contains association information of decimal labels, binary representations, number of grains, and pod bursting status; The encoding rule is as follows: the least significant bit of the binary number represents the pod-exploding state, 0 = no pod-exploding, 1 = pod-exploding; the index of the position of 1 in the second and above bits from the right represents the number of pods. Based on the encoding rules and the correspondence table, the optimized 3D point cloud model is segmented and labeled to distinguish between three semantic categories: popped pods, non-popped pods, and background. The number of grains in each pod is labeled, and a binary code conforming to the encoding rules is assigned to each pod instance. Step S15 specifically includes the following sub-steps: S151. Resampling: Set the number of target points N; use a random sampling algorithm to unify the point cloud scale. When the number of points exceeds N, downsampling is performed. When the number of points is less than N, the original data is retained. The sampling process is synchronously associated with spatial coordinates, normal vectors and label data. S152. Normalization: Calculate the geometric center point of the point cloud and translate the entire point cloud to a reference system with the center point as the origin; then, through proportional scaling, map all coordinates to the [-1,1] interval to eliminate scale differences. S153. Format Conversion and Storage: Convert the processed spatial coordinates, normal vectors, semantic tags, and instance tags into tensor format, combine and encapsulate them, and then serialize them into binary files for storage, with each plant as a unit.

5. The method for analyzing the whole soybean plant traits based on three-dimensional point clouds according to claim 1, characterized in that: The SoftGroup grouping module first obtains feature association information through dual-branch prediction, then adopts the Top-downGrouping clustering strategy, combined with the feature association information, to aggregate spatially adjacent points with consistent semantic categories into preliminary pod instance clusters; finally, it uses ScoreNet to score the confidence of the preliminary instance clusters, filters high-confidence instance clusters and removes redundant instances, and outputs accurate pod instance proposals. The feature aggregation module takes each pod instance proposal point set output by the SoftGroup grouping module as input, and maps the instance point set features through an MLP network. Each layer uses the ReLU activation function to introduce non-linearity, and uniformly transforms the high-dimensional features of instance point sets of different sizes into 32-dimensional fixed-dimensional instance feature vectors, providing standardized input for the multi-task learning head. The multi-task learning head includes a parallel geometric parameter regression head and a pod classification head. The geometric parameter regression head first extracts features from the instance feature vector using an MLP[128,64,32], and then outputs the original geometric parameters through a fully connected layer. The pod classification head first strengthens the classification features using an MLP[128,64,32], then performs feature mapping through a fully connected layer, and finally completes probability normalization through a Softmax layer, outputting the "non-pod" and "pod" class probabilities for each pod instance. The PCA refinement module performs calibration and optimization on the original geometric parameters; and performs geometric feature decomposition and precise calibration on the point cloud of the pod instance based on principal component analysis.

6. The method for analyzing the whole soybean plant traits based on three-dimensional point clouds according to claim 5, characterized in that: In the pod-perception attention backbone network, each Set Abstraction module first filters key sampling points by sampling the farthest point, then divides the local neighborhood point set by ball query with the sampling point as the center, and finally performs feature mapping on the neighborhood point set through MLP to improve the feature dimension. The SoftGroup grouping module has two branches: a semantic prediction branch and an offset prediction branch. The semantic prediction branch is used to output the semantic scores of each point in the three-dimensional point cloud of a single soybean plant, corresponding to the three categories of "non-exploded pod", "exploded pod", and "background". The offset prediction branch is used to output the X / Y / Z offset of each point relative to the center of its respective pod instance, so as to characterize the positional relationship between the point cloud and the center of the corresponding pod instance in three-dimensional space. In the feature aggregation module, the MLP network dimension is set to [128, 64, 32]; The original geometric parameters output by the geometric parameter regression head are K×3, where K is the number of instances and 3 corresponds to pod length, aspect ratio, and pod thickness. The processing flow of the PCA refinement module is as follows: first, extract the point cloud of a single pod instance output by the SoftGroup grouping module, calculate the mean coordinate of the point cloud and translate it to a local coordinate system with the mean as the origin; A 3×3 covariance matrix is ​​then constructed and its eigenvalues ​​are decomposed to obtain three eigenvalues ​​and their corresponding eigenvectors. The correlation between the eigenvalues ​​and the actual size of the pod is used to calculate the refined size along the principal axis. The refined size is then fused with the original geometric parameters output by the multi-task learning head according to preset weights to output the final accurate 3D size of the pod.

7. The method for analyzing the whole-plant traits of soybean based on three-dimensional point clouds according to claim 1, characterized in that: In step S3, the weight coefficients α, β, γ, δ of the loss function are set. The values ​​range from (0,1); The multi-task loss function is a weighted sum of the losses of each sub-task, and the sub-task loss includes the pod classification loss. L cls Geometric parameter regression loss L reg Semantic prediction loss L sem Instance rating loss L score and group loss L group The formula is as follows: ； Pod Explosion Classification Loss L cls The Focal Loss function is used to address the class imbalance problem in pod classification tasks and improve the classification accuracy of the minority classes; the expression is: ； In the formula, It is the probability that the model predicts the true class. γ It is an adjustable focusing parameter. It is a weighting factor used to balance the number of positive and negative samples; Geometric parameter regression loss L reg The Smooth L1 Loss function is used to optimize the regression accuracy of geometric parameters and reduce the impact of outliers on loss calculation; the expression is: ； In the formula, and These are the true value and the predicted value of the i-th sample, respectively. β It is a threshold parameter.

8. The method for analyzing the whole soybean plant traits based on three-dimensional point clouds according to claim 1, characterized in that: In step S5, the plant skeleton is extracted from the main point cloud of the plant, and the phenotypic data of the plant skeleton are obtained, specifically including: The PC-Skeletor algorithm was applied to the main point cloud of the plant to generate the original topological skeleton. Treating the topological framework as a graph, a graphical method is used to extract the main stem path of the plant from the topological framework. Subtract all nodes and edges on the main stem path from the topological skeleton, and the remaining part is the branch skeleton; Calculate the height of the main plant body based on the main stem path; A series of normal cutting planes are generated along the main stem path. For each normal cutting plane, the RANSAC algorithm is used to fit each normal cutting plane to a circle. The diameter of each fitted circle is the stem thickness of the plant body at the corresponding normal cutting plane. Each connection point between the branch skeleton and the main stem skeleton is defined as a node, and the total number of nodes is the number of main stem nodes of the plant. In the branch skeleton connected to the main stem skeleton, the branch skeleton with a length greater than the set branch length threshold is defined as a valid branch, and the number of valid branches is the number of branches of the main body of the plant. For each effective branch, take a segment of the skeleton vector near the connection point of the main stem skeleton, and take a segment of the skeleton vector of the main stem skeleton at that connection point. Calculate the branching angle between the two skeleton vectors, calculate the average value of all branching angles, and obtain the overall opening degree of the plant body.

9. The method for analyzing the whole soybean plant traits based on three-dimensional point clouds according to claim 8, characterized in that: The main stem path of the plant body is extracted from the topological skeleton using a graphical method, including: The main stem path is obtained by solving the following formula: ； in, Indicates the main stem path. Represents the topological skeleton from the root node The set of paths to all leaf nodes. Let e ​​represent the weight of edge e, and P represent the point cloud in the topological skeleton.

Images