Mobile phenotypic information collection platform and method based on the entire plant growth cycle

Abstract

This invention discloses a mobile phenotypic information acquisition platform and method based on the entire plant growth cycle, belonging to the field of intelligent plant phenotypic detection technology. It includes acquiring image sequences and depth point cloud data of multiple key growth stages of a target plant, and constructing a multi-dimensional phenotypic feature tensor representing the joint information of the plant's three-dimensional morphological structure and color texture through spatiotemporal registration and fusion. The tensor is input into a learnable phenotypic parsing network, and a dynamic phenotypic evolution map is generated by iteratively enhancing the plant organ feature response and suppressing the background feature response. Key phenotypic trait parameter sequences from budding to maturity are extracted, and the growth trend degree and developmental stability scores of each sequence are calculated, sorted, and integrated to form a hierarchical full-cycle phenotypic atlas. This method can achieve multi-dimensional phenotypic information fusion representation, weaken background interference, accurately depict the dynamic evolution process of phenotypic changes, and clearly present the correlation between phenotypic states at each growth stage.

Description

Technical Field

[0001] This invention belongs to the field of intelligent plant phenotypic detection technology, specifically a mobile phenotypic information collection platform and method based on the entire plant growth cycle. Background Technology

[0002] Current plant phenotypic information collection mostly focuses on a single growth stage, with two-dimensional images as the primary data carrier. Depth point cloud data and image data are largely processed independently, without spatiotemporal matching and integration. The models used in phenotypic analysis do not differentiate between plant organ features and background features, allowing background interference to directly affect phenotypic feature extraction, leading to biases in the results. After collecting phenotypic parameters throughout the plant's life cycle, quantitative analysis of the growth trends and developmental stability of these parameters is lacking. Parameter selection lacks corresponding quantitative evaluation criteria, failing to intuitively present the evolutionary relationships between phenotypic states at different growth stages. Conventional collection methods can only obtain discrete phenotypic data; the plant's three-dimensional spatial morphology and color texture information cannot form a joint representation, limiting the depiction of dynamic phenotypic changes and failing to fully reconstruct the entire phenotypic development process from budding to maturity.

[0003] Images and depth point cloud data from multiple plant growth stages cannot be spatiotemporally registered and fused, making it difficult to construct joint phenotypic features that integrate 3D morphology and color texture. During phenotypic analysis, plant organ features are easily obscured by background features, making it impossible to generate accurate dynamic evolution features of plant phenotypic characteristics. The lack of quantitative screening methods for growth trend and developmental stability of key phenotypic traits throughout the entire life cycle makes it impossible to construct a hierarchical full-cycle phenotypic atlas that reflects the evolutionary relationship of phenotypic states. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art;

[0005] Therefore, this invention proposes a mobile phenotypic information collection method based on the entire plant growth cycle, including:

[0006] Acquire image sequences and depth point cloud data of the target plant at multiple key growth stages;

[0007] The image sequence and the depth point cloud data are spatiotemporally registered and fused to construct a multidimensional phenotypic feature tensor, which is used to characterize the joint information of the plant's morphological structure and color texture in three-dimensional space.

[0008] The multidimensional phenotypic feature tensor is input into a learnable phenotypic parsing network, which generates a dynamic evolution map of plant phenotypic features by iteratively enhancing feature responses related to plant organs and suppressing background feature responses.

[0009] Based on the plant phenotypic dynamic evolution diagram, the sequences of all key phenotypic trait parameters from the germination stage to the maturity stage were extracted;

[0010] For each of the key phenotypic trait parameter sequences, calculate its growth trend degree and developmental stability score, and sort and filter the key phenotypic trait parameter sequences according to the growth trend degree and the developmental stability score;

[0011] The sequence of key phenotypic parameters after sorting and screening is integrated to form a hierarchical full-cycle phenotypic map. In the hierarchical full-cycle phenotypic map, each node represents the phenotypic state at a specific growth time, and each edge represents the evolutionary relationship between phenotypic states.

[0012] Furthermore, the image sequence and the depth point cloud data are spatiotemporally registered and fused to construct a multi-dimensional phenotypic feature tensor, including:

[0013] Feature point detection and matching are performed on the image sequence to estimate the spatial transformation relationship between images at different time points, thereby achieving temporal alignment of the image sequence;

[0014] Perform point cloud registration on the depth point cloud data to unify the point clouds collected at different times into the same world coordinate system;

[0015] In a unified spatiotemporal coordinate system, the pixel color information of the image sequence that has been time-aligned is mapped to the corresponding three-dimensional points of the depth point cloud data that has been registered with the point cloud, thereby generating a dense three-dimensional point cloud with true color information.

[0016] From the dense 3D point cloud with true color information, downsampling and feature extraction are performed according to a preset spatial voxel grid. The features include the color histogram, normal vector distribution and point density within the voxel.

[0017] All extracted voxel features are arranged according to their spatial positions to form a four-dimensional tensor, which is the multi-dimensional phenotypic feature tensor, and its four dimensions represent the three-dimensional spatial coordinates and feature channels, respectively.

[0018] Furthermore, the multi-dimensional phenotypic feature tensor is input into a learnable phenotypic parsing network. This network iteratively enhances feature responses related to plant organs and suppresses background feature responses to generate a dynamic evolution map of plant phenotypic characteristics, including:

[0019] The multidimensional phenotypic feature tensor is input into a three-dimensional convolutional encoder, which extracts plant structural features with multi-scale spatial receptive fields.

[0020] The encoded features are input into a feature refinement module based on a self-attention mechanism. The feature refinement module calculates the correlation between features at different spatial locations, enhances the feature response related to plant organs, and weakens the feature response related to soil and pot background.

[0021] The refined features are input into a three-dimensional convolutional decoder, which progressively upsamples and reconstructs the feature map, outputting a high-resolution plant phenotypic semantic segmentation voxel field.

[0022] In the plant phenotypic semantic segmentation voxel field, three-dimensional instances of each semantic category are extracted at different time points, and the states of the same instance at different time points are connected by directed edges to form the plant phenotypic dynamic evolution graph. The nodes in the plant phenotypic dynamic evolution graph have timestamps and three-dimensional spatial occupancy information.

[0023] Furthermore, based on the aforementioned plant phenotypic dynamic evolution diagram, the sequences of all key phenotypic trait parameters from germination to maturity were extracted, including:

[0024] Traverse each plant organ instance node in the plant phenotypic dynamic evolution diagram, where the plant organ instance nodes include roots, stems, leaves, flowers, and fruits;

[0025] For each plant organ instance node, its phenotypic parameters are calculated from the three-dimensional spatial occupancy information of the plant organ instance node at different time points. The phenotypic parameters include three-dimensional volume, surface area, skeleton length, spatial orientation, and color mean.

[0026] Arrange the values ​​of the same phenotypic trait parameter calculated at different time points for the same plant organ instance node in chronological order to form a sequence, thus forming the phenotypic parameter sequence of the plant organ instance node.

[0027] The sequence of all phenotypic parameters of all plant organ instance nodes is summarized to form the set of key phenotypic phenotypic parameter sequences.

[0028] Furthermore, for each of the key phenotypic trait parameter sequences, a growth trend score and a developmental stability score are calculated, including:

[0029] For a sequence of key phenotypic trait parameters, a trend line is fitted using the least squares method;

[0030] The determination coefficients of the actual observed values ​​of the key phenotypic trait parameter sequence and their corresponding predicted values ​​on the trend line are calculated. The normalized determination coefficients are used as the growth trend degree, which reflects the degree of clarity of the direction of trait development.

[0031] The standard deviation of the difference between observations at adjacent time points in the sequence of key phenotypic trait parameters is calculated, and the reciprocal of the standard deviation is normalized to obtain the developmental stability score, which reflects the degree of fluctuation in the trait development rate.

[0032] Furthermore, the integration of the sorted and filtered key phenotypic parameter sequences to form a hierarchical full-cycle phenotypic map includes:

[0033] The trait parameter sequences in which both the growth trend score and the developmental stability score are higher than a preset threshold are selected and marked as stable key trait sequences;

[0034] Using growth stages as the first level, the stable key trait sequences are classified into germination stage, vegetative growth stage, flowering stage, fruiting stage, and maturity stage;

[0035] At each growth stage level, the trait sequence is classified into root, stem, leaf, flower, and fruit, with plant organ type as the second level.

[0036] At each plant organ type level, the sequence is classified into volume, surface area, length, orientation, and color as the third level, based on specific phenotypic trait measures.

[0037] By combining the three-level structure of growth stage, plant organ type, and phenotypic trait measurement with the temporal evolution edges between sequences, a hierarchical full-cycle phenotypic map with a tree-like network structure is constructed.

[0038] Furthermore, a point cloud registration operation is performed on the depth point cloud data to unify the point clouds collected at different times into the same world coordinate system, including:

[0039] Select a depth point cloud acquired at a fixed time as the target point cloud, and set its corresponding spatial coordinate system as the world coordinate system;

[0040] For each frame of source point cloud acquired at a non-fixed time, extract its fast point feature histogram features with those in the target point cloud;

[0041] Based on the extracted fast point feature histogram features, the initial rigid body transformation matrix from the source point cloud to the target point cloud is calculated using the sample consistency initial registration algorithm;

[0042] Based on the initial rigid body transformation, an iterative nearest point algorithm is used for fine registration. By minimizing the distance error between corresponding points, the optimal rigid body transformation matrix from the source point cloud to the world coordinate system is obtained.

[0043] Based on the optimal rigid body transformation matrix, a spatial transformation is performed on each three-dimensional point in the source point cloud so that it is aligned with the target point cloud to the world coordinate system.

[0044] Furthermore, the encoded features are input into a feature refinement module based on a self-attention mechanism, including:

[0045] The encoded feature tensor output by the three-dimensional convolutional encoder is flattened in spatial dimension to obtain a series of feature vectors;

[0046] Calculate the query vector, key vector, and value vector for each feature vector;

[0047] The attention weight matrix is ​​obtained by calculating the dot product of the query vector and the key vector of all feature vectors, and then the attention weight matrix is ​​normalized by applying the Softmax function.

[0048] Multiply the normalized attention weight matrix by the value vector of all feature vectors to obtain a refined feature vector that incorporates global context information;

[0049] The refined feature vector is reshaped to have the same spatial dimension as the encoded feature tensor, and residually connected with the original encoded features to output the final refined features.

[0050] Furthermore, the method also includes autonomous path planning and data supplementation for the data acquisition platform:

[0051] In the plant phenotypic dynamic evolution map, plant organ regions or time points with three-dimensional reconstruction quality below a preset threshold are identified;

[0052] Based on the spatial location and time point of the identified low-quality areas, calculate the data types, optimal acquisition pose, and acquisition time that the mobile acquisition platform needs to acquire.

[0053] Generate a list of supplementary sampling task instructions, which includes the platform movement path, robotic arm adjustment parameters, and sensor triggering parameters.

[0054] The mobile acquisition platform is controlled to automatically acquire new data according to the list of acquisition task instructions, and the newly acquired data is sent to the image sequence and the depth point cloud data for iterative optimization.

[0055] Furthermore, the present invention also includes a mobile phenotypic information collection platform based on the entire plant growth cycle. The platform includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the mobile phenotypic information collection method based on the entire plant growth cycle described above.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] Spatiotemporal registration and fusion processing of image sequences and depth point cloud data from multiple key plant growth stages were performed to construct a multidimensional phenotypic feature tensor for characterizing the joint information of plant three-dimensional spatial morphological structure and color texture. The spatial structural features and surface texture and color features of plant phenotypes were synchronously integrated, the spatiotemporal deviation of data collected at different growth stages was eliminated, the representation dimension of phenotypic features was expanded in three dimensions, the problem of one-sided phenotypic information caused by single data types was improved, and the phenotypic features of each growth stage of plants could be fully presented in the form of a joint tensor. The completeness and correlation of phenotypic features were continuously optimized.

[0058] By inputting multi-dimensional phenotypic feature tensors into a learnable phenotypic parsing network, a dynamic evolution map of plant phenotypic characteristics is generated through iterative enhancement of feature responses related to plant organs and suppression of background feature responses. Feature interference from environmental background is continuously weakened, and the salience of phenotypic features of plant organs is continuously improved. The dynamic phenotypic changes of plants from germination to maturity are clearly depicted. The generation of the dynamic evolution map of phenotypic characteristics closely matches the actual growth and development laws of plants. The extraction of key phenotypic trait parameter sequences can accurately match the phenotypic development status of plants at each stage. The calculation of growth trend degree and developmental stability score relies on accurate parameter sequences. The nodes and evolutionary relationships of the hierarchical full-cycle phenotypic map are constructed to better fit the actual change logic of plant phenotypic characteristics. The selection and sorting of phenotypic parameters are in line with the inherent laws of plant growth and development. Attached Figure Description

[0059] Figure 1 This is a fishbone diagram of the mobile phenotypic information collection method based on the entire plant growth cycle described in this invention.

[0060] Figure 2 A flowchart for constructing a multidimensional phenotypic feature tensor;

[0061] Figure 3 A flowchart for generating a dynamic evolution diagram of plant phenotypes. Detailed Implementation

[0062] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] This invention provides a mobile phenotypic information acquisition method based on the entire plant growth cycle. The overall implementation scheme is as follows: The method begins by acquiring image sequences and depth point cloud data of the target plant at multiple key growth stages. Subsequently, the image sequences and depth point cloud data are spatiotemporally registered and fused. This step aims to align and combine data from different times and different sensor sources to construct a multi-dimensional phenotypic feature tensor. This tensor effectively represents the joint information of the plant's morphological structure and color texture in three-dimensional space. The constructed multi-dimensional phenotypic feature tensor is input into a learnable phenotypic parsing network. This network iteratively enhances feature responses related to plant organs and suppresses background feature responses through its internal mechanisms, thereby generating a dynamic evolution map of plant phenotypic changes over time. Based on this dynamic evolution map, all key phenotypic trait parameter sequences from budding to maturity can be systematically extracted. Next, the growth trend degree and developmental stability score are calculated for each key phenotypic trait parameter sequence, and the key phenotypic trait parameter sequences are sorted and filtered based on these two quantitative indicators. Finally, the key phenotypic parameter sequences that have been sorted and filtered are integrated to form a hierarchical full-cycle phenotypic map. In this map, each node represents the phenotypic state at a specific growth moment, and each edge represents the temporal evolution relationship between phenotypic states.

[0064] In one embodiment of the present invention, see [reference] Figure 2This study performs spatiotemporal registration and fusion of image sequences and depth point cloud data to construct a multi-dimensional phenotypic feature tensor, a process involving multiple steps. First, feature point detection and matching are performed on the image sequences. Temporal alignment of the image sequences is achieved by estimating the spatial transformation relationship between images at different time points. Next, point cloud registration is performed on the depth point cloud data. A depth point cloud acquired at a fixed time is selected as the target point cloud, and its corresponding spatial coordinate system is set as the world coordinate system. For each frame of source point cloud acquired at a non-fixed time, fast point feature histogram features are extracted from both the source and target point clouds. Based on the extracted fast point feature histogram features, the initial rigid body transformation matrix from the source point cloud to the target point cloud is calculated using a sample consistency initial registration algorithm. Based on this initial rigid body transformation, an iterative nearest-point algorithm is used for fine registration. By minimizing the distance error between corresponding points, the optimal rigid body transformation matrix from the source point cloud to the world coordinate system is obtained. According to the optimal rigid body transformation matrix, each 3D point in the source point cloud undergoes a spatial transformation to align it with the target point cloud to the world coordinate system. In a unified spatiotemporal coordinate system, the pixel color information of a temporally aligned image sequence is mapped onto corresponding 3D points in a point cloud data registered with the point cloud, generating a dense 3D point cloud with true-color information. From this dense 3D point cloud, sampling and feature extraction are performed according to a predefined spatial voxel grid. The extracted features include the color histogram, normal vector distribution, and point density within each voxel. All extracted voxel features are arranged according to their spatial positions to form a four-dimensional tensor, which is the multi-dimensional phenotypic feature tensor, with its four dimensions representing the spatial 3D coordinates and feature channels, respectively.

[0065] In practical implementation, the mobile phenotypic information acquisition method based on the entire plant growth cycle involves spatiotemporal registration and fusion of image sequences and depth point cloud data to construct a multi-dimensional phenotypic feature tensor. Temporal alignment of image sequences is achieved through feature point detection and matching. Feature point detection uses a scale-invariant feature transformation algorithm, and the matching process is based on the Euclidean distance of feature descriptors to estimate the spatial transformation relationship between images at different time points. This spatial transformation relationship is represented as an affine transformation matrix, thus achieving temporal alignment of the image sequences. In some embodiments, the point cloud registration operation of the depth point cloud data selects a depth point cloud acquired at a fixed time as the target point cloud, and sets the spatial coordinate system corresponding to the target point cloud as the world coordinate system. For each frame of source point cloud acquired at a non-fixed time, fast point feature histogram features are extracted from the source and target point clouds. Based on the extracted fast point feature histogram features, the initial rigid body transformation matrix from the source point cloud to the target point cloud is calculated using a sample consistency initial registration algorithm. Based on the initial rigid body transformation, the iterative nearest point algorithm is used for fine registration. The iterative nearest point algorithm optimizes the transformation matrix by minimizing the distance error between corresponding points. The distance error between corresponding points is defined as follows:

[0066]

[0067] in: Indicates the registration error. Indicates the number of corresponding point pairs. These are points in the source point cloud. It is the corresponding point in the target point cloud. It is a rotation matrix. It is a translation vector. By minimizing the registration error The optimal rigid body transformation matrix from the source point cloud to the world coordinate system is obtained through optimization. Based on the optimal rigid body transformation matrix, a spatial transformation is performed on each 3D point in the source point cloud so that the source point cloud and the target point cloud are aligned to the world coordinate system.

[0068] In practical implementation, within a unified spatiotemporal coordinate system, the pixel color information of time-aligned image sequences is mapped onto corresponding 3D points in point cloud data registered with point cloud standards, generating a dense 3D point cloud with true-color information. The mapping process is based on camera intrinsic and extrinsic parameters, associating image pixels with 3D points through back projection and assigning color values ​​to each 3D point. From the dense 3D point cloud with true-color information, sampling and feature extraction are performed according to a pre-defined spatial voxel grid. The spatial voxel grid divides the point cloud space into regular cubic units, calculating the color histogram, normal vector distribution, and point density within each voxel. The color histogram statistically analyzes the color distribution of all points within the voxel, the normal vector distribution calculates the point cloud normal vector and statistically analyzes its direction through principal component analysis, and the point density is the ratio of the number of points within the voxel to the voxel volume. Optionally, feature extraction also includes calculating other statistical features of points within the voxel, such as color mean and variance. All extracted voxel features are arranged according to their spatial location, forming a four-dimensional tensor. The four dimensions of the four-dimensional tensor represent the spatial 3D coordinates and feature channels, respectively. The three-dimensional spatial coordinates correspond to the voxel indices in the grid, and the feature channels correspond to features such as color histogram, normal vector distribution, and point density. It can be understood that the multi-dimensional phenotypic feature tensor effectively encodes the joint information of plant morphology, structure, color, and texture in three-dimensional space.

[0069] In some embodiments, data comparison evaluates registration quality by comparing the area of ​​the overlapping region between point clouds before and after registration. The area of ​​the overlapping region is calculated based on a distance threshold between point clouds. During feature extraction, the choice of voxel size affects the resolution and computational efficiency of the feature tensor. Smaller voxel sizes retain more details but increase computation, while larger voxel sizes reduce computation but may lose fine structure. By adjusting the voxel size, a balance can be achieved between accuracy and efficiency. Optionally, after constructing a multi-dimensional phenotypic feature tensor, the feature tensor can be normalized to eliminate differences between different feature scales. Normalization scales the values ​​of each feature channel to the same range, such as zero mean and unit variance. It is understood that normalization helps the subsequent training and convergence of the learnable phenotypic parsing network.

[0070] In one embodiment of the present invention, see [reference] Figure 3 The process involves inputting a multi-dimensional phenotypic feature tensor into a learnable phenotypic parsing network. This network iteratively enhances feature responses related to plant organs and suppresses background feature responses, generating a dynamic evolution map of plant phenotypic characteristics. First, the multi-dimensional phenotypic feature tensor is input into a 3D convolutional encoder, which extracts plant structural features with multi-scale spatial receptive fields. The encoded features are then input into a feature refinement module based on a self-attention mechanism. This module flattens the encoded feature tensor output from the 3D convolutional encoder in space, obtaining a series of feature vectors. For each feature vector, its query vector, key vector, and value vector are calculated. An attention weight matrix is ​​obtained by calculating the dot product of the query and key vectors of all feature vectors. This attention weight matrix is ​​then normalized using the Softmax function. The normalized attention weight matrix is ​​multiplied by the value vectors of all feature vectors to obtain refined feature vectors that incorporate global contextual information. These refined feature vectors are then reshaped to have the same spatial dimension as the encoded feature tensor and residually connected to the original encoded features, outputting the final refined features. The refined features are input into a 3D convolutional decoder, which progressively upsamples and reconstructs the feature map, outputting a high-resolution plant phenotypic semantic segmentation voxel field. Within this voxel field, 3D instances of each semantic category are extracted at different time points, and the states of the same instance at different time points are connected by directed edges to form a dynamic evolution graph of the plant phenotypic pattern. Nodes in this graph carry timestamps and 3D spatial occupancy information.

[0071] In specific implementation, a multi-dimensional phenotypic feature tensor is input into a learnable phenotypic parsing network to generate a dynamic evolution map of plant phenotypic characteristics. The learnable phenotypic parsing network first inputs the multi-dimensional phenotypic feature tensor into a 3D convolutional encoder, which consists of alternating stacks of multiple 3D convolutional layers, 3D batch normalization layers, and 3D max pooling layers. The 3D convolutional layers use 3x3x3 kernels to extract local features, the 3D batch normalization layers are used to stabilize the training process, and the 3D max pooling layers gradually expand the spatial receptive field of the features, enabling the 3D convolutional encoder to extract plant structural features with multi-scale spatial receptive fields. The encoded features have reduced spatial dimensions and increased feature channel dimensions. In some embodiments, the output of the 3D convolutional encoder is input into a feature refinement module based on a self-attention mechanism. The feature refinement module flattens the encoded feature tensor output by the 3D convolutional encoder in spatial dimensions, obtaining a series of feature vectors. For each feature vector, its query vector, key vector, and value vector are calculated. This calculation is implemented through three independent linear transformation layers, whose weight parameters are learned by the network. The attention weight matrix is ​​obtained by calculating the dot product of the query vector and the key vector of all feature vectors. This matrix reflects the correlation between features at different spatial locations. The attention weight matrix is ​​normalized using the Softmax function, ensuring that the sum of the attention weights for each feature location with respect to all other locations is 1. Multiplying the normalized attention weight matrix by the value vectors of all feature vectors yields a refined feature vector that incorporates global context information. The self-attention calculation process can be described as follows:

[0072]

[0073] in: This represents a matrix consisting of all query vectors. This represents a matrix consisting of all key vectors. This represents a matrix consisting of all value vectors. It is the dimension of the key vector. This is the set of refined feature vectors output. It can be understood that the self-attention mechanism enhances feature responses related to plant organs while weakening feature responses related to soil and pot background. The refined feature vectors are reshaped to have the same spatial dimension as the encoded feature tensor and residually connected to the original encoded features to output the final refined features.

[0074] In practice, the refined features are input into a 3D convolutional decoder. This decoder consists of multiple 3D transposed convolutional layers and 3D convolutional layers. The transposed convolutional layers progressively upsample the feature map to restore spatial resolution, while the 3D convolutional layers refine the upsampled features. The 3D convolutional decoder progressively upsamples and reconstructs the feature map, ultimately outputting a high-resolution plant phenotypic semantic segmentation voxel field. Each voxel in the plant phenotypic semantic segmentation voxel field is assigned a semantic category label, including root, stem, leaf, flower, fruit, and background. Within the plant phenotypic semantic segmentation voxel field, 3D instances of each semantic category are extracted at different time points. This extraction process is achieved through 3D connected component analysis, marking spatially connected voxels belonging to the same semantic category as the same instance. The states of the same instance at different time points are connected by directed edges, forming a dynamic evolution graph of the plant phenotypic structure. Nodes in this dynamic evolution graph carry timestamps and 3D spatial occupancy information, represented by the set of 3D coordinates of all voxels contained within the instance. In some embodiments, data comparison is performed by comparing the response differences between plant organ regions and background regions in the input and output feature maps of the feature refinement module. These response differences can be quantified by the ratio of average activation values ​​in the feature maps. Optionally, the last layer of the 3D convolutional decoder uses the Softmax activation function to generate a probability distribution for each voxel belonging to a specific semantic category. It can be understood that the dynamic evolution map of plant phenotypic characteristics structurally represents the morphological changes of plant organs in three-dimensional space over time.

[0075] In one embodiment of the present invention, all key phenotypic trait parameter sequences from germination to maturity are extracted based on a plant phenotypic dynamic evolution diagram. This extraction process traverses each plant organ instance node in the plant phenotypic dynamic evolution diagram, including roots, stems, leaves, flowers, and fruits. For each plant organ instance node, its phenotypic trait parameters are calculated from its three-dimensional spatial occupancy information at different time points. These parameters include three-dimensional volume, surface area, skeleton length, spatial orientation, and mean color. Following chronological order, the values ​​of the same phenotypic trait parameter calculated for the same plant organ instance node at different time points are arranged into a sequence, forming the trait parameter sequence for that plant organ instance node. Finally, all trait parameter sequences of all plant organ instance nodes are summarized to constitute a set of key phenotypic trait parameter sequences.

[0076] In specific implementation, the sequence of all key phenotypic parameters from germination to maturity is extracted based on the plant phenotypic dynamic evolution diagram. The extraction process traverses each plant organ instance node in the diagram. A plant organ instance node represents a three-dimensional instance of a specific plant organ at a specific time point, including roots, stems, leaves, flowers, and fruits. In some embodiments, the traversal process uses a depth-first search algorithm to access all nodes in the diagram, and each node is labeled with its corresponding plant organ type and timestamp. For each plant organ instance node, its phenotypic parameters are calculated from the three-dimensional spatial occupancy information at different time points. This three-dimensional spatial occupancy information is represented by the set of three-dimensional coordinates of all voxels constituting the instance. The calculated three-dimensional volume is obtained by counting the total number of voxels occupied by the plant organ instance node and multiplying it by the physical volume of a single voxel. The surface area is calculated by summing the areas of all exposed voxel surfaces of the plant organ instance node. The formula for calculating the surface area is:

[0077]

[0078] in: This represents the calculated surface area of ​​the plant organ instance node. This represents the set of all exposed surfaces of this instance. This represents the area of ​​a surface patch. The skeleton length is obtained by extracting the central axis using a refinement algorithm on the 3D voxel model of the plant organ instance node and calculating the total length of the central axis. The spatial orientation is calculated using principal component analysis to determine the covariance matrix of all voxel coordinates of the plant organ instance node, and the direction of the eigenvector corresponding to the largest eigenvalue is taken as the principal axis direction of that instance. The color mean is obtained by extracting the color values ​​(e.g., RGB values) associated with each voxel in the plant organ instance node and calculating the arithmetic mean of all color values.

[0079] In practice, the phenotypic parameter values ​​of the same plant organ instance node calculated at different time points are arranged into a sequence according to time order, forming a phenotypic parameter sequence for the plant organ instance node. The arrangement process is based on the timestamp information of the nodes in the plant phenotypic dynamic evolution diagram, sorting all timestamps according to the order of collection and filling the corresponding parameter values ​​into the sequence. Optionally, before arranging into a sequence, instances of the same organ at different time points can be associated and matched to ensure that the sequences originate from the same physical organ. The association matching is based on the spatial overlap and phenotypic similarity of instances at adjacent time points. All phenotypic parameter sequences of all plant organ instance nodes are summarized to form a key phenotypic parameter sequence set. The summarization process creates a data structure indexed by plant organ type and phenotypic parameter type, storing all corresponding time series data. It can be understood that the key phenotypic parameter sequence set systematically characterizes the dynamic changes of various traits in different plant organs throughout the entire growth cycle. Refer to Table 1, which shows the surface area parameter sequence of a leaf instance:

[0080] Table 1: Surface area parameter sequence of blade examples at different time points

[0081] Time point (day) Surface area (square centimeters) 1 5.2 4 12.7 7 24.3 10 41.5 13 60.8

[0082] In some embodiments, data comparison can be performed by comparing morphological differences in the same trait parameter sequences between different organs or different plants, such as comparing the slopes of growth curves for the surface area sequences of two leaves. Optionally, when calculating the color mean, the color space can be converted from RGB to other color spaces, such as HSV, to better separate brightness and color information. In skeleton length calculation, the thinning algorithm needs to handle the topological structure of the three-dimensional voxel model to ensure that the extracted central axis is single-pixel wide and maintains connectivity. It can be understood that directly calculating phenotypic trait parameters from three-dimensional spatial occupancy information can obtain more accurate and three-dimensionally meaningful trait data than traditional two-dimensional image analysis.

[0083] In one embodiment of the present invention, the growth trend degree and developmental stability score are calculated for each key phenotypic trait parameter sequence. For a key phenotypic trait parameter sequence, a trend line is fitted using the least squares method. The determination coefficient between the actual observed value of the key phenotypic trait parameter sequence and the predicted value on the corresponding trend line is calculated. This determination coefficient is normalized and used as the growth trend degree, which reflects the clarity of the trait development direction. The standard deviation of the difference between observed values ​​at adjacent time points in the key phenotypic trait parameter sequence is calculated. The reciprocal of this standard deviation is normalized to obtain the developmental stability score, which reflects the degree of fluctuation in the trait development rate. The sorted and screened key phenotypic trait parameter sequences are integrated to form a hierarchical full-cycle phenotypic map. Trait parameter sequences with both growth trend degree and developmental stability score higher than a preset threshold are selected and marked as stable key trait sequences. Taking the growth stage as the first level, the stable key trait sequences are classified into budding stage, vegetative growth stage, flowering stage, fruiting stage, and maturity stage. Within each growth stage level, plant organ type is used as the second level, categorizing trait sequences into roots, stems, leaves, flowers, and fruits. Within each plant organ type level, specific phenotypic trait measures are used as the third level, categorizing sequences into volume, surface area, length, orientation, and color. By combining this three-tiered structure of growth stage, plant organ type, and phenotypic trait measures with temporal evolution edges between sequences, a hierarchical, full-cycle phenotypic map with a tree-like network structure is constructed.

[0084] In practice, the growth trend and developmental stability scores are calculated for each key phenotypic parameter sequence. A key phenotypic parameter sequence is represented as a set of observations arranged in chronological order. The corresponding set of time points is For a sequence of key phenotypic parameters, a trend line is fitted using the least squares method. The trend line model typically employs a linear model. ,in and The model parameters are obtained by minimizing the sum of squared residuals. At a certain point in time The predicted values. Calculate the coefficient of determination between the actual observed values ​​of the key phenotypic trait parameter sequence and the corresponding predicted values ​​on the trend line. The calculation formula is:

[0085]

[0086] in: These are actual observed values. It is the trend line prediction value. It is the average of the actual observed values. It is the sequence length. The coefficient of determination... After normalization, it is used as the growth trend degree. The normalization process directly converts... The value is used as a measure of growth trend because The value ranges from 0 to 1. Growth trend reflects the clarity of the developmental direction of the trait. The standard deviation of the difference between observations at adjacent time points in the key phenotypic trait parameter sequence is calculated; the sequence of differences between observations at adjacent time points is... Calculate the standard deviation of the difference sequence. Standard deviation The developmental stability score is obtained by normalizing the reciprocal of the derivative. The normalization process uses the formula... This leads to a developmental stability score It falls within the range of 0 to 1. The developmental stability score reflects the degree of fluctuation in the rate of development of a trait.

[0087] In practice, the key phenotypic trait parameter sequences, after sorting and filtering, are integrated to form a hierarchical full-cycle phenotypic map. Trait parameter sequences with growth trend and developmental stability scores both higher than preset thresholds (set by the user according to specific application needs) are selected and marked as stable key trait sequences. See Table 2 for the results of scoring and filtering different trait parameter sequences:

[0088] Table 2: Growth trend, developmental stability score and screening markers for trait parameter sequences

[0089] phenotypic parameter sequence description Growth trend Developmental stability score Whether to mark as a stable key trait sequence Leaf A surface area sequence 0.94 0.87 yes Stem main stem length sequence 0.88 0.92 yes Flower B color saturation sequence 0.45 0.31 no Fruit C-volume sequence 0.91 0.89 yes

[0090] Using growth stage as the first level, stable key trait sequences are categorized into budding, vegetative growth, flowering, fruiting, and maturity stages. The categorization is based on which growth stage the trait sequence primarily falls within. Under each growth stage level, plant organ type is used as the second level, categorizing trait sequences into roots, stems, leaves, flowers, and fruits. Under each plant organ type level, specific phenotypic trait measures are used as the third level, categorizing sequences into volume, surface area, length, orientation, and color. This three-level structure can be understood as a tree-like classification system, with each stable key trait sequence being a leaf node in the tree. By combining the three-level structure of growth stage, plant organ type, and phenotypic trait measures with temporal evolution edges between sequences, connecting the state nodes of the same trait sequence at different time points, a hierarchical, full-cycle phenotypic map with a tree-like network structure is constructed.

[0091] In some embodiments, the growth trend degree can also be calculated using a nonlinear trend model, such as a logarithmic growth model or a logistic model, and the corresponding coefficient of determination can be calculated. Optionally, the preset threshold can be set with different values ​​for different trait types (such as morphological traits and color traits) to perform differentiated screening. When constructing a hierarchical full-cycle phenotypic atlas, the atlas data structure can be stored and represented using a graph database, where nodes and edges in the tree-like network structure are accompanied by attribute information. It can be understood that the hierarchical full-cycle phenotypic atlas provides a structured phenotypic data view that integrates temporal evolution information from macroscopic growth stages to microscopic organ traits. In some embodiments, data comparison can be performed by comparing the distribution differences of growth trend degree and developmental stability scores between stable key trait sequences included in the atlas and sequences not included, and the distribution differences can be visualized using box plots.

[0092] In one embodiment of the present invention, the method further includes autonomous planning of the mobile acquisition platform's movement path and data re-acquisition. In the dynamic evolution map of plant phenotypic data, plant organ regions or time points with 3D reconstruction quality below a preset threshold are identified. Based on the spatial location and time point of the identified low-quality regions, the data types, optimal acquisition pose, and acquisition time that the mobile acquisition platform needs to re-acquire are calculated. A re-acquisition task instruction list is generated, which includes the platform's movement path, robotic arm adjustment parameters, and sensor trigger parameters. The mobile acquisition platform is controlled to automatically re-acquire data according to the re-acquisition task instruction list, and the newly acquired data is fed into image sequences and depth point cloud data for iterative optimization.

[0093] In practical implementation, the mobile phenotypic information acquisition method based on the entire plant growth cycle also includes autonomous planning of the acquisition platform's movement path and data supplementation. In the dynamic evolution map of plant phenotypic data, plant organ regions or time points with 3D reconstruction quality below a preset threshold are identified. The assessment of 3D reconstruction quality is based on the integrity and consistency of the voxel field of plant phenotypic semantic segmentation. For each plant organ instance region, the porosity and surface smoothness of its voxel model are calculated. Porosity is defined as the ratio of missing voxels within the instance model to the expected volume of the total voxels. Surface smoothness is quantified by calculating the average angle difference of the surface normal vector change of the instance model. The preset threshold can be set separately for different organ types. Quality assessment value. The calculation can be expressed as:

[0094]

[0095] in: Indicates the porosity. This represents the normalized surface smoothness score. and It is a weighting coefficient and When the quality assessment value When the quality of a region falls below a preset threshold for the corresponding organ type, the region and its corresponding time point are marked as low-quality regions.

[0096] In practical implementation, based on the spatial location and time point of the identified low-quality areas, the data types, optimal acquisition pose, and acquisition time required for the mobile acquisition platform to be acquired are calculated. The calculation process is based on the 3D reconstruction quality assessment results and the 3D spatial occupancy information provided by the plant phenotypic dynamic evolution map. For low-quality areas, the data types requiring acquisition include color images and depth point clouds. The optimal acquisition pose is obtained by solving an optimization problem. The optimization objective is to minimize the angle between the expected acquisition viewpoint and the surface normal of the low-quality area, and to maximize the coverage of the target area by the sensor's field of view, while considering the kinematic constraints of the robotic arm. The acquisition time is selected during periods when the plant's growth state is relatively static, such as early morning or evening, to reduce imaging errors caused by the plant's daytime movement. In some embodiments, when calculating the optimal acquisition pose, it is also necessary to consider avoiding collisions between the platform, robotic arm, and the plant itself or the environment. Collision detection is based on a known 3D scene model. A list of acquisition task instructions is generated, which includes the platform movement path, robotic arm adjustment parameters, and sensor trigger parameters. The platform movement path is given in the form of a sequence of 3D spatial coordinate points and robotic arm end-effector posture points. The robotic arm's adjustment parameters include the target angles of each joint. Sensor triggering parameters include camera aperture, exposure time, and LiDAR scanning mode.

[0097] The mobile data acquisition platform automatically performs supplementary data acquisition according to the supplementary acquisition task instruction list, and sends the newly acquired data to the image sequence and depth point cloud data for iterative optimization. During the automatic supplementary acquisition process, the navigation system of the mobile acquisition platform guides the platform to move along the planned path, the robotic arm controller drives the robotic arm to adjust to the target pose, and the sensor controller triggers data acquisition at a specified time. In some embodiments, data supplementary acquisition can be performed multiple times. After each supplementary acquisition, the plant phenotypic dynamic evolution map is immediately updated with new data, and the 3D reconstruction quality is reassessed. If the quality still does not meet the requirements, a new round of supplementary acquisition task calculation and execution is initiated. It can be understood that this closed-loop autonomous planning and supplementary acquisition mechanism can dynamically improve the completeness and accuracy of the phenotypic dataset. Optionally, the generation of the supplementary acquisition task instruction list also considers the overall time and energy consumption of task execution, prioritizing the acquisition scheme with shorter paths and less robotic arm movement while meeting quality requirements. Iterative optimization by feeding newly acquired data into image sequences and depth point cloud data means re-executing steps such as spatiotemporal registration, feature tensor construction, and phenotypic analysis network inference using the complete dataset containing the new data, thereby generating updated and higher-quality dynamic evolution maps of plant phenotypic patterns. Iterative optimization can be understood as a continuous improvement process until the 3D reconstruction quality of all key regions meets the preset standards.

[0098] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A mobile phenotyping method based on the whole cycle of plant growth, characterized in that, The method comprises the following steps: acquiring image sequences and depth point cloud data of a target plant at multiple key growth stages; spatiotemporally registering and fusing the image sequences and the depth point cloud data to construct a multi-dimensional phenotype feature tensor, which is used to represent the joint information of the morphological structure and color texture of the plant in three-dimensional space; inputting the multi-dimensional phenotype feature tensor into a learnable phenotype analysis network, which generates a plant phenotype dynamic evolution map by iteratively enhancing the feature response related to plant organs and suppressing the background feature response; extracting all key phenotype trait parameter sequences from the germination stage to the mature stage according to the plant phenotype dynamic evolution map; calculating the growth trend degree and the development stability score of each key phenotype trait parameter sequence, and sorting and screening the key phenotype trait parameter sequences according to the growth trend degree and the development stability score; integrating the sorted and screened key phenotype trait parameter sequences to form a hierarchical whole-cycle phenotype atlas, wherein each node in the hierarchical whole-cycle phenotype atlas represents a phenotype state at a specific growth time, and each edge represents the evolution relationship between the phenotype states.

2. The mobile phenotyping information acquisition method based on the whole growth cycle of plants according to claim 1, characterized in that, The method comprises the following steps: detecting and matching feature points of the image sequences to estimate the spatial transformation relationship between images at different time points, thereby realizing the time sequence alignment of the image sequences; performing point cloud registration on the depth point cloud data to unify the point clouds collected at different time points to the same world coordinate system; in the unified spatiotemporal coordinate system, mapping the pixel color information of the time sequence aligned image sequences to the corresponding three-dimensional points of the depth point cloud data after point cloud registration to generate a dense three-dimensional point cloud with true color information; performing downsampling and feature extraction on the dense three-dimensional point cloud with true color information according to a preset spatial voxel grid, wherein the features include the color histogram, the normal vector distribution and the point density in the voxel; arranging all the voxel features according to their spatial positions to form a four-dimensional tensor, wherein the four-dimensional tensor is the multi-dimensional phenotype feature tensor, and the four dimensions represent the spatial three-dimensional coordinates and the feature channels.

3. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 1, wherein, The method comprises the following steps: inputting the multi-dimensional phenotype feature tensor into a three-dimensional convolutional encoder, which extracts plant structure features with multi-scale spatial receptive fields; inputting the encoded features into a feature refining module based on the self-attention mechanism, which calculates the correlation between features at different spatial positions, enhances the feature response related to plant organs, and weakens the feature response related to the soil and the pot background; The refined features are input into a three-dimensional convolutional decoder, which progressively upsamples and reconstructs the feature map, outputting a high-resolution plant phenotypic semantic segmentation voxel field. In the plant phenotypic semantic segmentation voxel field, three-dimensional instances of each semantic category are extracted at different time points, and the states of the same instance at different time points are connected by directed edges to form the plant phenotypic dynamic evolution graph. The nodes in the plant phenotypic dynamic evolution graph have timestamps and three-dimensional spatial occupancy information.

4. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 3, wherein, Based on the aforementioned plant phenotypic dynamic evolution diagram, the sequences of all key phenotypic parameters from germination to maturity were extracted, including: Traverse each plant organ instance node in the plant phenotypic dynamic evolution diagram, where the plant organ instance nodes include roots, stems, leaves, flowers, and fruits; For each plant organ instance node, its phenotypic parameters are calculated from the three-dimensional spatial occupancy information of the plant organ instance node at different time points. The phenotypic parameters include three-dimensional volume, surface area, skeleton length, spatial orientation, and color mean. Arrange the values ​​of the same phenotypic trait parameter calculated at different time points for the same plant organ instance node in chronological order to form a sequence, thus forming the phenotypic parameter sequence of the plant organ instance node. The sequence of all phenotypic parameters of all plant organ instance nodes is summarized to form the set of key phenotypic phenotypic parameter sequences.

5. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 4, wherein, For each of the key phenotypic trait parameter sequences, calculate its growth trend degree and developmental stability score, including: For a sequence of key phenotypic trait parameters, a trend line is fitted using the least squares method; The determination coefficients of the actual observed values ​​of the key phenotypic trait parameter sequence and their corresponding predicted values ​​on the trend line are calculated. The normalized determination coefficients are used as the growth trend degree, which reflects the degree of clarity of the direction of trait development. The standard deviation of the difference between observations at adjacent time points in the sequence of key phenotypic trait parameters is calculated, and the reciprocal of the standard deviation is normalized to obtain the developmental stability score, which reflects the degree of fluctuation in the trait development rate.

6. The mobile phenotyping information acquisition method based on the whole cycle of plant growth according to claim 5, wherein, The integrated sequence of key phenotypic trait parameters, after sorting and filtering, forms a hierarchical full-cycle phenotypic map, including: The trait parameter sequences in which both the growth trend score and the developmental stability score are higher than a preset threshold are selected and marked as stable key trait sequences; Using growth stages as the first level, the stable key trait sequences are classified into germination stage, vegetative growth stage, flowering stage, fruiting stage, and maturity stage; At each growth stage level, the trait sequence is classified into root, stem, leaf, flower, and fruit, with plant organ type as the second level. At each plant organ type level, the sequence is classified into volume, surface area, length, orientation, and color as the third level, based on specific phenotypic trait measures. By combining the three-level structure of growth stage, plant organ type, and phenotypic trait measurement with the temporal evolution edges between sequences, a hierarchical full-cycle phenotypic map with a tree-like network structure is constructed.

7. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 2, wherein, Perform point cloud registration on the depth point cloud data to unify point clouds collected at different times into the same world coordinate system, including: Select a depth point cloud acquired at a fixed time as the target point cloud, and set its corresponding spatial coordinate system as the world coordinate system; For each frame of source point cloud acquired at a non-fixed time, extract its fast point feature histogram features with those in the target point cloud; Based on the extracted fast point feature histogram features, the initial rigid body transformation matrix from the source point cloud to the target point cloud is calculated using the sample consistency initial registration algorithm; Based on the initial rigid body transformation, an iterative nearest point algorithm is used for fine registration. By minimizing the distance error between corresponding points, the optimal rigid body transformation matrix from the source point cloud to the world coordinate system is obtained. Based on the optimal rigid body transformation matrix, a spatial transformation is performed on each three-dimensional point in the source point cloud so that it is aligned with the target point cloud to the world coordinate system.

8. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 3, wherein, The encoded features are input into a feature refinement module based on a self-attention mechanism, including: The encoded feature tensor output by the three-dimensional convolutional encoder is flattened in spatial dimension to obtain a series of feature vectors; Calculate the query vector, key vector, and value vector for each feature vector; The attention weight matrix is ​​obtained by calculating the dot product of the query vector and the key vector of all feature vectors, and then the attention weight matrix is ​​normalized by applying the Softmax function. Multiply the normalized attention weight matrix by the value vector of all feature vectors to obtain a refined feature vector that incorporates global context information; The refined feature vector is reshaped to have the same spatial dimension as the encoded feature tensor, and residually connected with the original encoded features to output the final refined features.

9. The mobile phenotyping information collection method based on the whole growth cycle of plants according to claim 1, wherein, The method also includes autonomous path planning and data supplementation for the data acquisition platform: In the plant phenotypic dynamic evolution map, plant organ regions or time points with three-dimensional reconstruction quality below a preset threshold are identified; Based on the spatial location and time point of the identified low-quality areas, calculate the data types, optimal acquisition pose, and acquisition time that the mobile acquisition platform needs to acquire. Generate a list of supplementary sampling task instructions, which includes the platform movement path, robotic arm adjustment parameters, and sensor triggering parameters. The mobile acquisition platform is controlled to automatically acquire new data according to the list of acquisition task instructions, and the newly acquired data is sent to the image sequence and the depth point cloud data for iterative optimization. 10.A mobile phenotyping platform based on the whole cycle of plant growth, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the mobile phenotypic information collection method based on the entire plant growth cycle as described in any one of claims 1 to 9.

Images