Three-dimensional phenotype segmentation and labeling system for field corn crops

Abstract

The invention relates to the field of intelligent identification, and more specifically relates to a three-dimensional phenotype segmentation and labeling system for field corn crops, comprising: an initialization module for executing opening, loading and preprocessing tasks of field corn point cloud data files; the ground segmentation module is used for executing an automatic segmentation task of the corn plant population and the ground; the single plant identification module is used for executing a task of accurately segmenting single plants in the plant population and generating identification information; the skeleton classification module is used for executing skeleton calculation and data point category division tasks of the single-plant corn point cloud; the organ labeling module is used for executing stem and leaf organ structure division and labeling tasks of the single-plant corn point cloud; and the result interaction module is used for executing visualization of output data of the key steps of the modules and storage tasks of projection, segmentation, labeling and classification result data output by the modules.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, and more specifically to a three-dimensional phenotypic segmentation and annotation system for maize crops in the field. Background Technology

[0002] With the deepening development of crop phenotyping research and agricultural intelligence, 3D point cloud data, due to its advantages such as strong structural integrity and rich spatial information, is widely used for crop structure modeling and trait extraction. However, existing 3D point cloud processing processes are fragmented, with some steps requiring manual intervention in point cloud cropping, labeling, and individual plant separation, which is time-consuming, labor-intensive, and has poor repeatability. Especially in densely planted field environments, achieving automated individual plant identification and fine organ classification still faces challenges such as an imperfect overall process, insufficient segmentation and classification accuracy, and low efficiency in acquiring original labeled data, thus hindering the high-throughput phenotyping extraction application of 3D phenotypic data in large-scale experiments.

[0003] To address the aforementioned issues, there is an urgent need for a streamlined method based on point cloud processing and structural feature analysis to efficiently segment and label data from populations to individual plants and from whole organisms to organs. This would improve segmentation accuracy and labeling efficiency while reducing manual intervention, generating high-quality labeled data for intelligent analysis of phenotypic features. Summary of the Invention

[0004] In view of this, the present invention provides a three-dimensional phenotypic segmentation and annotation system for maize crops in the field, aiming to solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A three-dimensional phenotypic segmentation and annotation system for maize crops in the field, comprising: The initialization module is used to perform tasks such as opening, loading, and preprocessing field corn point cloud data files; The ground segmentation module is used to perform the task of automatically segmenting the corn plant population from the ground. The single plant identification module is used to perform the task of accurately segmenting individual plants in a plant population and generating identification information; The skeleton classification module is used to perform skeleton calculation and data point classification tasks for single corn plant point clouds; The organ annotation module is used to perform the task of classifying and annotating the stem and leaf organ structures of a single maize point cloud; The results interaction module is used to visualize the output data of the key steps of the above modules and to save the projection, segmentation, annotation, and classification results data output by each module.

[0006] Furthermore, the initialization module supports opening and loading 3D point cloud data files in PLY, PCD, TXT, and XYZ formats, and selectively performs integrated processing of noise filtering, point cloud downsampling, coordinate space normalization, and connectivity enhancement. The initialization module includes a data sharing interface, which is used to provide corresponding point cloud data to other modules respectively.

[0007] Furthermore, the ground segmentation module is used to perform the automatic segmentation task between the corn plant population and the ground. Specifically, it first performs uniform hierarchical division based on the vertical height of the point cloud data; then, based on the growth structure of the corn plants, it performs density analysis only on the middle and lower layers to determine the layer with the lowest density of each plant as the stem positioning layer; then, it performs top-down projection of the point cloud on the stem positioning layer to obtain a two-dimensional projection point set; on the obtained two-dimensional projection point set, it performs erosion, dilation edge calculation and center clustering operations to locate the stem center point; finally, it back-projects to the three-dimensional coordinate system, and uses the determined stem center points as growth points. Each plant independently performs vertical downward regional growth. When the density change exceeds the average value or an adaptive threshold, it is considered that the stem point has contacted the ground point, and growth stops at this time.

[0008] Furthermore, the single plant identification module is used to achieve accurate segmentation of individual plants in the point cloud data of the corn plant population. Specifically, by projecting the three-dimensional point cloud data onto a two-dimensional plane and performing density statistics, morphological processing and connected component analysis, the smallest independent distribution area of ​​each plant on the plane is effectively identified. Combined with the height information of the original point cloud, these planar areas are expanded into corresponding three-dimensional cube units, thereby defining the spatial boundary of each plant and providing initial constraints for single plant segmentation. If overlapping, occlusion, or intersection of leaves is detected, resulting in overlapping contours, the relevant cubic units are further merged and marked as an occlusion subset. The overlapping subset is decomposed by performing high-level hierarchical and layer-by-layer feature analysis. Based on the inherent relationship between the features of each layer, a plant body with a spatially connected structure is constructed to complete the merging of the hierarchical features of the same plant and realize the division of occluded plants. Based on the actual planting location in the field, the system automatically marks the plant positions by sorting them according to row and column rules in the XY direction of the coordinate axis and generating an ordered point cloud file name format with labels.

[0009] Furthermore, the skeleton classification module performs structural constraint contraction processing and spatial neighborhood search on the input single maize point cloud data to obtain a set of skeleton points representing the trunk structure and extracts skeleton information representing the spatial structure of the maize plant. Based on this skeleton information, a topology graph structure is constructed, and each skeleton point is divided into different topology categories according to its local geometric connectivity, including endpoints connected to only one neighbor, intermediate points connected to two neighbors, branching points connected to three or more neighbors, and isolated points with no neighbors. Based on topological categories, utilizing the topological pattern of a single main stem and multiple branches in maize plants and the significant organ distribution characteristics, the longitudinally continuous intermediate points and a small number of bifurcation points constitute the main axis of the stem. The bifurcation points serve as the starting points of the leaves, extending outwards to the endpoints to form the leaf skeleton, while isolated points correspond to noise or broken areas. A graph-based organ hierarchy model is constructed, where nodes represent key points of organs and edges represent main axis paths of organs, thereby achieving the classification objective of point clouds of a single maize plant skeleton.

[0010] Furthermore, the organ labeling module, based on the classification results of the skeleton classification module or on existing skeleton data with classification information, uses nearest neighbor search to coarsely determine the class of different points, achieving preliminary organ segmentation and obtaining the initial point set of stems and leaves and the point set to be assigned. For the point set to be assigned, the Euclidean distance, surface projection distance, normal vector consistency, topological connectivity, and assignment confidence parameters between it and each initial leaf are calculated. Then, based on the comprehensive distance scoring function, the points to be assigned are assigned to the optimal class point set. After all assignments are completed, secondary assignment correction is performed on points with confidence scores lower than a preset threshold.

[0011] Furthermore, the result interaction module supports the display and evaluation of the output results of key steps in projection, clustering, segmentation, annotation, and phenotypic parameter extraction of each module by drawing two-dimensional or three-dimensional graphics. Meanwhile, it supports each module to organize and save the specified output data in the standard PLY format or the TXT format containing XYZ coordinates and classification information, and the labeled point cloud data can be directly used as the training input for the deep learning models PointNet and PointNet++.

[0012] Furthermore, the single-plant identification module, skeleton classification module, and organ annotation module call the initialization module to load the required data separately. While supporting linked calls, they can also run independently when the single-plant identification module, skeleton classification module, and organ annotation module are not executed or have not received the required data.

[0013] Compared with existing technologies, this invention discloses a three-dimensional phenotypic segmentation and labeling system for maize crops in the field. This invention uses a ground-plant approach, based on the different density of hierarchical point clouds within the maize crop population and the differences in stem structure, to cluster, locate, and calculate individual plants, achieving relatively accurate segmentation of plants and the ground under non-single standards. It innovatively associates key features of adjacent layers with local features at different levels, reconstructing the complete plant structure with continuous layer spatial information, effectively solving the problem of misclassification of organs such as leaves caused by occlusion and confusion. For individual plant segmentation and labeling, the point cloud is spatially sorted according to the actual planting row and column position of the plant, and label names are automatically generated according to coordinate positions, achieving individual plant numbering and spatial location labeling without manual intervention. Based on the geometric structural connections of the individual maize plant skeleton, the classification of the skeleton is extended to the classification of organ categories of the entire plant. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the functional module structure of a field corn dot cloud segmentation system according to the present invention.

[0016] Figure 2 This is a schematic diagram of the algorithm operation flow of a field corn point cloud segmentation system according to the present invention.

[0017] Figure 3 This is a schematic diagram of a field corn planting plan according to the present invention. The method of planting corn and collecting data in this manner can achieve the best results.

[0018] Figure 4 This is a schematic diagram of an automatic ground and plant segmentation algorithm according to the present invention, belonging to the category of ground segmentation modules. Figure 4 (a) shows the original data. Figure 4 (b) is a diagram illustrating the key layer data obtained after layering. Figure 4 (c) represents the center point of each plant after clustering. Figure 4 (d) Visualizes the final plant and ground segmentation results.

[0019] Figure 5 This is a schematic diagram of a group segmentation method for single-plant maize point clouds and its positional labeling algorithm according to the present invention. It belongs to the category of single-plant identification modules. The output result is the original single-plant maize point cloud with row and column label information. Figure 5 (a) shows the original population plant data. Figure 5 (b) is the XY plane projection diagram obtained through projection. Figure 5 (c) represents the boundary outline information of the plant. Figure 5 (d) represents the subset of plants with occlusion extracted. Figure 5 (e) is a schematic diagram of the hierarchical analysis of occlusion subsets and the establishment of plant connectivity structure. Figure 5 (f) Demonstration of the effect of shading plant subset treatment. Figure 5 (g) Visualizes the results of single-plant segmentation and label information generation.

[0020] Figure 6 This is a schematic diagram of a single-plant maize skeleton point cloud segmentation and classification algorithm for stem, leaf, and organ classification, belonging to the skeleton classification module category of the present invention. The output result is single-plant maize skeleton point cloud classification data with sequence (first leaf, second leaf, etc.) and category information (stem, leaf, etc.). Figure 6 (a) The original plant data obtained from the segmentation. Figure 6 (b) is the point cloud information of the plant skeleton. Figure 6 (c) is the midpoint of the skeleton. Figure 6 (d) is the skeleton connection point. Figure 6 (e) represents the category classification of the skeleton.

[0021] Figure 7 This is a schematic diagram of a single maize point cloud organ segmentation algorithm that integrates skeleton information according to the present invention. It belongs to the category of organ annotation modules, and the output result is single maize point cloud data with organ classification information. Figure 7 (a) is a single raw data. Figure 7 (b) Information on the classification of skeleton points. Figure 7 (c) represents the superimposed representation of single original data and skeleton point category classification information. Figure 7 (d) shows the category labeling results for plant organs. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See appendix Figure 1-7 To make the above-mentioned objectives of the present invention more apparent and understandable, and to demonstrate the features and advantages of the inventive method, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1The diagram shows the functional modules of the field corn dot cloud segmentation system, indicating that the invention mainly includes three main modules: population and ground segmentation, individual plant segmentation, and plant organ segmentation, along with visualization and data storage functions.

[0025] Includes an initialization module, used to perform tasks such as opening, loading, and preprocessing field corn point cloud data files; The ground segmentation module is used to perform the task of automatically segmenting the corn plant population from the ground. The single plant identification module is used to perform the task of accurately segmenting individual plants in a plant population and generating identification information; The skeleton classification module is used to perform skeleton calculation and data point classification tasks for single corn plant point clouds; The organ annotation module is used to perform the task of classifying and annotating the stem and leaf organ structures of a single maize point cloud; The results interaction module is used to visualize the output data of the key steps of the above modules and to save the projection, segmentation, annotation, and classification results data output by each module.

[0026] The connection relationships between the above modules are as follows: The initialization module includes a data sharing interface, which establishes connections with all other modules to transmit loaded and preprocessed data to each module, supporting independent or collaborative execution of each module. The ground segmentation module is connected to the single-plant identification module, which transmits the processed data and calls the result interaction module to visualize the intermediate results and save the data. The single-plant identification module is connected to the skeleton classification module, which transmits the processed data and calls the result interaction module to visualize the intermediate results and save the data. In addition, the single-plant identification module is also connected to the initialization module so that it can independently acquire and process data without executing the ground segmentation module. The skeleton classification module is connected to the organ annotation module to transmit processed data and call the result interaction module to visualize intermediate results and save data. In addition, the skeleton classification module is also connected to the initialization module to independently acquire and process data without executing the single-plant identification module. The organ annotation module is connected to the results interaction module, which is used to visualize intermediate results and save data. Furthermore, the organ annotation module is also connected to the initialization module to independently acquire and process data even without executing the skeleton classification module. The results interaction module is connected to all other modules to receive intermediate results for graphical visualization and to save selected intermediate result data to a specified local path.

[0027] Figure 2The flowchart shows the algorithm operation of the corn point cloud segmentation system in the field. Its operation logic is as follows: First, the initialization module opens the specified 3D point cloud data file, loads the data, and performs optional preprocessing procedures that may affect the calculation accuracy and efficiency of subsequent steps (this process in this system design includes noise point removal, downsampling, coordinate space normalization, and connectivity enhancement).

[0028] Next, the ground segmentation module receives preprocessed data from the initialization module and performs segmentation processing on the corn plant population and the ground. After segmentation: 1) Provide the individual plant identification module with point cloud data of the group of maize plants after removing ground points; 2) Simultaneously, the results interaction module is invoked to display the intermediate results of the segmentation process; 3) It can also optionally save the point cloud data of the maize plant population to the local machine; Next, the single-plant identification module receives the output data from the ground segmentation module, performs segmentation and labeling processing of the corn single-plant point cloud, and then: 1) Provide segmented and labeled point cloud data of individual maize plants to the skeleton classification module; 2) Simultaneously, the results interaction module is invoked to display the intermediate segmentation results; 3) Optionally, the point cloud data of a single corn plant can be saved locally with the identification information as the file name; 4) In addition, the single plant identification module also supports loading point cloud data of a group of maize plants with the ground removed directly from the initialization module, and can run independently without skipping the ground segmentation module; Then, the skeleton classification module receives the output data from the single-plant recognition module, performs skeleton calculation and point classification processing on the single-plant corn point cloud, and after completion: 1) Provide the classified point cloud data of a single maize plant skeleton to the organ annotation module; 2) Simultaneously, the results interaction module is invoked to display intermediate classification results; 3) It can also optionally save the single-plant corn skeleton classification point cloud data to the local machine; 4) In addition, the skeleton classification module also supports loading segmented and identified single corn point cloud data directly from the initialization module, and running independently without skipping the single-plant recognition module; Finally, the organ annotation module receives the output data from the skeleton classification module, performs organ classification processing on the point cloud of a single maize plant, and after completion: 1) Call the results interaction module to display intermediate classification results; 2) It can also optionally save the point cloud data of individual maize organ classification to the local machine; In addition, the organ annotation module also supports loading single maize point cloud data and corresponding skeleton classification data directly from the initialization module, and can run independently without skipping the skeleton classification module.

[0029] Figure 3 This is a schematic diagram of corn planting in a field. Under similar planting methods, the inventors have verified the actual effect of the algorithm, which can meet the requirements of subsequent data processing such as phenotypic detection.

[0030] Figure 4 This is a schematic diagram of an automatic segmentation algorithm for ground and plant areas, based on raw data. Figure 4 (a) Reach the layer and obtain the key layer data. Figure 4 (b) Then, the center position points of each plant are obtained by fitting clusters. Figure 4 (c) Then, based on the center point position, density growth is used to obtain the final segmentation result. Figure 4 (d)

[0031] Figure 5 This diagram illustrates the point cloud segmentation of individual maize plants within a population and its bitwise labeling algorithm, using the original population plant data. Figure 5 (a) As input, obtain the XY plane projection through projection. Figure 5 (b) Subsequently, the boundary contour information of the plant was extracted through morphological processing of the two-dimensional image. Figure 5 (c) After that, extend it to a three-dimensional coordinate system and include the occlusion subset with projection overlap. Figure 5 (d) Perform tomography and establish connectivity structures Figure 5 (e) parses out the complete, unshaded plant. Figure 5 (f) Then, the above-mentioned non-occluded subsets are combined to form the single plant segmentation result, and the label information is generated based on the coordinate position. Figure 5 (g)

[0032] Figure 6 This is a schematic diagram of a point cloud segmentation and classification algorithm for stem, leaf, and organ systems of a single maize plant, using the original plant data obtained from the segmentation. Figure 6 (a) Obtaining plant skeleton point cloud information using the open-source KD-Tree algorithm Figure 6 (b) After analyzing the connection characteristics and local structural information of each skeleton point, the skeleton points were divided into intermediate points. Figure 6 (c) and connection point Figure 6 (d) After that, the skeletal category classification is achieved by the point-level category classification. Figure 6 (e).

[0033] Figure 7 This is a schematic diagram of an organ segmentation algorithm for a single maize point cloud that integrates skeletal information, using a single raw data source. Figure 7 (a) and skeleton point classification information Figure 7(b), represented by superposition Figure 7 (c) The point categories of the entire plant are classified using the nearest neighbor calculation method, thereby realizing the category representation of plant organs. Figure 7 (d)

[0034] Because 3D point cloud data contains both dense and sparse areas, especially with a significant density transition at the junction of corn stalks and the ground, density stratification technology could be considered to segment corn stalks from the ground in the 3D point cloud. However, trials revealed that in actual field environments, due to uneven ground and factors such as seed quality, fertility, water, and light, corn plant growth varies, and not all plant stalks have the same horizontal segmentation point on the ground. Therefore, simply using density stratification to segment stalks from the ground has certain inaccuracies and is prone to missegmentation. Furthermore, manually segmenting using existing software requires checking and locating from multiple directions, and when plants are densely planted, accurate segmentation of internal plants becomes impossible.

[0035] To address this issue, this invention proposes a method that, based on longitudinal (Z-axis) stratification, combines density and height constraints to first determine the layer containing all plant stems. Then, after projecting this layer in two dimensions, the neighborhood threshold is dynamically adjusted for each projection point by finding its nearest neighbor and averaging the values. Clustering is then performed using the DBSCAN clustering algorithm (Martin Ester et al., 1996) to determine the plant center points. The number of plant clusters corresponds to the number of center points. Next, these center points are back-projected onto three-dimensional data coordinates and used as seed points for density growth towards the soil. When the density growth rate exceeds 1% of the global density mean, the stem is considered to have "contacted" the soil, and the segmented region is achieved. This growth process is executed cyclically for each individual plant, avoiding mis-segmentation of some plants from the ground due to using a uniform standard.

[0036] To segment the point cloud data of each plant from the plant population after removing the ground point cloud, this invention considers two main steps: determining the distribution range (XY) of each plant's point cloud through two-dimensional projection, and extending the two-dimensional distribution range to three dimensions (XYZ) for plant point cloud segmentation. Specifically, the three-dimensional point cloud data of the maize population is first projected onto a two-dimensional plane (XY plane) to form a projected image. Through density statistics, morphological operations (such as opening or closing operations), and connected component extraction, the distribution area of ​​each plant is identified. In particular, areas with overlapping leaves are treated as a single occluded area, and their XY boundary range is extracted. This step requires that the input point cloud has undergone coordinate correction, the XY plane is parallel to the ground, and the positive Z-coordinate is vertically upward. Two custom-defined triangular structuring elements are used in this step for region closure.

[0037] (1) (2) Next, to address the issue of leaf occlusion in the XY projection when multiple corn plants have overlapping or intersecting leaves in space, the method of this invention, based on the boundary range of the occlusion area on the XY plane and combined with a preset height range (Z coordinate range), crops a local subset of the point cloud in three-dimensional space, constructing a three-dimensional point cloud cube region containing one or more occluded plants. The extracted local point cloud is then layered along the Z-axis height direction (e.g., every 5 cm or 10 cm as a height slice), and the point cloud within each layer is projected again onto the XY plane in two dimensions to extract the point cloud distribution structure of that layer. Image connected component analysis or two-dimensional clustering methods are used to identify the possible leaf structures of multiple plants within that height layer.

[0038] Subsequently, spatial structural connectivity analysis is performed on the segmentation results of each height layer to determine their consistency in the vertical direction. Spatial clustering methods or geometric center trajectory comparison of connected regions are used to merge the clustering results of different layers belonging to the same plant, thus completing the identification of the shaded plants. Furthermore, spatial features such as point cloud density, principal direction vector (based on local principal component analysis (PCA)), normal vector, or curvature can be introduced to assist in the judgment and optimize the clustering of point clouds in shaded areas, enhancing the ability to distinguish regions with blurred boundaries.

[0039] Finally, the 3D cube spatial segmentation information with added Z-coordinates is combined with the plant population to segment individual plants, achieving complete 3D point cloud extraction at the individual corn plant level. For visualizing the segmentation effect, the following matrix is ​​used to connect the boundaries of the cube to form a cube bounding box.

[0040] (3) The proposed method fully utilizes the hierarchical differences in plant structure along the Z-axis, exhibiting good adaptability to maize populations in densely planted environments. While improving segmentation accuracy, it effectively solves the problem of XY projection missegmentation caused by leaf occlusion, and provides accurate data support for subsequent individual plant trait extraction and phenotypic analysis. Furthermore, the method can be integrated with existing two-dimensional segmentation frameworks without requiring additional hardware support, ensuring high execution efficiency.

[0041] Since each segmented plant needs to be saved as a separate point cloud file, corresponding label information needs to be generated to clearly mark the original location of the plant. For example, the information "2-3.ply" represents the third corn plant in the [number]th column (this step needs to be combined with the actual planting plan, and the number of rows and columns needs to be clearly defined; in the experiment, the planting scale was 4*15). The plant information is sorted in ascending order according to the Y-coordinate value, and then divided into groups according to the number of rows. Within each complete group, the plants are sorted according to the X-coordinate. If there is an incomplete group (i.e., the number of plants and the number of rows in the group are inconsistent), it needs to be handled separately.

[0042] After obtaining the point cloud of a single maize plant, further refined processing requires categorizing and classifying the various parts or organs of the plant. This invention selects the geometric connections of the plant skeleton and its local structures as the basis for segmentation, dividing the skeleton according to local features before segmenting the entire plant. Known methods in the art (such as Laplacian shrinkage, KD-Tree neighborhood indexing, Dijkstra path extraction, etc.) are used to achieve skeleton extraction, neighborhood construction, path analysis, and other functions, enabling the computation of the plant point cloud skeleton.

[0043] Subsequently, based on the skeleton classification, the task of classifying local point clouds of maize plant organs is achieved through nearest neighbor search of skeleton points, thereby realizing the classification of the entire plant. This involves two steps: preliminary coarse segmentation and fine segmentation based on a comprehensive evaluation function. Coarse-grained organ extraction is performed on the input plant point cloud to obtain multiple initial leaf point sets. Extract the set of points to be assigned. (This part may not have been correctly assigned in the initial steps due to curling, occlusion, dense overlap, etc.) For each point to be assigned... Iterate through each initial leaf point set. Calculate the following characteristic terms: Euclidean distance (Minimum distance between a point and the center or boundary of the blade), surface projection distance (By fitting the local surface of the blade region and calculating the projection distance of points on that surface), normal vector consistency. (Mean angle between the normal vectors of the point and the points within the blade region), topological connectivity (Whether a point has an adjacency graph path connection to the set of points on that blade). Confidence scoring function: (4) Finally, the results are combined into a scoring function: (5) Each point is assigned to the leaf with the lowest score; if the score of the point is lower than a preset threshold, it is marked as low confidence; after all initial assignments are completed, low confidence points are re-evaluated or reassigned through global optimization (e.g., minimum cut optimization).

[0044] To meet the analytical needs of point cloud data from different growth stages of crops such as maize, a cross-temporal segmentation result fusion mechanism is introduced. Using methods such as whole-plant skeleton structure, ground height alignment, or ICP, multi-temporal point clouds are registered to a common coordinate system. Leaf mappings are constructed for each pair of adjacent temporal phases, and the IoU value between leaf point sets is calculated; their principal direction angles are compared; if the consistency score is higher than a threshold, they are considered to be the same leaf structure. The leaf structure of the previous temporal phase guides the determination of the segmentation point attribution in the current temporal phase, forming a time-aware continuity analysis capability and reducing temporal jumps, misjudgments, and drift.

[0045] In summary, this application possesses the following six core processing methods and functions: 1. Automatic segmentation of ground and plants: By using segmentation methods based on layer density variation, height threshold, two-dimensional clustering, and back projection, combined with the uneven distribution of plant point cloud density and the structural characteristics of the plants themselves, the method first locates the layer that roughly includes all plant stems as much as possible. Then, using this layer as the main analysis object, contour and cluster analysis are used to locate the key positions of each plant stem. Finally, using these key positions as growth points, the plant grows along the stem towards the ground, ultimately achieving automatic segmentation of ground and plants. This method can adapt to complex ground environments and situations where plants are not fully developed or unevenly grown. In addition, experiments have shown that it can also handle situations where potted plants are grown in foam boxes or flower pots.

[0046] 2. Segmentation of individual crops in a population: Based on a specific planting method, the overall distribution range is determined according to the planar area of ​​plant growth. Then, the vertical height is superimposed, and the plants are segmented in the form of a three-dimensional spatial cube with the bounding box of the outline. This can segment individual crops with high precision in relatively dense crop populations.

[0047] 3. Predefined labels for growth location: Based on specific planting methods, the cubic coordinates of segmented plants are sorted by location. Following the planting orientation, the system automatically assigns a label defined by rows and columns to each individual crop, facilitating subsequent classification and analysis. Because this method is based on high-precision segmented cubic information, and the plants in the field do not experience movement or other positional changes, it can be applied throughout the entire crop lifecycle, dynamically monitoring plant distribution and growth status.

[0048] 4. Segmentation and Point-Level Classification of Stems, Leaves, and Other Organs: Utilizing the spatial topology and geometric features of the point cloud, based on the classification of skeleton points, two steps are taken: coarse segmentation using the distance values ​​between the plant point cloud and the skeleton classification point cloud, and fine segmentation based on a comprehensive distance scoring function and confidence score using multiple indicators. This achieves the optimal classification of stems, leaves, and other organs of the maize crop, and each point cloud data point is labeled. If there is data from multiple growth stages, temporal consistency scoring can be performed using the IoU overlap rate between leaf point sets across time points and the angular similarity between the principal directions of corresponding leaves at different time periods. This increases the accuracy of segmentation and allows for the comprehensive utilization of time-series data, constructing a cross-time period plant organ segmentation consistency evaluation.

[0049] 5. Modular Design and Processing: The system provides modular processing capabilities, allowing users to initiate the entire process with simple operations. Furthermore, each module can be called independently, and intermediate and processing results can be saved separately as files for easy subsequent data analysis and deep learning model training.

[0050] 6. Labeled Data Generation: Through efficient processing and accurate segmentation and labeling, this system can generate high-quality labeled data for deep learning and neural network models, supporting crop phenotypic analysis tasks.

[0051] Compared to traditional manual annotation methods, the method of this invention has a high degree of automation, significantly improved processing efficiency, and standardized output format. Without relying on complex models, it can achieve accurate segmentation and classification of individual plants and organs by combining traditional methods, providing high-quality data support for phenotypic research. It is especially suitable for the needs of large-scale, accurate labeled data in deep learning and intelligent analysis tasks.

[0052] This application's ground and plant segmentation method utilizes the varying point cloud density of maize crop populations and the differences in stem structure to perform clustering, localization, and plant segmentation calculations, achieving relatively accurate plant-to-ground segmentation under non-single-standard conditions. Individual plant segmentation and labeling spatially sort the point cloud based on the actual planting row and column positions of the plant, automatically generating label names according to coordinate positions, achieving individual plant numbering and spatial location annotation without manual intervention. The individual plant organ classification method supports point-level classification, classifying each data point in the 3D point cloud, enabling the identification and automatic labeling of maize stems, leaves, and other organs, offering higher efficiency and accuracy compared to traditional manual labeling methods. The processing results can be directly used to train deep neural network models, extracting high-throughput phenotypic information from field maize crops. It is widely applicable to agricultural intelligent applications such as crop semantic segmentation, structural modeling, growth monitoring, and intelligent analysis, offering advantages such as automated and modular processing, high-precision and efficient segmentation results, and visualized and interactive segmentation process.

[0053] This invention realizes a series of processes from field population point cloud data to ground segmentation, individual plant segmentation, organ segmentation, and category labeling. Based on the different hierarchical densities of the 3D point cloud layers and the growth differences in stem structure of field maize populations, the invention achieves relatively accurate segmentation of plants and uneven ground under non-single-standard conditions by using layer-based local positioning and local density-based plant segmentation calculations. Recognizing the significant viewpoint-dependent characteristics of leaf occlusion in maize populations, the invention innovatively associates key features of adjacent layers with local hierarchical features, reconstructing the complete plant structure with continuous hierarchical spatial information, effectively solving the problem of misclassification of organs such as leaves caused by occlusion confusion. Based on the geometric connectivity of the individual maize plant's skeleton, the classification of the skeleton is extended to the classification of organ categories of the entire plant. Simultaneously, the output results are data point-level category information, which can be directly applied to phenotypic feature calculation and deep learning model training.

[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0055] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field, characterized in that, include: The initialization module is used to perform tasks such as opening, loading, and preprocessing field corn point cloud data files; The ground segmentation module is used to perform the task of automatically segmenting the corn plant population from the ground. The single plant identification module is used to perform the task of accurately segmenting individual plants in a plant population and generating identification information; The skeleton classification module is used to perform skeleton calculation and data point classification tasks for single corn plant point clouds; The organ annotation module is used to perform the task of classifying and annotating the stem and leaf organ structures of a single maize point cloud; The results interaction module is used to visualize the output data of the key steps of the above modules and to save the projection, segmentation, annotation, and classification results data output by each module.

2. The three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The initialization module supports opening and loading 3D point cloud data files in PLY, PCD, TXT, and XYZ formats, and selectively performs integrated processing of noise filtering, point cloud downsampling, coordinate space normalization, and connectivity enhancement. The initialization module includes a data sharing interface, which is used to provide corresponding point cloud data to other modules respectively.

3. The three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The ground segmentation module is used to perform the automatic segmentation of the corn plant population from the ground. Specifically, it first performs uniform hierarchical division based on the vertical height of the point cloud data; then, based on the growth structure of the corn plants, it performs density analysis only on the middle and lower layers to determine the layer with the lowest plant density as the stem positioning layer; then, it performs top-down projection of the point cloud on the stem positioning layer to obtain a two-dimensional projection point set; on the obtained two-dimensional projection point set, it performs erosion, dilation edge calculation and center clustering operations to locate the stem center point; finally, it back-projects to the three-dimensional coordinate system, and uses the determined stem center points as growth points. Each plant independently performs vertical downward regional growth. When the density change exceeds the average value or an adaptive threshold, it is considered that the stem point has contacted the ground point, and growth stops at this time.

4. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field, as described in claim 1, is characterized in that... The single plant identification module is used to achieve accurate segmentation of individual plants in the point cloud data of maize plant population. Specifically, by projecting the three-dimensional point cloud data onto a two-dimensional plane and performing density statistics, morphological processing and connected component analysis, the module effectively identifies the smallest independent distribution area of ​​each plant on the plane. Combined with the height information of the original point cloud, these planar areas are expanded into corresponding three-dimensional cube units, thereby defining the spatial boundary of each plant and providing initial constraints for single plant segmentation. If overlapping, occlusion, or intersection of leaves is detected, resulting in overlapping contours, the relevant cubic units are further merged and marked as an occlusion subset. The overlapping subset is decomposed by performing high-level hierarchical and layer-by-layer feature analysis. Based on the inherent relationship between the features of each layer, a plant body with a spatially connected structure is constructed to complete the merging of the hierarchical features of the same plant and realize the division of occluded plants. Based on the actual planting location in the field, the system automatically marks the plant positions by sorting them according to row and column rules in the XY direction of the coordinate axis and generating an ordered point cloud file name format with labels.

5. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The skeleton classification module performs structural constraint shrinkage processing and spatial neighborhood search on the input single maize point cloud data to obtain a set of skeleton points representing the trunk structure and extracts skeleton information representing the spatial structure of the maize plant. Based on this skeleton information, a topology graph structure is constructed, and each skeleton point is divided into different topology categories according to its local geometric connectivity, including endpoints connected to only one neighbor, intermediate points connected to two neighbors, branching points connected to three or more neighbors, and isolated points with no neighbors. Based on topological categories, utilizing the topological pattern of a single main stem and multiple branches in maize plants and the significant organ distribution characteristics, the longitudinally continuous intermediate points and a small number of bifurcation points constitute the main axis of the stem. The bifurcation points serve as the starting points of the leaves, extending outwards to the endpoints to form the leaf skeleton, while isolated points correspond to noise or broken areas. A graph-based organ hierarchy model is constructed, where nodes represent key points of organs and edges represent main axis paths of organs, thereby achieving the classification objective of point clouds of a single maize plant skeleton.

6. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The organ labeling module, based on the classification results of the skeleton classification module or on existing skeleton data with classification information, uses nearest neighbor search to coarsely determine the class of different points, achieving preliminary organ segmentation and obtaining initial point sets for stems and leaves, as well as point sets to be assigned. For the point sets to be assigned, the Euclidean distance, surface projection distance, normal vector consistency, topological connectivity, and assignment confidence parameters between each point and each initial leaf are calculated. Then, based on the comprehensive distance scoring function, the points to be assigned are assigned to the optimal class point set. After all assignments are completed, secondary assignment correction is performed on points with confidence scores below a preset threshold.

7. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The results interaction module supports the display and evaluation of the key steps of projection, clustering, segmentation, annotation, and phenotypic parameter extraction in each module by drawing two-dimensional or three-dimensional graphics. Meanwhile, it supports each module to organize and save the specified output data in the standard PLY format or the TXT format containing XYZ coordinates and classification information, and the labeled point cloud data can be directly used as the training input for the deep learning models PointNet and PointNet++.

8. A three-dimensional phenotypic segmentation and annotation system for maize crops in the field according to claim 1, characterized in that, The single-plant identification module, skeleton classification module, and organ annotation module call the initialization module to load the required data separately. While supporting linked calls, they can also run independently when the single-plant identification module, skeleton classification module, and organ annotation module are not executed or have not received the required data.

Images