A three-dimensional phenotype segmentation and labeling system for field corn crops

Abstract

The present application relates to the field of intelligent identification, more particularly to a three-dimensional phenotype segmentation and labeling system for field corn crops, comprising: an initialization module for performing opening, loading and preprocessing tasks of field corn point cloud data files; a ground segmentation module for performing automatic segmentation tasks of corn plant groups and ground; a single plant recognition module for performing accurate segmentation of single plants in the plant group and generating identification information tasks; a skeleton classification module for performing skeleton calculation and data point category division tasks of single corn point clouds; an organ labeling module for performing stem and leaf organ structure division and labeling tasks of single corn point clouds; and a result interaction module for performing visualization of output data of key steps of each module and saving projection, segmentation, labeling and classification result data output by each module.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent identification, more particularly to a three-dimensional phenotype segmentation and labeling system for field corn crops. BACKGROUND

[0002] With the in-depth development of crop phenotype research and agricultural intelligence, three-dimensional point cloud data is widely used in crop structure modeling and trait extraction due to its strong structural integrity and rich spatial information. The existing three-dimensional point cloud processing process is scattered, and some processes require manual participation in point cloud cutting, labeling and single plant separation, which is time-consuming, labor-intensive and has poor repeatability. Especially in the field of dense planting environment, automatic single plant recognition and organ fine classification are still faced with the problems of imperfect whole process, insufficient segmentation and classification accuracy, and low efficiency of original labeling data acquisition, which restricts the application of three-dimensional phenotype data in high-throughput phenotype extraction in large-scale experiments.

[0003] To solve the above problems, a process-perfect method based on point cloud processing and structural feature analysis is needed to efficiently realize the segmentation and labeling from group to single plant and from whole to organ, improve the segmentation accuracy and labeling efficiency while reducing manual intervention, and generate high-quality labeled data for intelligent analysis of phenotype characteristics. SUMMARY

[0004] Therefore, the present application provides a three-dimensional phenotype segmentation and labeling system for field corn crops to solve the above technical problems.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] A three-dimensional phenotype segmentation and labeling system for field corn crops comprises:

[0007] An initialization module is used to perform the opening, loading and preprocessing tasks of field corn point cloud data files;

[0008] A ground segmentation module is used to perform the automatic segmentation task of corn plant groups and the ground;

[0009] A single plant recognition module is used to perform the task of accurately segmenting single plants in the plant group and generating identification information;

[0010] A skeleton classification module is used to perform the tasks of skeleton calculation and data point category division of single corn point cloud;

[0011] An organ labeling module is used to perform the tasks of stem and leaf organ structure division and labeling of single corn point cloud;

[0012] A result interaction module is configured to perform the visualization of the output data of the key steps of the modules and the saving of the projection, segmentation, labeling and classification result data of the modules.

[0013] Further, the initialization module supports opening and loading of three-dimensional point cloud data files in PLY, PCD, TXT and XYZ formats, and selectively performs integrated processing of noise filtering, point cloud downsampling, coordinate space normalization and connectivity enhancement.

[0014] The initialization module includes a data sharing interface configured to provide corresponding point cloud data to other modules.

[0015] Further, the ground segmentation module is configured to perform the automatic segmentation of corn plant groups and the ground. Specifically, first, the point cloud data is uniformly divided into layers based on the longitudinal height; then, only the middle and lower layers are subjected to density analysis according to the growth structure of the corn plants, and the layer with the minimum plant density is determined as the stem positioning layer; then, the point cloud is projected from above to obtain a two-dimensional projection point set, and the center clustering operation is performed on the two-dimensional projection point set to locate the stem center point; finally, the three-dimensional coordinate system is obtained by back projection, and the growth of each plant is independently performed in the longitudinal direction downward. When the density change exceeds the average value or the adaptive threshold, it is considered that the stem point has contacted the ground point, and the growth is stopped at this time.

[0016] Further, the single plant recognition module is configured to realize the accurate segmentation of a single plant in the point cloud data of a corn plant group. Specifically, the three-dimensional point cloud data is projected onto a two-dimensional plane, and density statistics, morphological processing and connected component analysis are performed to effectively identify the minimum independent distribution area of each plant on the plane. Combined with the height information of the original point cloud, these plane areas are expanded into corresponding three-dimensional cubic units, thereby limiting the spatial boundary of each plant and providing an initial constraint for single plant segmentation.

[0017] If the overlapping of the contours caused by the overlap, occlusion or intersection of the leaves is detected, the related cubic units are further merged and marked as an occluded subset; the occluded subset is subjected to height layering and layer-by-layer feature analysis to realize the decomposition of the overlapping contours, construct a spatial structure connected plant body according to the internal relationship of the features of each layer, complete the merging of the hierarchical features of the same plant, and realize the single plant determination of the occluded plant.

[0018] According to the actual field planting orientation, the plant positions are automatically calibrated after being sorted according to the row and column rules in the XY direction of the coordinate axis, and an ordered point cloud file name format with labels is generated.

[0019] Further, the skeleton classification module performs structure constraint shrinkage processing and spatial neighborhood search on the input single corn point cloud data to obtain a skeleton point set for representing the main stem structure and extract skeleton information representing the spatial structure of the corn plant; based on the skeleton information, a topological graph structure is constructed, and the local geometric connectivity of each skeleton point is divided into different topological categories, including an end point connected to only one adjacent point, an intermediate point connected to two adjacent points, a branching point connected to three or more adjacent points, and an isolated point without adjacent points;

[0020] Based on the topological category, the single main stem + multi-branch topological rule and the significant organ distribution characteristics of the corn plant are used, the longitudinally continuous intermediate points and a small number of branching points form the main stem axis, the branching points serve as the starting position of the leaves, and the leaf skeleton is formed by extending outward until the end point, and the isolated points correspond to noise or broken areas.

[0021] An organ hierarchical model based on a graph structure is constructed, in which the nodes represent the key points of the organs, and the edges represent the organ main axis path, thereby completing the classification target of the single corn plant skeleton point cloud.

[0022] Further, the organ labeling module, based on the classification results of the skeleton classification module or based on existing skeleton data with classification information, realizes the rough determination of the class of different points through nearest neighbor search, realizes preliminary organ segmentation, and obtains the point set of the initial stem and leaf and the point set to be allocated; for the point set to be allocated, the Euclidean distance, surface projection distance, normal vector consistency, topological connectivity, and allocation confidence parameters of each initial leaf are calculated, and then the point to be allocated is attributed to the optimal class point set based on the comprehensive distance scoring function, and after all the allocation is completed, the points with a confidence lower than a preset threshold are subjected to secondary allocation correction.

[0023] Further, the result interaction module supports the display and effect evaluation of the output results of the key steps of the projection, clustering, segmentation, labeling and phenotype parameter extraction of each module by drawing two-dimensional or three-dimensional graphics.

[0024] At the same time, each module supports the organization and saving of the specified output data in the standard PLY format or the TXT format containing XYZ coordinates and classification information, and the labeled point cloud data is directly used for the training input of the deep learning models PointNet and PointNet++.

[0025] Further, the single plant recognition module, the skeleton classification module and the organ labeling module call the initialization module to load the required data independently, support the linkage calling, and independently run in the case that the single plant recognition module, the skeleton classification module and the organ labeling module do not execute and do not receive the required data.

[0026] Compared with the prior art, the three-dimensional phenotype segmentation and labeling system for field corn crops is provided. The present application realizes more accurate segmentation of plants and ground under non-single standard through clustering positioning and branch calculation according to the different hierarchical point cloud densities of corn crop populations and the differences in the stem structure of corn crops. The present application innovatively associates adjacent layer key features through hierarchical local feature, reconstructs the complete plant structure with continuous layer space information, and effectively solves the problem of misclassification of organs such as leaves caused by shielding confusion factors. The single crop segmentation and identification realizes single number and spatial position labeling without manual intervention by spatially sorting point clouds according to the actual planting row and column positions of the plants and automatically generating label names according to the coordinate positions. The organ category division of the whole plant is realized by extending the classification of the skeleton to the organ category division of the whole plant according to the geometric structure connection relationship of the single corn skeleton. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0028] Figure 1 It is a functional module structure diagram of the field corn point cloud segmentation system of the present application.

[0029] Figure 2 It is an algorithm running process diagram of the field corn point cloud segmentation system of the present application.

[0030] Figure 3 It is a field corn planting plane diagram. The method plants corn and collects data in this way, and the best effect can be achieved.

[0031] Figure 4 It is an automatic ground and plant segmentation algorithm diagram of the present application, which belongs to the ground segmentation module category, Figure 4 (a) is a raw data display, Figure 4 (b) is a key layer data display obtained after layering, Figure 4 (c) is a center position point of each plant after clustering, Figure 4 (d) is the final plant and ground segmentation result visualization.

[0032] Figure 5 It is a group segmentation single corn point cloud and its bit marking algorithm diagram of the present application, which belongs to the single plant identification module category, and the output result is a single corn point cloud with row and column label information, wherein, Figure 5 (a) is the original group plant data,Figure 5 (b) is the XY plane projection map obtained by projection, Figure 5 (c) is the boundary contour information of the plant, Figure 5 (d) is the extracted subset of plants with occlusion, Figure 5 (e) is the occlusion subset hierarchical analysis and plant connectivity structure establishment diagram, Figure 5 (f) is the occlusion plant subset processing effect display, Figure 5 (g) is the single plant segmentation result and label information generation result visualization.

[0033] Figure 6 It is a single plant corn skeleton point cloud stem and leaf organ segmentation and classification algorithm diagram, belongs to the skeleton classification module category, and the output result is single plant corn skeleton point cloud classification data with sequence (first leaf, second leaf, etc.) and category information (stem, leaf, etc.), wherein, Figure 6 (a) is the plant original data obtained by segmentation, Figure 6 (b) is the plant skeleton point cloud information, Figure 6 (c) is the skeleton intermediate point, Figure 6 (d) is the skeleton connection point, Figure 6 (e) is the category division of the skeleton.

[0034] Figure 7 It is a single plant corn point cloud organ segmentation algorithm diagram fusing skeleton information, belonging to the organ labeling module category, and the output result is single plant corn point cloud data with organ classification information, wherein, Figure 7 (a) is single original data, Figure 7 (b) is skeleton point category division information, Figure 7 (c) is the superimposed representation of single original data and skeleton point category division information, Figure 7 (d) is the category labeling result of the plant organ. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] Referring to the accompanying Figures 1-7 To make the above-mentioned purposes of the present application more obvious and easy to understand, and to embody the characteristics and advantages of the inventive method, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0037] Figure 1A field corn point cloud segmentation system function module structure diagram shows that the present application mainly includes three main modules related to visualization and data saving functions of group and ground segmentation, single plant segmentation and plant organ segmentation.

[0038] The initialization module includes an initialization module for performing opening, loading and preprocessing tasks of field corn point cloud data files;

[0039] The ground segmentation module is used to perform the automatic segmentation task of corn plant groups and the ground;

[0040] The single plant recognition module is used to perform the task of accurately segmenting single plants in the plant group and generating identification information;

[0041] The skeleton classification module is used to perform the skeleton calculation and data point category division tasks of single corn point cloud;

[0042] The organ labeling module is used to perform the stem and leaf organ structure division and labeling task of single corn point cloud;

[0043] The result interaction module is used to perform the visualization of the output data of the key steps of the above modules and the projection, segmentation, labeling and classification result data saving tasks of the output of each module.

[0044] The connection relationship between the above modules is:

[0045] The initialization module includes a data sharing interface, which is connected with all other modules, and is used to transmit the loaded and preprocessed data to each module to support independent or cooperative execution of each module;

[0046] The ground segmentation module is connected with the single plant recognition module, transmits the processed data and calls the result interaction module to perform intermediate result visualization display and data saving;

[0047] The single plant recognition module is connected with the skeleton classification module, transmits the processed data and calls the result interaction module to perform intermediate result visualization display and data saving; and the single plant recognition module is also connected with the initialization module to independently obtain and process data without executing the ground segmentation module;

[0048] The skeleton classification module is connected with the organ labeling module, transmits the processed data, and calls the result interaction module to perform intermediate result visualization display and data saving; and the skeleton classification module is also connected with the initialization module to independently obtain and process data without executing the single plant recognition module;

[0049] The organ labeling module is connected with the result interaction module, and calls it to visualize and save the intermediate results. In addition, the organ labeling module is also connected with the initialization module to independently obtain and process data without executing the skeleton classification module.

[0050] The result interaction module is connected with all other modules to receive intermediate results for graphical visualization and save selected intermediate result data to a specified local path.

[0051] Figure 2 The algorithm running flowchart of the field corn point cloud segmentation system is as follows, and the running logic is as follows:

[0052] First, the initialization module opens the specified three-dimensional point cloud data file, loads the data, and performs optional preprocessing that may affect the calculation accuracy and efficiency of the subsequent steps (this process includes noise point removal, downsampling, coordinate space normalization, and connectivity enhancement, etc.).

[0053] Next, the ground segmentation module receives the preprocessed data from the initialization module, performs segmentation of corn plant groups and ground, and after segmentation:

[0054] 1) provides group corn plant point cloud data after removing ground points to the single plant recognition module;

[0055] 2) simultaneously calls the result interaction module to display the intermediate results of the segmentation process;

[0056] 3) and optionally saves the group corn plant point cloud data to the local;

[0057] After that, the single plant recognition module receives the output data from the ground segmentation module, performs segmentation and identification of corn single plant point cloud, and after completion:

[0058] 1) provides segmented and identified single corn point cloud data to the skeleton classification module;

[0059] 2) simultaneously calls the result interaction module to display the intermediate segmentation results;

[0060] 3) and optionally saves the single corn point cloud data to the local with identification information as the file name;

[0061] 4) In addition, the single plant recognition module also supports loading group corn plant point cloud data without ground from the initialization module, and independently running by skipping the ground segmentation module;

[0062] Then, the skeleton classification module receives the output data from the single plant recognition module, performs skeleton calculation and point classification of single corn point cloud, and after completion:

[0063] 1) Provide the classified single corn skeleton classification point cloud data to the organ labeling module;

[0064] 2) Call the result interaction module to display the intermediate classification results at the same time;

[0065] 3) And optionally save the single corn skeleton classification point cloud data to the local;

[0066] 4) In addition, the skeleton classification module also supports loading the segmented and identified single corn point cloud data from the initialization module, and independently runs by skipping the single plant recognition module;

[0067] Finally, the organ labeling module receives the output data from the skeleton classification module, performs organ classification processing on the single corn point cloud, and after completion:

[0068] 1) Call the result interaction module to display the intermediate classification results;

[0069] 2) And optionally save the single corn organ classification point cloud data to the local;

[0070] In addition, the organ labeling module also supports loading the single corn point cloud data and the corresponding skeleton classification data from the initialization module, and independently runs by skipping the skeleton classification module.

[0071] Figure 3 For field corn planting plan view, under similar planting mode, the inventor has verified the actual effect of the algorithm, which can meet the requirements of subsequent data processing such as phenotype detection.

[0072] Figure 4 For ground and plant automatic segmentation algorithm view, from the original data Figure 4 (a) to layered and key layer data Figure 4 (b) is obtained, then the center position point of each plant is obtained by fitting clustering Figure 4 (c), then the final segmentation result is obtained according to the center position by density growth Figure 4 (d).

[0073] Figure 5 For group segmentation single corn point cloud and its bit marking algorithm view, the original group plant data Figure 5 (a) as input, XY plane projection is obtained by projection Figure 5 (b), after morphological processing of two-dimensional image, the boundary contour information of plant is extracted Figure 5 (c), it is extended to three-dimensional coordinate system, and the occluded subset existing in projection overlap Figure 5 (d) is tomographic and connected structure is established Figure 5 (e), the complete non-occluded plant is analyzed Figure 5(f), and the aforementioned non-occluded subset is combined to form a single plant segmentation result and label information is generated according to the coordinate position Figure 5 (g).

[0074] Figure 6 Figure 1 is a schematic diagram of a single corn plant skeleton point cloud stem and leaf organ segmentation and classification algorithm, which utilizes the original plant data obtained by segmentation Figure 6 (a) The open-source KD-Tree algorithm is used to obtain plant skeleton point cloud information Figure 6 (b), and then the connection characteristics and local structure information of each skeleton point are analyzed to divide the skeleton points into middle points Figure 6 (c) and connection points Figure 6 (d) After the point-level classification division, the skeleton is classified Figure 6 (e).

[0075] Figure 7 Figure 2 is a schematic diagram of a single corn plant point cloud organ segmentation algorithm that fuses skeleton information, which utilizes single original data Figure 7 (a) and skeleton point classification information Figure 7 (b), which is superimposed Figure 7 (c), which divides the point classification of the whole plant in the nearest neighbor calculation manner, thereby realizing the classification representation of the plant organs Figure 7 (d).

[0076] Because the three-dimensional point cloud data has dense and sparse regions, especially the connection between the corn stem and the ground has obvious density transition, so the density layering technology can be considered to realize the segmentation of the corn stem and the ground in the three-dimensional point cloud. However, through trial, it is found that in the actual field environment, due to the unevenness of the ground and the influence of factors such as germplasm, fertility, moisture, and light, the growth of corn plants has certain differences, and the stem and ground segmentation points of all plants are not on the same horizontal layer. Therefore, the method of simply realizing the stem and ground segmentation through density layering has certain inaccuracy and is prone to missegmentation. Manual segmentation using existing software not only requires multiple direction viewing and positioning, but when the plants are densely planted, the internal plants cannot be accurately segmented.

[0077] To address this issue, this invention proposes a method that, based on longitudinal (Z-axis) stratification, combines density and height constraints to first determine the layer containing all plant stems. Then, after projecting this layer in two dimensions, the neighborhood threshold is dynamically adjusted for each projection point by finding its nearest neighbor and averaging the values. Clustering is then performed using the DBSCAN clustering algorithm (Martin Ester et al., 1996) to determine the plant center points. The number of plant clusters corresponds to the number of center points. Next, these center points are back-projected onto three-dimensional data coordinates and used as seed points for density growth towards the soil. When the density growth rate exceeds 1% of the global density mean, the stem is considered to have "contacted" the soil, and the segmented region is achieved. This growth process is executed cyclically for each individual plant, avoiding mis-segmentation of some plants from the ground due to using a uniform standard.

[0078] To segment the point cloud data of each plant from the plant population after removing the ground point cloud, this invention considers two main steps: determining the distribution range (XY) of each plant's point cloud through two-dimensional projection, and extending the two-dimensional distribution range to three dimensions (XYZ) for plant point cloud segmentation. Specifically, the three-dimensional point cloud data of the maize population is first projected onto a two-dimensional plane (XY plane) to form a projected image. Through density statistics, morphological operations (such as opening or closing operations), and connected component extraction, the distribution area of ​​each plant is identified. In particular, areas with overlapping leaves are treated as a single occluded area, and their XY boundary range is extracted. This step requires that the input point cloud has undergone coordinate correction, the XY plane is parallel to the ground, and the positive Z-coordinate is vertically upward. Two custom-defined triangular structuring elements are used in this step for region closure.

[0079] (1)

[0080] (2)

[0081] Next, to address the issue of leaf occlusion in the XY projection when multiple corn plants have overlapping or intersecting leaves in space, the method of this invention, based on the boundary range of the occlusion area on the XY plane and combined with a preset height range (Z coordinate range), crops a local subset of the point cloud in three-dimensional space, constructing a three-dimensional point cloud cube region containing one or more occluded plants. The extracted local point cloud is then layered along the Z-axis height direction (e.g., every 5 cm or 10 cm as a height slice), and the point cloud within each layer is projected again onto the XY plane in two dimensions to extract the point cloud distribution structure of that layer. Image connected component analysis or two-dimensional clustering methods are used to identify the possible leaf structures of multiple plants within that height layer.

[0082] Subsequently, the segmentation results of each height layer are analyzed for spatial structure connectivity to determine their consistency in the vertical direction. The spatial clustering method or the geometric center trajectory comparison method of connected regions is used to merge the clustering results of each layer belonging to the same plant, and the determination of the branching of the occluded plant is completed. Further, the point cloud density, principal direction vector (based on local principal component analysis PCA), normal vector or curvature, etc. spatial features can be introduced to assist in judging and optimizing the clustering of the occluded region point cloud, and to enhance the discrimination ability of the boundary fuzzy area.

[0083] Finally, the three-dimensional cubic space segmentation information with additional Z coordinates is combined with the plant population to segment the individual plants, and the complete three-dimensional point cloud extraction of the corn individual level is realized. When visualizing the segmentation effect, the following matrix is used to connect the boundaries of the cubes to form the visualization effect of the cubic bounding box.

[0084] (3)

[0085] The related method fully utilizes the hierarchical differences of plant structures in the Z direction, has good adaptability to corn populations in dense planting environments, effectively solves the XY projection misclassification problem caused by leaf occlusion while improving the segmentation accuracy, and can also provide accurate data support for subsequent individual trait extraction and phenotype analysis. Moreover, the method can also be combined with existing two-dimensional segmentation frameworks without the need for additional hardware support, ensuring execution efficiency.

[0086] Since each segmented plant needs to be saved as a separate point cloud file, in order to clearly label the original position of the plant, corresponding label information needs to be generated, for example: "2-3.ply" information indicates the third corn in the third column (this step needs to be combined with the actual planting plan, and the number of planting rows and columns needs to be clear, and in the experiment, the planting scale is 4*15). The plant information is sorted in ascending order according to the Y coordinate value, and then divided into groups according to the number of rows. The X coordinate is sorted within each complete group, and if an incomplete group (i.e. the number of plants in the group and the number of rows are inconsistent) appears, it needs to be handled separately.

[0087] After obtaining the single corn plant point cloud, in order to be able to perform more detailed processing, it is necessary to classify and divide each part or organ of the plant. The present invention selects the geometric connection relationship of the plant skeleton and its local structure as the basis for division, and divides the entire plant based on the division of each part of the skeleton according to the local features. Methods known in the art (such as Laplace shrinkage, KD-Tree neighborhood indexing, Dijkstra path extraction, etc.) are used to realize functions such as skeleton extraction, neighborhood construction, path analysis, etc., to realize the calculation of the plant point cloud skeleton.

[0088] After the skeleton classification, the local point cloud classification task of the corn plant organs is realized by the near neighbor search of the skeleton points, and then the classification of the whole plant is realized. Specifically, it includes two steps: preliminary coarse segmentation and fine segmentation based on comprehensive evaluation function. The input plant point cloud is subjected to coarse-grained organ extraction to obtain multiple initial leaf point sets , and the point set to be allocated is extracted (this part may not be correctly attributed in the preliminary step due to curling, occlusion, dense overlap, etc.). For each point to be allocated , each initial leaf point set is traversed, and the following feature items are calculated: Euclidean distance (minimum distance of the point to the center or boundary of the leaf), surface projection distance (projected distance of the point on the local surface fitted in the leaf area), normal vector consistency (mean of the normal angle between the point and the points in the leaf area), topological connectivity (whether the point is connected with the leaf point set through the adjacency graph path). The confidence score function is:

[0089] (4)

[0090] The final comprehensive score function is:

[0091] (5)

[0092] Each point is allocated to the leaf with the lowest score; if the score of the point is lower than the preset threshold, it is marked as low confidence; after all the initial allocation is completed, the low-confidence points are re-judged, or re-allocated through global optimization (such as minimum cut optimization).

[0093] To adapt to the analysis needs of point cloud data of different growth stages of crops such as corn, the application introduces a cross-time segmentation result fusion mechanism. The whole plant skeleton structure, ground height alignment or ICP method is used to register the multi-time point cloud to the common coordinate system. For each pair of adjacent time phases, a leaf mapping is constructed, and the IoU value between the leaf point sets is calculated; the main direction angle is compared; if the consistency score is higher than the threshold, it is considered as the same leaf structure. The segmentation point attribution judgment of the current time phase is guided by the leaf structure of the previous time phase, forming a time-aware continuity analysis capability, reducing the time sequence jump, misjudgment and drift.

[0094] In summary, the present application has the following 6 core processing methods and functions:

[0095] 1. Automatic segmentation of ground and plants: Through segmentation methods based on layer density changes, height threshold, two-dimensional clustering and back projection, combined with the uneven distribution of plant point cloud density and the structural characteristics of the plant itself, the approximate layer containing all plant stems is first located, and then the key position of each plant stem is located through contour and clustering analysis. The key position is used as the growth point, and the stem grows towards the ground. Finally, the automatic segmentation of ground and plants is realized. This method can adapt to complex ground environment and incomplete and uneven plant development. At the same time, it can also deal with the situation of using foam boxes, flowerpots and other potted plants.

[0096] 2. Segmentation of single plants in the population: Based on a specific planting method, the overall distribution range is determined according to the plane area of plant growth, and then the longitudinal height is superimposed. The plant is segmented in the form of a three-dimensional cubic box surrounded by the contour frame. This method can accurately segment single plants in a relatively dense crop population.

[0097] 3. Predefined label marking of growth position: Based on a specific planting method, the cubic coordinates of the segmented plants are sorted based on position, and then the system automatically assigns a label defined by rows and columns to each single plant according to the arrangement direction, facilitating subsequent classification and analysis. Because this method is based on high-precision segmented cubic information and there is no position change such as movement in the field, it can be applied to the entire life cycle of crops to dynamically monitor the distribution and growth of plants.

[0098] 4. Segmentation and point-level classification of organs such as stems and leaves: Using the spatial topology and geometric features of point cloud, based on the classification of skeleton points, two steps are taken: coarse segmentation based on the distance between plant point cloud and skeleton classified point cloud, and fine segmentation based on multi-index comprehensive distance scoring function and confidence. This method realizes the optimal attribution division of organs such as stems and leaves of corn crops and labels each point cloud data point. If there are multiple growth period data, the time consistency score can be calculated by the IoU overlap rate between leaf point sets at different time points and the angle similarity between the main directions of leaves at different time periods, which not only increases the accuracy of segmentation but also utilizes time series data to construct cross-period plant organ segmentation consistency evaluation.

[0099] 5. Modular design and processing: The system provides modular processing functions, and users can start the entire process with simple operations. At the same time, each module can be called independently, and intermediate results and processing results can be saved as files separately for subsequent data analysis and deep learning model training.

[0100] 6. Tagged data generation: Through an efficient processing flow and accurate segmentation annotation, the system can generate high-quality tagged data for deep learning and neural network models to support crop phenotype analysis tasks.

[0101] Compared with the traditional manual annotation method, the method flow of the application has high automation degree, significantly improved processing efficiency, and standard and normative output format. Without the aid of complex models, the combination of traditional methods can realize accurate segmentation and classification of single plants and organs, providing high-quality data support for phenotype research, especially suitable for the demand for large-scale and accurate annotation data in deep learning and intelligent analysis tasks.

[0102] The ground and plant method of the application realizes relatively accurate segmentation of plants and ground under non-single standard according to the different hierarchical point cloud densities of corn crop populations and the differences in corn crop stem structures, clusters, locates and calculates the number of plants, realizes single crop segmentation and identification according to the spatial ordering of point clouds based on the actual planting row and column positions of plants, automatically generates label names according to coordinate positions, realizes single plant numbering and spatial position annotation without manual intervention. The single plant organ classification method supports point-level classification, that is, each data point in the three-dimensional point cloud is classified, which can realize the identification and automatic annotation of corn stems, leaves and other organs, and has higher efficiency and accuracy compared with the traditional manual marking method. The processing result can be directly used to train a deep neural network model, high-throughput extract phenotype information of field corn crops, and is widely applicable to crop semantic segmentation, structure modeling, growth monitoring and intelligent analysis and other agricultural intelligent application scenarios, and has the advantages of automatic processing flow, modularization, high precision and high efficiency of segmentation results, visualization and easy interaction of the segmentation process.

[0103] The application realizes a series of processes from field population point cloud data to ground segmentation, single plant segmentation, organ segmentation and class labeling. According to the different hierarchical densities of three-dimensional point clouds of field corn populations and the growth differences of stem structures, the application realizes relatively accurate segmentation of plants and uneven ground under non-single standard through layer positioning and local density plant number calculation; according to the obvious view angle dependent characteristics of the leaf shading of the corn population, the application innovatively associates the key features of adjacent layers through hierarchical local feature correlation to reconstruct the complete plant structure with continuous layer spatial information, effectively solving the problem of misclassification of organs such as leaves caused by shading confusion factors. According to the geometric connection relationship of single corn skeletons, the classification of the skeletons is extended to the organ classification of the whole plant. At the same time, the output result is the class information of the data points, which can be directly applied to phenotype feature calculation and deep learning model training.

[0104] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0105] The foregoing description of the disclosed embodiments enables one skilled in the art to make or use the application. Numerous modifications of those embodiments can be apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of the innovation falling outside the spirit and scope of the application. Therefore, the application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional phenotyping segmentation and annotation system for field corn crops, characterized by, The method comprises the following steps: An initialization module is used to perform the opening, loading and preprocessing of field corn point cloud data files; A ground segmentation module is used to perform the automatic segmentation of corn plant groups and the ground. The ground segmentation module is used to perform the automatic segmentation of corn plant groups and the ground. Specifically, first, the point cloud data is uniformly divided into layers based on the longitudinal height. Then, only the middle and lower layers are analyzed for density based on the growth structure of the corn plants, and the layer with the minimum plant density is determined as the stem positioning layer. Then, the point cloud is projected from above to obtain a two-dimensional projection point set. The obtained two-dimensional projection point set is subjected to erosion, expansion edge calculation and center clustering operations to locate the stem center point. Finally, the projection is reversed to the three-dimensional coordinate system, and the determined stem center points are used as growth points. Each plant independently performs region growing in the longitudinal downward direction. When the density change exceeds the average value or the adaptive threshold, it is considered that the stem point has contacted the ground point, and the growth is stopped at this time. A single plant recognition module is used to perform the accurate segmentation of single plants in the plant group and generate identification information. A skeleton classification module is used to perform the skeleton calculation and data point category division of single corn point clouds. An organ labeling module is used to perform the stem and leaf organ structure division and labeling of single corn point clouds. A result interaction module is used to perform the visualization of the output data of the key steps of the above modules and the saving of the projection, segmentation, labeling and classification result data output by the modules.

2. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The initialization module supports the opening and loading of three-dimensional point cloud data files in PLY, PCD, TXT and XYZ formats, and selectively performs integrated processing such as noise filtering, point cloud downsampling, coordinate space normalization and connectivity enhancement. The initialization module includes a data sharing interface for providing corresponding point cloud data to other modules.

3. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The single plant recognition module is used to accurately segment single plants in the corn plant group point cloud data. Specifically, the three-dimensional point cloud data is projected onto a two-dimensional plane, and density statistics, morphological processing and connected domain analysis are performed to effectively identify the minimum independent distribution area of each plant on the plane. Combined with the height information of the original point cloud, these plane areas are expanded into corresponding three-dimensional cubic units, thereby limiting the spatial boundary of each plant and providing initial constraints for single plant segmentation. If overlapping, occlusion or intersection of leaf blades is detected, the related cubic units are further merged and labeled as an occluded subset. The occluded subset is subjected to height layering and layer-by-layer feature analysis to decompose the overlapping contours, construct a spatially structured plant body based on the internal relationship of the features of each layer, merge the hierarchical features of the same plant, and determine the single plant. According to the actual field planting orientation, the plant positions are automatically calibrated after being sorted in a row-column regular pattern in the XY coordinate axis direction, and an ordered point cloud file name format with labels is generated.

4. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The skeleton classification module performs structure constraint shrinkage processing and spatial neighborhood search on the input single corn point cloud data to obtain a skeleton point set for representing the main stem structure and extract skeleton information representing the spatial structure of the corn plant; based on the skeleton information, a topological graph structure is constructed, and the local geometric connectivity of each skeleton point is divided into different topological categories, including an end point connected to only one adjacent point, an intermediate point connected to two adjacent points, a branching point connected to three or more adjacent points, and an isolated point without adjacent points; Based on the topological categories, the single main stem + multi-branch topological law and the significant organ distribution characteristics of the corn plant are used to form the stem main axis with the longitudinally continuous intermediate points and a small number of branching points, the branching points as the starting position of the leaves, and the leaf skeleton formed by extending outward until the end point, and the isolated points corresponding to noise or broken areas; An organ hierarchical model based on a graph structure is constructed, in which the nodes represent the key points of the organs, and the edges represent the organ main axis paths, thereby completing the classification of the single corn plant skeleton point cloud.

5. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The organ labeling module realizes the coarse determination of the categories of different points based on the classification results of the skeleton classification module or based on the existing skeleton data with classification information, realizes preliminary organ segmentation, and obtains the point sets of the initial stems and leaves and the point set to be allocated; for the point set to be allocated, the Euclidean distance, surface projection distance, normal vector consistency, topological connectivity, and allocation confidence parameters of each initial leaf are calculated, and then the point to be allocated is attributed to the optimal category point set based on the comprehensive distance scoring function, and after all the allocation is completed, the points with a confidence lower than a preset threshold are executed for secondary allocation correction.

6. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The result interaction module supports the display and effect evaluation of the output results of the key steps of the projection, clustering, segmentation, labeling, and phenotype parameter extraction of each module by drawing two-dimensional or three-dimensional graphics; At the same time, each module supports the organization and saving of the specified output data in the standard PLY format or the TXT format containing XYZ coordinates and classification information, and the labeled point cloud data is directly used for the training input of the deep learning models PointNet and PointNet++.

7. The three-dimensional phenotyping segmentation and annotation system for field corn crop of claim 1, wherein, The single plant recognition module, the skeleton classification module, and the organ labeling module call the initialization module to load the required data independently, support the linkage calling, and independently run in the case that the single plant recognition module, the skeleton classification module, and the organ labeling module do not execute and do not receive the required data.

Images