Corn phenotype parameter extraction method based on point cloud data

Abstract

The invention discloses a corn phenotype parameter extraction method based on point cloud data. The method comprises the following steps: step 1, constructing an experimental data set; step 2, corn point cloud segmentation; and step 3, corn phenotype parameter extraction. The method constructs a corn point cloud data set, deeply explores a segmentation technology of a corn population and organs thereof based on three-dimensional point cloud data, can realize complete extraction and segmentation of the population and a single corn plant, accurately extracts field corn phenotype parameters, is not only suitable for phenotype parameter extraction of a single corn plant, but also suitable for phenotype parameter extraction of a corn population, and improves the extraction efficiency of the corn population. The method shows high applicability and accuracy, provides an effective technical means for corn phenotype analysis, and can provide an automatic, efficient and low-cost solution for accurate measurement of crop individual phenotype information.

Description

Technical Field

[0001] The invention relates to a method for acquiring corn phenotypic parameters, and in particular to a method for extracting corn phenotypic parameters based on point cloud data. Background Art

[0002] In the development of the modern corn industry, it is necessary to combine genomics and phenomics to extract the genetic structure of key phenotypic traits, so as to screen and cultivate corn crops with high-quality traits. Efficient and non-destructive methods for obtaining corn phenotypic parameters are crucial for crop breeding research. It can not only provide basic data and method support for breeding work, but also significantly improve breeding efficiency.

[0003] The extraction of maize plant phenotypic parameters is a key link in precision agriculture research, which involves the precise measurement of the characteristic parameters of the whole plant and its organs. Miao Yanlong et al. used ground-based laser radar to collect three-dimensional point cloud data of maize in a field environment, and realized an effective method for plant height measurement through ground point segmentation and highest point extraction technology. Zong Ze et al. proposed an effective point cloud clustering and denoising method to address the challenges faced by three-dimensional point clouds in the maize growth cycle in estimating phenotypic parameters. On this basis, they further used skeleton extraction technology to accurately measure parameters such as leaf length, leaf inclination, leaf top length, leaf azimuth, leaf height and plant height of maize. Zheng Chenxi et al. provided a new perspective and method for the precise calculation of phenotypic parameters of wheat plants. The method they proposed realized the automated extraction and measurement of multiple key phenotypic parameters of wheat plants through point cloud processing algorithms. These parameters cover multiple aspects such as length and angle, providing rich data support for a comprehensive understanding of the growth status of wheat plants. Liu et al. used ground-based laser radar technology in their research to collect point cloud data of maize in pots. By using the regional growing algorithm and Euclidean clustering method, the accurate segmentation of corn stem and leaf tissue was successfully achieved. In addition, they further obtained key physiological indicators such as leaf length, width, area and angle, providing important data support for corn growth research. Although the technology for collecting three-dimensional point cloud information of crop plants in field environments has developed, it still faces multiple challenges such as low collection efficiency and susceptibility to light environment. In order to obtain crop phenotypic parameters more accurately, achieving efficient and stable extraction methods is still the focus and difficulty of current research, which requires further exploration and optimization. Summary of the invention

[0004] In order to improve the automation and intelligence of crop phenotypic extraction and realize the accurate extraction of phenotypic parameters of multiple periods, multiple habitats and multiple organs, the present invention provides a method for extracting corn phenotypic parameters based on point cloud data. This method constructs a corn point cloud dataset, deeply explores the segmentation technology of corn populations and their organs based on three-dimensional point cloud data, and realizes the application of point cloud technology for extracting phenotypic parameters at the plant population scale and organ scale.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A method for extracting corn phenotypic parameters based on point cloud data comprises the following steps:

[0007] Step 1: Experimental dataset construction

[0008] Step 1.1: Multi-view sequence data collection of corn plants

[0009] Step 1.1.1 Use a drone equipped with an RGB camera to collect multi-view sequence data of corn at the three-leaf and six-leaf stages in a field environment. Adjust the height and angle of the drone to ensure that a complete image with a high overlap rate is obtained;

[0010] Step 1.1.2 Build a laboratory acquisition platform to obtain multi-view images of potted corn at the three-leaf and six-leaf stages;

[0011] Step 1.2: 3D reconstruction and preprocessing of corn plants

[0012] Step 1.2.1 Use the watershed algorithm to remove the background of the video frame, and reconstruct the indoor single corn 3D point cloud model with color information through the motion recovery structure (SfM) algorithm. Based on the generated indoor single corn 3D point cloud data, down-sample it through the improved voxel grid method, separate the crop and flower pot point cloud from the original point cloud through the color threshold method, and obtain the corn point cloud with the growth direction in the positive direction of the Z axis through point cloud coordinate scale correction and coordinate transformation. Then use straight-through filtering and color filtering to extract the corn point cloud and flower pot point cloud data to construct an indoor single corn point cloud segmentation dataset;

[0013] Step 1.2.2 Reconstruct a 3D point cloud model of corn colonies with color information through the structure-from-motion algorithm. Based on the generated 3D point cloud data of corn colonies, extract the region of interest, filter the point cloud, and perform improved voxel grid downsampling on the original point cloud. Separate the crop and ground point clouds through the color threshold method. Construct a point cloud segmentation dataset of corn colonies in the field through point cloud coordinate scale correction, soil plane fitting, and point cloud rotation.

[0014] Step 2: Corn point cloud segmentation

[0015] Step 2.1: Use the improved Quickshift algorithm to segment the field group corn point cloud:

[0016] Step 2.1.1 Color space conversion

[0017] Process the extracted crop point cloud group point cloud dataset and convert the RGB color to the HSI color space;

[0018] Step 2.1.2 Calculate point density

[0019] A KD-Tree data structure is established for the point cloud data set, and the density of each point is calculated according to the weighted density formula;

[0020] Step 2.1.3 calculates the cluster center of the point cloud on the projection surface;

[0021] Step 2.1.4 Cluster the remaining points in three dimensions

[0022] After calculating the cluster core, traverse all points. If they are not in the cluster core, perform 3D QuickShift clustering on the point and move each sample point to its nearest neighbor with higher density until it reaches a cluster core.

[0023] Step 2.2: Perform cloud organ segmentation on individual corn points based on cylindrical fitting and Euclidean clustering algorithm:

[0024] Step 2.2.1: For the point cloud data of a single corn plant, a random sampling consensus algorithm is used to perform cylindrical fitting of the stalk, thereby effectively separating the corn leaves and stalks;

[0025] Step 2.2.2: For the point cloud data with the stem removed, use the Euclidean clustering segmentation algorithm to accurately segment individual leaves and complete the fine segmentation of corn organs;

[0026] Step 3: Extraction of corn phenotypic parameters

[0027] Step 3.1 Based on the single corn point cloud data after organ segmentation, the key parameters related to corn breeding and growth, such as plant height, stem diameter, stem height, and leaf area, are accurately and losslessly extracted: the plant height information is obtained by calculating the vertical distance from the top of the corn plant to the soil plane; the stem height is extracted using the cylinder fitting algorithm; the major and minor axis diameters of the stem are measured using the ellipse fitting algorithm; and the leaf area is extracted using the greedy projection triangulation algorithm;

[0028] Step 3.2 For the segmented field group corn point cloud data, the layered calculation method is used to obtain the leaf area index, and the mean of parameters such as plant height and stem thickness of all plants in the group is calculated.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] (1) A field and indoor corn point cloud dataset was constructed. The present invention built a field and indoor corn multi-view data acquisition platform. The multi-view data was imported into the Pix4Dmapper software with the SfM algorithm as the core, and three-dimensional reconstruction was performed to generate a three-dimensional point cloud model of the corn. The corn point cloud was preprocessed using open source libraries such as Open3D and PCL and CloudCompare software, including processing such as region of interest extraction, point cloud filtering, and coordinate scale correction. The point cloud was downsampled using an improved voxel grid method, retaining the original structural information of the point cloud. The separation of corn plants and soil background was achieved through a color threshold-based segmentation method, and a point cloud segmentation dataset was successfully constructed.

[0031] (2) By studying the group and organ segmentation algorithm of the collected three-dimensional point cloud data, the extraction of individual plants and organs was achieved. Based on the group corn crop point cloud data, the density-based QuickShift clustering method was improved, and a density clustering core clustering method with additional color weights was proposed. The algorithm is based on the preprocessed corn crop point cloud, and optimizes the clustering density calculation process by introducing color weights. The point cloud is projected to the horizontal plane to determine the density clustering core. Subsequently, the remaining points are clustered and segmented by QuickShift in three-dimensional space. The experimental results show that the clustering accuracy of the method proposed in the present invention in field corn crops reached 96.82%, which is significantly improved compared with other clustering methods, and the accurate extraction of single corn point cloud data is achieved. At the same time, the organ segmentation of single corn plants is achieved by the cylindrical fitting method and the Euclidean clustering algorithm, and the stalk point cloud and leaf point cloud are obtained, which provides reliable data support for the subsequent extraction of corn phenotypic parameters.

[0032] (3) Based on the corn point cloud data after organ segmentation, the parameters closely related to corn breeding and growth, such as plant height, stem thickness, stem height, and leaf area, are accurately and losslessly extracted. For group corn data, the present invention uses a hierarchical calculation method to obtain the leaf area index, and statistically obtains the mean of parameters such as plant height and stem thickness of all plants in the group. After comparative analysis of the extracted values ​​and the measured values, the relative coefficients of plant height, major axis stem thickness, and minor axis stem thickness of indoor single corn are all above 0.9, achieving a high extraction accuracy.

[0033] (4) The method of the present invention can achieve complete extraction and segmentation of groups and individual corn plants, and accurately extract field corn phenotypic parameters. It is not only applicable to the extraction of phenotypic parameters of individual corn plants, but also to the extraction of phenotypic parameters of groups of corn. It shows high applicability and accuracy, provides an effective technical means for corn phenotypic analysis, and can provide an automated, efficient, and low-cost solution for accurately measuring individual crop phenotypic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Provide a technical roadmap for the extraction of maize phenotypic parameters based on point cloud data;

[0035] Figure 2 It is a schematic diagram of equipment placement and a top view of the corn multi-view image acquisition system;

[0036] Figure 3 For the improved voxel grid method;

[0037] Figure 4 This is a schematic diagram of the point cloud numbering rules;

[0038] Figure 5 Extract point cloud flow chart based on HSI color space model;

[0039] Figure 6 It is a schematic diagram of point cloud rotation;

[0040] Figure 7 For the improved corn point cloud segmentation process;

[0041] Figure 8 This is a schematic diagram of extracting the vertices of the crop plant canopy;

[0042] Fig. 9 It is a schematic diagram of the distance from a point to a plane;

[0043] Fig.10 This is a schematic diagram for leaf area index calculation;

[0044] Fig.11 The comparison results are as follows: the measured values ​​of corn phenotypes in the field and the automatically extracted values;

[0045] Fig.12 This is the comparison result between the automatically extracted value and the measured value of the single corn phenotype. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0047] The present invention provides a method for extracting corn phenotypic parameters based on point cloud data. The method takes corn as the research object, and in view of the difficulties in crop organ segmentation and the problems existing in phenotypic data acquisition in crop phenotypic acquisition, a systematic study of crop three-dimensional reconstruction, population and organ segmentation, and phenotypic parameter extraction methods is conducted. The present invention can quickly extract key phenotypic parameters such as plant height, stem diameter, stem height, and leaf area, which not only provides strong technical support for the breeding of high-yield corn varieties, but also provides an effective means for the rapid screening of field population phenotypic traits. Figure 1 As shown, the specific steps include:

[0048] 1. Experimental Dataset Construction

[0049] The present invention constructs corn phenotype datasets in two different scenarios: field and indoor. The experimental conditions for collecting the two data are shown in Table 1.

[0050] Table 1 Data collection conditions

[0051]

[0052]

[0053] 1 Construction of indoor single corn dataset

[0054] 1.1 Data acquisition based on image acquisition platform

[0055] The experimental material selected by the present invention is the corn variety Xianyu 335, and the growth period is 96 to 125 days. After the experimental material is selected, the corn seeds are subjected to water bath heating treatment, and the water temperature is controlled at 39 degrees Celsius to promote the germination of the corn seeds. The treated seeds are soaked in flower pots and numbered, and then the growth of the corn is observed. In order to achieve the stem and leaf segmentation of the point cloud of corn plants in multiple growth periods, the point cloud data of 12 pots of corn 25 days (V3 period) and 45 days (V6) after emergence are collected respectively. While collecting pictures, key phenotypic parameters such as plant height, stem thickness, leaf length, leaf width and leaf area are recorded manually.

[0056] 1.2 Data Preprocessing

[0057] 1.2.1 Single corn plant reconstruction method

[0058] Before the potted corn is reconstructed using the Structure from Motion (SfM) algorithm, the background of the video frame is removed using the watershed algorithm. In this algorithm, the potted plant is considered as high ground, while the image background is considered as low ground. The grayscale level of each pixel is sorted from high to low, and then the entire image is filled with the upper left corner pixel as the starting point to form a watershed. By replacing all pixels in the low ground with white, the irrelevant image background is removed.

[0059] 1.2.2 Improved voxel grid downsampling

[0060] The present invention adopts an improved voxel grid downsampling method to divide the initial point cloud data into voxel grids, and replaces other points in the voxel with the nearest points of the centroid to simplify the point cloud data and improve the subsequent processing speed. The main steps include:

[0061] 1) Construct a KD-Tree for the point cloud set P and determine the minimum bounding box of the target point cloud. The length of the bounding box is L x ,Ly ,L z , and then calculate the volume V1 of the bounding box.

[0062] 2) Side length of the voxel grid K is the scale coefficient, α is the scale factor. Finally, divide the point cloud into n small cubes: n = l × w × h, l = ceil (L x / L),w=ceil(L y / L),h=ceil(L z / L), where ceil() is the rounding up function.

[0063] 3) Calculate the center of gravity p0(x0,y0,z0) of each voxel cube and construct a new point cloud set P c .in N1 is the number of samples.

[0064] 4) Traverse the point cloud data, use the KD-Tree structure to quickly find the nearest points of each point cloud center of gravity, and construct these neighboring points into a new set P e The process of downsampling in the voxel grid method is as follows Figure 3 shown.

[0065] 1.2.3 Coordinate scale correction of potted plant point cloud model

[0066] The present invention successfully segmented the flower pot point cloud from the original point cloud by setting the HSI color space threshold. Then, the main program analysis algorithm was used to calculate the normal vector of the soil plane of the flower pot. Based on this, the entire original point cloud was rotated to the positive direction of the Z axis and projected onto the horizontal plane. Subsequently, the minimum closed circle algorithm was used to calculate the diameter d of the flower pot. ture and the reconstruction pot radius d MVS , and obtain the correction ratio γ, as shown in formula (1). The maximum cross-sectional diameter of the flowerpot is fitted using a random sampling consensus algorithm to restore the true scale of the plant model.

[0067]

[0068] In each iteration of the flowerpot section circle fitting algorithm, the least squares method is used to estimate the model parameters. First, three sample points are randomly selected from the point cloud, and the center O and radius r are obtained by fitting these sample points. Subsequently, the point cloud data is traversed and the distance from each point to the fitted circle model is calculated. If the distance is less than the set threshold t (the present invention takes 0.05), it is marked as an inlier, otherwise it is regarded as an outlier. The inliers and their number obtained in each iteration are recorded, and the circle model parameters with the largest number of inliers during N iterations (the present invention takes 1000) are regarded as the final fitting result. Since the flowerpots of some corn pots may be deformed, the present invention reduces the error in measuring the diameter of the flowerpot by fitting the section circle multiple times. Using the true diameter d of the flowerpot true (12.5cm) and the mean diameter of the section fitting d MVS The scale factor γ shown in the calculation formula is then multiplied by the point cloud coordinates and the scale factor one by one to achieve scale correction of the corn 3D model.

[0069] 1.2.4 Pass-through filtering

[0070] Corn point cloud and flower pot point cloud data are extracted through pass filtering and color thresholding.

[0071] 2 Construction of field population corn dataset

[0072] 2.1 Data acquisition based on multi-view image sequences of UAVs

[0073] Under windless weather conditions, corn was selected at the three-leaf stage (V3) and six-leaf stage (V6) of growth, and was filmed by a DJI Phantom 4 drone equipped with a camera. The camera has 12.4 million pixels and a maximum resolution of 5472×3648, ensuring clear video and image data captured during movement. The flight path was set to a circular trajectory to fly around the plant, and each flight span was 30 seconds. Frames were selected from the video sequence as input images. There were 110 images used for reconstruction, with a resolution of 3840×2160 pixels and a size of about 10MB each. In order to facilitate the accurate evaluation of the subsequent phenotypic algorithm measurement values, after taking the pictures, the key quantitative trait data of each corn plant, including plant height, stem thickness and stem height, were manually recorded using measuring tools such as rulers and vernier calipers. At the same time, the leaf area index of the area was measured using the leaf area index meter LAI-2200C.

[0074] 2.2 Data Preprocessing

[0075] 2.2.1 3D reconstruction based on structure from motion

[0076] (1) Data collection: Use drones or other aerial photography equipment to collect high-resolution images, ensuring that the collected images cover every corner of the target area and have sufficient overlap;

[0077] (2) Set project parameters: Set project parameters in Pix4Dmapper, including coordinate system, ground resolution, etc.

[0078] (3) Image processing: The software processes the imported images, including image matching and feature extraction. This process automatically creates associations between images and their positions in three-dimensional space.

[0079] (4) Point cloud generation: Based on the processed image, the software will generate a point cloud model to represent the surface contour of the target area.

[0080] The processing time for the entire process is around 5 minutes, depending on the number of images involved and the density of the canopy.

[0081] 2.2.2 Region of Interest Extraction

[0082] First, the 3D point cloud processing software CloudCompare is used for preliminary data preprocessing, aiming to accurately extract the target point cloud of the area of ​​interest. To facilitate subsequent analysis and processing, the present invention will number the reconstructed point cloud according to the environment and period, using a four-digit definition rule. Among them, the first digit represents the environment number, where 1 represents the field group corn data, and 2 represents the laboratory environment single corn data; the second digit represents the period number, where 3 and 6 represent the V3 period and V6 period of corn growth; the third and fourth digits represent the number of the single corn in the point cloud, and are numbered from right to left and from top to bottom. For specific numbering rules, see Figure 4 For example, the point cloud group number in V6 is 1600, while in V3 Figure 4 The corn plant shown in (b) is numbered 1302. The point clouds of potted corn are numbered from 00 to 12.

[0083] 2.2.3 Voxel-based point cloud downsampling

[0084] The present invention selects a voxel-based point cloud downsampling method. Table 2 shows the application effect of the voxel-based downsampling method in the simplification of corn population point clouds at different periods. By comparing the number of original and simplified point clouds, it can be clearly observed that the number of point clouds is significantly reduced, which provides strong support for improving data processing efficiency.

[0085] Table 2 Field corn point cloud simplification results

[0086]

[0087] 2.2.4 Crop group target point cloud extraction method

[0088] The strategy of converting RGB space to HSI space is adopted: by using the H component in the HSI color space to extract color, the point cloud data of the crop group can be extracted more effectively, thereby improving the accuracy and reliability of data extraction. The crop point cloud extraction method based on the HSI color model distinguishes soil and crops by comparing different color components, especially the H component, and then effectively extracts the point cloud data of the target plant. Figure 5 The figure is a flow chart of point cloud extraction based on HSI color model. The results of the point cloud data extraction experiment are shown in Table 3. The relative accuracy of the target extraction method based on HSI color model proposed in the present invention and the manual extraction of point cloud data are both above 95%. It takes a long time to manually extract crop point clouds, but by using the method described in the present invention, the extraction process of crop group point cloud data can be completed in only about 20 seconds, which greatly improves the processing efficiency.

[0089] Table 3 Point cloud data extraction results

[0090]

[0091] 2.2.5 Coordinate scale correction of group corn point cloud model

[0092] The actual row spacing H of corn plants in the field real Relative to the row spacing H of the reconstructed corn plants reconstructed The scaling coefficient is obtained, and the calculation formula for the row spacing of corn crops in the field is shown in formula (2):

[0093]

[0094] Among them, H real is the actual row spacing of corn plants, H reconstructed To reconstruct the row spacing of corn plants, R1 is the scaling factor.

[0095] 2.2.6 Soil plane fitting

[0096] The soil plane is fitted using the RANSAC algorithm, and the fitted plane equation and normal vector are shown in Table 4. The fitted plane can represent the surface soil and provide a basis for subsequent data processing and analysis.

[0097] Table 4 Soil plane fitting results

[0098]

[0099] 2.2.7 Point Cloud Rotation

[0100] In the CloudCompare software, the rotation matrix of the soil point cloud plane to the horizontal plane is calculated, and the entire point cloud is rotated to the positive direction of the Z axis to facilitate subsequent operations. In the process of converting the camera coordinates to the origin coordinates (actual plane coordinates), there are three main steps in the conversion of the spatial three-dimensional coordinate system:

[0101] (1) Calculate the normal vector of the plane fitted by the ground point cloud;

[0102] (2) Calculate the rotation matrix between the two vectors;

[0103] (3) Rotate to align with the specified reference coordinate system.

[0104] The process of converting the Euler angle from the relative coordinates (camera coordinates) to the origin coordinates (actual plane coordinates) is to first rotate along the x-axis, then along the y-axis, and finally along the z-axis, and finally obtain the origin coordinate system perpendicular to the visual screen. Among them, the rotation angles are α1, β1, and θ1, respectively, so the rotation matrix from the instrument coordinate system to the origin coordinate system is:

[0105]

[0106] The transformation equation is:

[0107] X True =X ground x′y′z′ (4)

[0108] Where: X True is the actual three-dimensional coordinate of the field plane; X ground is the coordinate of 3D reconstruction.

[0109] The point cloud rotation process is as follows Figure 6 shown.

[0110] 2. Corn Point Cloud Segmentation Method

[0111] The segmentation of different monomers and organs of corn is a key step in the automatic measurement of phenotypic parameters. Due to the complex field environment, the corn point cloud presents irregular characteristics, so it is very difficult to segment the corn population in the field. In response to this challenge, the present invention proposes a point cloud segmentation method based on an improved QuickShift algorithm, which can achieve automatic and accurate segmentation of the corn population in the field. Further using cylinder fitting and Euclidean clustering technology, the organ segmentation of the single corn point cloud is completed, which provides an effective technical means for realizing high-throughput phenotypic detection and three-dimensional geometric reconstruction of corn.

[0112] 1. Field corn group point cloud segmentation

[0113] The present invention proposes a QuickShift method suitable for point cloud segmentation of corn crop groups at the farmland scale. The method is optimized and can efficiently process point cloud data of corn crops and achieve accurate group segmentation.

[0114] Define a parameter β (0<β<1) that describes the core of the cluster. If M is is a connected set, then M is the cluster core of the original point cloud. When β→0, the cluster core is the local maximum of the point cloud density; when β→1, the cluster core is the entire point cloud. After adding color weights, the corn stalk point cloud has a larger density value, which meets the definition of the cluster core. The point cloud is projected onto the xoy surface, and the area with the densest points is pre-clustered in two-dimensional space by calculating the cluster core, and then the remaining leaf point cloud is QuickShift clustered in three-dimensional space. In this way, three-dimensional information is taken into account without affecting the normal clustering process of clusters with higher density in two-dimensional space, so that points that are close in three dimensions and points in extremely dense areas in the two-dimensional projection are in the same cluster. Plants that are close in two dimensions but far apart in three dimensions can be distinguished, and the double sowing phenomenon can be cleverly solved. The improved corn point cloud segmentation process is as follows: Figure 7 The specific implementation steps of the algorithm are as follows:

[0115] (1) Color space conversion

[0116] The extracted crop point cloud group point cloud dataset is processed to convert the RGB color to the HSI color space. The calculation formula of the H component is:

[0117]

[0118] Among them, R, G, and B are the red, green, and blue coordinates of a color, respectively, and their values ​​are real numbers between 0 and 1.

[0119] (2) Calculate point density

[0120] Establish a KD-Tree data structure for the point cloud dataset W. For any point x i ∈W, calculate its density p(x i ): Calculate the density of each point according to the weighted density formula, as shown in formula (6):

[0121]

[0122] Where H[x] is the H component value of the HSI space at point x. a is the color parameter, which represents the impact of H[x] on density. k is the number of neighboring points to be found, c is the D is the volume of the unit range, N is the total number of data, R k(x) is the distance from point x to its kth neighboring point, D is the sample dimension, and V is the region volume.

[0123] (3) Calculate the cluster center of the point cloud on the projection surface

[0124] Forming density core: Let λ be the density of a point, and build a graph G(λ) with density values ​​as vertices and distances as edges. Any point x on the graph must satisfy {x∈X [n] :P(x)≥λ}, the edge formed by any two points x and x′ in the graph must satisfy formula (7):

[0125]

[0126] Sort the calculated densities from large to small so that each point P(x i )≥P(x i+1 ). Let λ=P(x i ), set a loop from i=0 to i=n, ​​and calculate the connected component a of the graph G((1-β)λ). If a does not coincide with other cluster centers, then a is a cluster core.

[0127] (4) Cluster the remaining points in three dimensions

[0128] After calculating the cluster core, traverse all points. If they are not in the cluster core, perform 3D QuickShift clustering on the point and move each sample point to its nearest neighbor with higher density until it reaches a cluster core. The specific operation is to first create a new graph G to store the distance information between points. Traverse all points x that are not in the cluster core i , use KD-Tree to find the distance point x i The nearest point x j , and needs to satisfy P(x j )>P(x i ), record x i With point x j The connected edge is E ij , store a line from x in G i to x j The edge E ij Until the end of the loop, all points have an edge leading to the point of increased density, and the end point of moving along these edges must be a cluster core. Retrieve all points in the graph, and assign all points that reach the same cluster core after moving to a cluster core, so that each point has a unique cluster core.

[0129] 2 Corn organ point cloud segmentation

[0130] The present invention realizes corn point cloud stalk segmentation and leaf segmentation based on corn point cloud group segmentation by using a cylinder fitting algorithm and a Euclidean clustering algorithm.

[0131] 3. Automatic Extraction Method of Maize Phenotypic Parameters

[0132] 1. Automatic phenotype extraction method

[0133] 1.1 Plant height extraction method

[0134] When calculating the plant height, the "difference method" is used to calculate the height difference between the highest vertex of the crop canopy and the ground. The present invention obtains the three-dimensional point cloud coordinate data through motion recovery and reconstruction technology, performs outlier filtering on the ground point cloud obtained based on color segmentation in the preprocessing, and performs RANSAC plane fitting to obtain the plane equation of the ground, and then calculates the plant height H based on these data. After obtaining the plane equation of the ground, the present invention completes the plant vertex coordinate extraction algorithm on PyCharm, and its extraction process is as follows Figure 8 After extracting the vertices of each crop plant, the vertical distances from these vertices to the fitted soil plane are further calculated. The plant height can be calculated by the distance from the vertex to the plane. The distance from the point to the plane is calculated as follows: Fig. 9 shown.

[0135] 1.2 Stem height extraction method

[0136] The stem height refers to the vertical distance between the highest and lowest points of the stem. Using the results of cylinder fitting, the cylinder equation of the stem point cloud can be accurately obtained. The axis l of the cylinder can be obtained during the cylinder fitting process. Further determine the coordinates of the highest and lowest points in the cylinder at the axis l, recorded as points A (x1, y1, z1) and B (x2, y2, z2), then the stem height H steam The calculation formula is:

[0137]

[0138] During data collection, the stem data may be missing. To ensure accuracy, the upper half of the broken stem point cloud is fitted to obtain a cylinder, and the internal point cloud is retained. At this time, the distance from the highest point in the cylinder to the soil plane is the stem height. This method can cope with data loss and improve accuracy.

[0139] 1.3 Stem diameter parameter extraction method

[0140] The point cloud data of the stalk at the specified position is intercepted by the straight-through filtering method, and projected onto the horizontal plane for ellipse fitting to obtain the major axis stem diameter and minor axis stem diameter of the corn stalk.

[0141] 1.4 Leaf area extraction method

[0142] The present invention uses a greedy projection triangulation algorithm based on a point cloud library to quickly and effectively mesh point cloud data, providing a reliable basis for subsequent area calculations. The present invention sets the threshold of triangulation based on experience: the maximum plane angle is 45°, and the maximum and minimum angles of each triangle are 10° and 150° respectively.

[0143] The point cloud model after greedy projection triangulation is composed of a series of triangles. The Heron formula in formula (9) is used to calculate the area of ​​these triangles one by one, and then the total area of ​​a single leaf is calculated by formula (10):

[0144]

[0145]

[0146] Where: p i is half the perimeter of the triangulated triangle, a i , b i 、c i is the length of each side of the triangulated triangle, n is the number of triangulated triangles, and i is the face index. S is the area of ​​the corn leaf, S i is the area of ​​the i-th small triangle.

[0147] 1.5 Leaf area index extraction method

[0148] The leaf area index is the ratio of the total area of ​​plant leaves per unit land area to the land area. The leaf projection method and the layered calculation method are used to calculate the leaf area index in the present invention.

[0149] 2 Results and analysis of corn phenotype extraction in field populations

[0150] The present invention selected 63 corn plant point cloud data at two different growth stages as research samples to quantitatively evaluate the accuracy of the phenotypic parameter extraction method. Linear regression analysis was used to explore the correlation between the phenotypic parameters obtained by manual measurement and the parameter extraction values ​​in the present invention, and R 2 , RMSE and MAPE for quantitative evaluation.

[0151] Linear regression analysis was performed on the extracted values ​​and manually measured values ​​of the four phenotypic traits (plant height, stem diameter, stem height, and leaf area index) at the V3 and V6 stages. The comparison results are as follows: Fig.11 The red line in the figure represents the least squares linear regression line. Comparison of the extracted values ​​with the measured values ​​shows that in the V3 period, the R 2 , root mean square error (RMSE) and mean percentage error (MAPE) were 0.97, 0.82 cm and 3.62% respectively; the R 2, RMSE and MAPE were 0.95, 0.06 cm and 2.94 respectively; R 2 , RMSE and MAPE were 0.97, 0.06 cm and 3.63 respectively; R 2 , RMSE and MAPE were 0.72, 1.64 cm and 10.82 respectively. 2 , RMSE and MAPE were 0.93, 1.14 cm and 3.18 respectively; R 2 , RMSE and MAPE were 0.92, 0.08 cm and 3.83 respectively; R 2 , RMSE and MAPE were 0.94, 0.08 cm and 2.82 respectively; R 2 , RMSE and MAPE were 0.79, 1.63cm and 7.72 respectively. The overall accuracy of the measurement results of various morphological parameters of corn was high, and the relative coefficients of plant height, major axis stem diameter and minor axis stem diameter were all above 0.95. Based on the above data, the extraction accuracy of plant height and stem diameter was high, while the extraction result of stem height was relatively poor.

[0152] Tables 5 and 6 present the statistical description results of the phenotypes of corn crop populations in V3 and V6 calculated based on point cloud data. The specific measurement results are as follows: the average plant height of corn in the field in V3 is 18.93 cm, and the relative error relative to the actual field measurement mean is 9.45%; the average long axis stem diameter of corn in the field is 1.53 cm, and the relative error is 1.99%; the average short axis stem diameter of corn in the field is 1.48 cm, and the relative error is 1.22%; the average stalk height of corn in the field is 12.3 cm, and the relative error is 4.91%; the average leaf area index of corn in the field is 0.23, and the relative error is 12.59%. In the V6 period, the average plant height of field corn was 29.89 cm, with a relative error of 1.59%; the average diameter of the major axis stem of field corn was 2.42 cm, with a relative error of 0.62%; the average diameter of the minor axis stem of field corn was 2.38 cm, with a relative error of 1.189%; the average stalk height of field corn was 17.28 cm, with a relative error of 1.61%; the average leaf area index of field corn was 0.50, with a relative error of 11.55%.

[0153] Table 5 Phenotypic statistical description and error of crop population in V3 period

[0154]

[0155] Table 6 Statistical description and error of plant phenotypes of crop populations during V6 period

[0156]

[0157] The present invention achieves high-precision measurement based on point cloud data reconstructed from drone image sequences. Specifically, the relative errors of the major and minor axis stem diameters are small, showing good measurement stability. However, there are errors in plant height measurement, which mainly comes from the non-absolute plane of the soil plane, resulting in errors introduced in post-processing. The stem height error mainly comes from the overlap of some new leaves with the stem, making it difficult to distinguish using the cylindrical fitting method. The partial missing problem of point cloud data leads to large errors in leaf area index measurement.

[0158] In the V3 to V6 growth stages, the average stem diameter and leaf area of ​​the corn population increased significantly, while the stem height increased relatively slowly, which reflects the importance of root development during corn growth and is consistent with the actual growth law in the field. The present invention shows that under specific conditions and farmland areas, the method proposed in the present invention can effectively meet the needs of crop phenotyping and provide a fast and non-destructive measurement method for field crop breeding and growth monitoring.

[0159] In the experiment of leaf area index extraction method, the present invention compares and analyzes the results obtained by leaf projection method and hierarchical calculation method respectively. The experimental results show that the relative error of leaf area index calculated by leaf projection method in V3 period is 18.30%, while the relative error of hierarchical calculation method is 12.59%; the relative error of leaf area index calculated by leaf projection method in V6 period is 20.11%, while the relative error of hierarchical calculation method is 11.55%. The hierarchical calculation method can take into account the information of occluded leaves and effectively reduce the extraction error.

[0160] 3. Results and analysis of indoor single-plant corn phenotype extraction

[0161] This paper uses point cloud data of 12 indoor corn plants at two different growth stages as research samples to quantitatively evaluate the accuracy of the phenotypic parameter extraction method. We compare the plant height, stem diameter, stem height and leaf area values ​​of V3 and V6 periods automatically extracted by the algorithm with the corresponding manual measurement values ​​to evaluate the accuracy of the phenotypic extraction method, thereby providing accuracy guarantee for subsequent large-scale experiments. The comparison results between the automatically extracted values ​​and the manually measured values ​​are shown in Figure 2. Fig.12 After comparing the extracted values ​​with the measured values, the following results were obtained: In the V3 period, the R 2 They are 0.96, 0.98, 0.98, 0.70 and 0.91 respectively, and their corresponding RMSE are 0.63cm, 0.03cm, 0.02cm, 0.87cm and 8.23cm respectively. 2, MAPE were 2.39, 3.21, 3.44, 5.54 and 4.02 respectively. In V6 period, R 2 They are 0.98, 0.98, 0.98, 0.80 and 0.96 respectively, and their corresponding RMSE are 0.80cm, 0.07cm, 0.07cm, 2.28cm and 25.83cm respectively. 2 , MAPE are 1.91, 3.10, 3.19, 9.03 and 4.50 respectively. For the indoor single corn point cloud, the R 2 The above data show that the point cloud data reconstructed by the phenotypic acquisition platform built in the laboratory shows good measurement accuracy, and its performance is sufficient to meet the needs of crop phenotypic measurement, providing a fast and non-destructive measurement method for crop breeding.

Claims

1. A method for extracting corn phenotypic parameters based on point cloud data, characterized in that The method comprises the following steps: Step 1: Experimental dataset construction Step 1.1: Multi-view sequence data collection of corn plants Step 1.1.1 Use a drone equipped with an RGB camera to collect multi-view sequence data of corn at the three-leaf and six-leaf stages in a field environment. Adjust the height and angle of the drone to ensure that a complete image with a high overlap rate is obtained; Step 1.1.2 Build a laboratory acquisition platform to obtain multi-view images of potted corn at the three-leaf and six-leaf stages; Step 1.2: 3D reconstruction and preprocessing of corn plants Step 1.2.1 Use the watershed algorithm to remove the background of the video frame, and reconstruct the indoor single corn 3D point cloud model with color information through the motion recovery structure algorithm. Based on the generated indoor single corn 3D point cloud data, down-sample it through the improved voxel grid method, separate the crop and flower pot point cloud from the original point cloud through the color threshold method, and obtain the corn point cloud with the growth direction in the positive direction of the Z axis through point cloud coordinate scale correction and coordinate transformation. Then use straight-through filtering and color filtering to extract the corn point cloud and flower pot point cloud data, and construct an indoor single corn point cloud segmentation dataset; Step 1.2.2 Reconstruct a 3D point cloud model of corn colonies with color information through the structure-from-motion algorithm. Based on the generated 3D point cloud data of corn colonies, extract the region of interest, filter the point cloud, and perform improved voxel grid downsampling on the original point cloud. Separate the crop and ground point clouds through the color threshold method. Construct a point cloud segmentation dataset of corn colonies in the field through point cloud coordinate scale correction, soil plane fitting, and point cloud rotation. Step 2: Corn point cloud segmentation Step 2.1, segment the field group corn point cloud using the improved Quickshift algorithm; Step 2.2, segment the individual corn points into cloud organs based on cylinder fitting and Euclidean clustering algorithm; Step 3: Extraction of corn phenotypic parameters Step 3.1 Based on the single corn plant point cloud data after organ segmentation, the key parameters related to corn breeding and growth are accurately and losslessly extracted; Step 3.2 For the segmented field group corn point cloud data, the layered calculation method is used to obtain the leaf area index, and the mean values ​​of the plant height and stem diameter parameters of all plants in the group are calculated.

2. The method for extracting corn phenotypic parameters based on point cloud data according to claim 1, characterized in that In step 1, the improved voxel grid downsampling processing steps are as follows: 1) Construct a KD-Tree for the point cloud set P and determine the minimum bounding box of the target point cloud. The length of the bounding box is L x ,L y ,L z , and then calculate the volume V1 of the bounding box; 2) Side length of the voxel grid K is the scale coefficient, α is the scale factor, and the point cloud is divided into n small cubes: n = l × w × h, l = ceil (L x / L),w=ceil(L y / L),h=ceil(L z / L), where ceil() is the upward rounding function; 3) Calculate the center of gravity p0(x0,y0,z0) of each voxel cube and construct a new point cloud set P c ,in N1 is the number of samples; 4) Traverse the point cloud data, use the KD-Tree structure to quickly find the nearest points of each point cloud center of gravity, and construct these neighboring points into a new set P e .

3. The method for extracting corn phenotypic parameters based on point cloud data according to claim 1, characterized in that The specific steps of step 2.1 are as follows: Step 2.1.1 Color space conversion Process the extracted crop point cloud group point cloud dataset and convert the RGB color to the HSI color space; Step 2.1.2 Calculate point density A KD-Tree data structure is established for the point cloud data set, and the density of each point is calculated according to the weighted density formula; Step 2.1.3 calculates the cluster center of the point cloud on the projection surface; Step 2.1.4 Cluster the remaining points in three dimensions After calculating the cluster core, traverse all points. If the point is not in the cluster core, perform three-dimensional QuickShift clustering on the point and move each sample point to its nearest neighbor with higher density until it reaches a cluster core.

4. The method for extracting corn phenotypic parameters based on point cloud data according to claim 1, characterized in that The specific steps of step 2.2 are as follows: Step 2.2.1: For the point cloud data of a single corn plant, a random sampling consensus algorithm is used to perform cylindrical fitting of the stalk, thereby effectively separating the corn leaves and stalks; Step 2.2.2: For the point cloud data with the stems removed, use the Euclidean clustering segmentation algorithm to accurately segment individual leaves and complete the fine segmentation of corn organs.

5. The method for extracting corn phenotypic parameters based on point cloud data according to claim 1, characterized in that The specific steps of step 3.1 are as follows: The plant height information was obtained by calculating the vertical distance from the top of the corn plant to the soil plane; the stem height was extracted using a cylinder fitting algorithm; the major and minor axis diameters of the stem were measured using an ellipse fitting algorithm; and the leaf area was extracted using a greedy projection triangulation algorithm.

Images