Plant phenotype parameter extraction method, device and equipment and storage medium

Abstract

Embodiments of the present disclosure provide a plant phenotype parameter extraction method, device and equipment and a storage medium, relating to the technical field of data processing. The method obtains plant preliminary voxel data from plant point cloud data, and performs voxel segmentation on the plant preliminary voxel data to obtain leaf point cloud data of the plant, thereby performing parameter extraction on different plant parts, improving data processing efficiency and result accuracy. Next, principal component analysis is performed on the plant point cloud data to obtain plant height and plant width, and similarity clustering is performed on the leaf point cloud data to obtain the number of leaves of the plant. Then, principal component analysis is performed on the leaf point cloud data to obtain leaf length and leaf width, and surface reconstruction is performed on the leaf point cloud data to obtain leaf area. Finally, rich phenotype parameters are obtained from the point cloud data, including plant height, plant width, number of leaves, leaf length, leaf width and leaf area.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a plant phenotype parameter extraction method, device, equipment and storage medium. BACKGROUND

[0002] In the field of modern agriculture, the importance of plant phenotype collection technology is increasingly prominent. Through this technology, we can accurately and quickly obtain various parameters related to plant traits, thereby realizing the monitoring of plant growth, the selection of high-quality varieties, and the in-depth study of the molecular mechanisms of trait regulation, etc. In the growth and development process of crops, the whole plant and local organs will continuously change in geometry and properties, so it is particularly important to continuously monitor crops. Improving the efficiency of plant phenotype parameter collection and realizing the dynamic quantification of key phenotype parameters at various growth stages and growth monitoring are of great significance for crop breeding, cultivation and yield improvement.

[0003] In related technologies, sensors are used to collect different types of raw data of plants, including RGB images, spectral images, thermal infrared images, etc., and then the raw data is analyzed to obtain phenotype parameters. Part of them is to analyze two-dimensional images, but due to the existence of occlusion between plants or the lack of depth information, the accuracy is not high, and it is not suitable for measuring three-dimensional morphological parameters such as leaf length. Another part is to extract parameters based on three-dimensional data of spectral information. If accurate spectral information is to be obtained, a hyperspectral image sensor needs to be configured, which is costly. Therefore, it is necessary to propose a low-cost plant phenotype parameter extraction method with high parameter collection accuracy. SUMMARY

[0004] The main purpose of the embodiments of the present disclosure is to propose a plant phenotype parameter extraction method, device, equipment and storage medium, which can reduce the cost of parameter extraction and improve the accuracy of parameter collection.

[0005] To achieve the above purpose, a first aspect of an embodiment of the present disclosure proposes a plant phenotype parameter extraction method, comprising:

[0006] Obtaining plant point cloud data of a plant;

[0007] Obtaining plant preliminary voxel data from the plant point cloud data, and performing voxel segmentation on the plant preliminary voxel data to obtain a stem-leaf segmentation result of the plant, wherein the stem-leaf segmentation result comprises leaf point cloud data;

[0008] Performing principal component analysis on the plant point cloud data to obtain plant height and plant width of the plant, and performing similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant;

[0009] perform principal component analysis on the leaf point cloud data to obtain a leaf length and a leaf width of the plant, and perform surface reconstruction on the leaf point cloud data to obtain a leaf area of the plant;

[0010] generate a phenotype parameter of the plant based on the plant height, the plant width, the leaf number, the leaf length, the leaf width, and the leaf area.

[0011] In some embodiments, the obtaining of the plant preliminary voxel data from the plant point cloud data comprises:

[0012] voxelize the plant point cloud data based on an octree to obtain a plurality of voxel units;

[0013] calculate a similarity value between each adjacent voxel unit, merge the voxel units with high similarity values to obtain new voxel units, and update a center voxel, and repeat the step until the position of the center voxel is unchanged, to obtain the plant preliminary voxel data.

[0014] In some embodiments, the voxel segmentation of the plant preliminary voxel data to obtain a stem-leaf segmentation result of the plant comprises:

[0015] obtain a centroid and a corresponding normal vector of two adjacent voxel units, and generate a voxel vector based on the positions of the two centroids;

[0016] generate a first convexity of the two voxel units based on a first included angle between the voxel vector and the two normal vectors;

[0017] calculate a cross product between the two normal vectors and a second included angle between the voxel vector and the cross product, generate a second convexity of the two voxel units based on a relationship between the second included angle and a preset threshold, and obtain a convexity of the two voxel units based on the first convexity and the second convexity, the convexity comprising convexity and concavity;

[0018] select any voxel unit as a seed point, label a region based on the convexity of the voxel unit, if the voxel unit is convex, label it as the region, until the seed point and the adjacent voxel units are all concave, select other voxel units to update the seed point, repeat the step until each voxel unit is labeled;

[0019] obtain each region according to the labeling result, and obtain the stem-leaf segmentation result according to the regions.

[0020] In some embodiments, the stem-leaf segmentation result further comprises stem point cloud data; and the obtaining of each region according to the labeling result and the obtaining of the stem-leaf segmentation result according to the regions comprise:

[0021] randomly select a point cloud of a region as a sample subset;

[0022] obtain a cylindrical model based on minimum variance estimation, calculate sample deviations of sample points in the sample subset from the cylindrical model, and take sample points with sample deviations less than a preset threshold as inliers of the cylindrical model, and count the number of inliers;

[0023] repeat the iteration process, and take the cylindrical model with the largest number of inliers as a target model;

[0024] take the inliers of the target model as stem points, take the plant point cloud data corresponding to the region based on the stem points as stem part point cloud data, and take others as leaf part point cloud data.

[0025] In some embodiments, the principal component analysis on the plant point cloud data obtains the plant height and plant width, comprising:

[0026] obtain the centroid of the plant point cloud data, establish a first covariance matrix based on the centroid, solve the first covariance matrix to obtain three first eigenvalues and first eigenvectors of the first eigenvalues, and sort the first eigenvalues in descending order, and obtain a first plant direction, a second plant direction and a third plant direction according to the first eigenvectors;

[0027] establish a rotation and translation matrix based on the first plant direction and the centroid, and perform coordinate correction on the plant point cloud data using the rotation and translation matrix to obtain corrected point cloud data;

[0028] generate a rectangular enclosing region according to the corrected point cloud data and a reference coordinate system, and calculate the side length of the rectangular enclosing region;

[0029] take the side length in the first plant direction as the plant height;

[0030] select the larger value between the side length in the second plant direction and the side length in the third plant direction as the plant width.

[0031] In some embodiments, the similarity clustering on the leaf part point cloud data obtains the number of leaves of the plant, comprising:

[0032] obtain a set of neighboring points of each point in the leaf part point cloud data;

[0033] obtain a corresponding fitting plane according to the set of neighboring points, calculate the normal and curvature of the fitting plane, and select the point corresponding to the fitting plane with the smallest curvature as an initial seed point;

[0034] merge similar points in a neighborhood of the initial seed point to a growing region where the initial seed point is located based on a normal angle threshold and a curvature threshold;

[0035] select a new seed point, merge similar points in a neighborhood of the new seed point to the growing region, and repeat the step until the growing region iteration is completed to obtain a leaf region;

[0036] obtain a number of points in the leaf region, take the leaf region with the number greater than a preset number threshold as a target leaf region, and obtain the number of leaves according to the number of the target leaf region.

[0037] In some embodiments, the principal component analysis on the leaf point cloud data to obtain a leaf length and a leaf width of the plant comprises:

[0038] obtain a centroid of the leaf point cloud data, establish a second covariance matrix based on the centroid, solve the second covariance matrix to obtain three second eigenvalues and a second eigenvector of the second eigenvalues, and sort the second eigenvalues in descending order, and obtain a first leaf direction and a second leaf direction according to the second eigenvector;

[0039] perform a curve mapping step, the curve mapping step comprising:

[0040] select a maximum point and a minimum point of the target leaf direction in a reference coordinate system, take the minimum point as an initial candidate point, and obtain neighboring points of the initial candidate point;

[0041] calculate a first distance of each neighboring point from the initial candidate point and a second distance of each neighboring point from the maximum point, and calculate a distance sum of the first distance and the second distance, respectively;

[0042] take the neighboring point with the smallest distance sum as a new candidate point, if a distance between the new candidate point and the initial candidate point meets a preset distance threshold, add the new candidate point to a target point set of a target curve, otherwise, reselect the new candidate point;

[0043] take the new candidate point as the initial candidate point, continue iteration until no new point is added to the target point set, obtain the target curve according to a projection of the target point set, and calculate a target length according to the target curve;

[0044] When the target leaf direction is the first leaf direction, the target curve is a midrib curve, the target point set is a midrib point set, and the target length is a leaf length; when the target leaf direction is the second leaf direction, the target curve is a leaf length curve, the target point set is a leaf length point set, and the target length is a leaf width.

[0045] In some embodiments, the surface reconstruction on the leaf point cloud data obtains a leaf area of the plant, and the surface reconstruction on the leaf point cloud data comprises:

[0046] projecting the leaf point cloud data based on a normal to obtain a two-dimensional projection surface;

[0047] selecting an initial triangular surface from the two-dimensional projection surface as a starting surface, selecting adjacent points of the starting surface for boundary expansion to obtain a surface boundary, and connecting adjacent points of the surface boundary to form a triangular surface; repeating the step until a leaf grid surface composed of the triangular surfaces is obtained;

[0048] calculating the leaf area based on the leaf grid surface.

[0049] In some embodiments, initial plant point cloud data of a plant is obtained;

[0050] performing multiple diameter fittings on a cultivation container of the plant to obtain an average diameter, and obtaining a point cloud recovery ratio based on an actual diameter of the cultivation container and the average diameter;

[0051] recovering the scale of the initial plant point cloud data based on the point cloud recovery ratio to obtain plant point cloud size data;

[0052] obtaining a histogram of the plant point cloud size data, removing a background based on the histogram to obtain plant point cloud foreground data;

[0053] selecting a corresponding filter for noise removal based on a type of noise in the plant point cloud foreground data to obtain the plant point cloud data.

[0054] To achieve the above object, a second aspect of the embodiments of the present disclosure provides a plant phenotype parameter extraction device, comprising:

[0055] a point cloud data acquisition module configured to obtain plant point cloud data of a plant;

[0056] a stem-leaf segmentation module configured to obtain plant preliminary voxel data from the plant point cloud data, and perform voxel segmentation on the plant preliminary voxel data to obtain a stem-leaf segmentation result of the plant, the stem-leaf segmentation result comprising stem point cloud data and leaf point cloud data;

[0057] The first parameter acquisition module is configured to perform principal component analysis on the plant point cloud data to obtain the plant height and plant width of the plant, and perform similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant.

[0058] The second parameter acquisition module is configured to perform principal component analysis on the leaf point cloud data to obtain the leaf length and leaf width of the plant, and perform surface reconstruction on the leaf point cloud data to obtain the leaf area of the plant.

[0059] The phenotype parameter acquisition module is configured to generate the phenotype parameters of the plant based on the plant height, the plant width, the number of leaves, the leaf length, the leaf width and the leaf area.

[0060] To achieve the above object, a third aspect of the embodiment of the present disclosure provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0061] To achieve the above object, a fourth aspect of the embodiment of the present disclosure provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0062] The plant phenotype parameter extraction method, device, equipment and storage medium provided by the embodiment of the present disclosure obtain the plant preliminary voxel data through the plant point cloud data, perform voxel segmentation on the plant preliminary voxel data to obtain the leaf point cloud data of the plant, then perform principal component analysis on the plant point cloud data to obtain the plant height and plant width of the plant, and perform similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant; then perform principal component analysis on the leaf point cloud data to obtain the leaf length and leaf width of the plant, and perform surface reconstruction on the leaf point cloud data to obtain the leaf area of the plant, and finally obtain the phenotype parameters including the plant height, the plant width, the number of leaves, the leaf length, the leaf width and the leaf area. The embodiment of the present application first performs segmentation on the point cloud data, thereby extracting parameters for different plant parts, improving the data processing efficiency and result accuracy, then uses the principal component analysis method to obtain the plant height, the plant width, the leaf length and the leaf width, uses the similarity clustering to obtain the number of leaves, and then performs surface reconstruction to obtain the leaf area, which can obtain rich phenotype parameters from the point cloud data. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a flowchart of the plant phenotype parameter extraction method provided by the embodiment of the present disclosure.

[0064] Figure 2 is a schematic diagram of an image acquisition system of the plant phenotype parameter extraction method provided by the embodiment of the present disclosure.

[0065] Figure 3 FIG. 1 is a schematic diagram of a video frame sequence of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0066] Figure 4 FIG. 2 is a schematic diagram of a video frame image denoising effect of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0067] Figure 5 FIG. 3 is a schematic diagram of plant initial point cloud data of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0068] Figure 6 FIG. 4 is a schematic diagram of a maximum diameter fitting process of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0069] Figure 7 FIG. 5 is a schematic diagram of a histogram of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0070] Figure 8 FIG. 6 is a schematic diagram of plant point cloud foreground data of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0071] Figure 9 FIG. 7 is a schematic diagram of plant point cloud data of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0072] Figure 10 FIG. 8 is a schematic diagram of plant point cloud data after simplification of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0073] Figure 11 FIG. 9 is a schematic diagram of stem and leaf segmentation results of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0074] Figure 12 FIG. 10 is a schematic diagram of stem point cloud data of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0075] Figure 13 FIG. 11 is a schematic diagram of leaf point cloud data of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0076] Figure 14 FIG. 12 is a schematic diagram of plant height and plant width calculation of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0077] Figure 15 FIG. 13 is a schematic diagram of multiple leaf regions of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0078] Figure 16 FIG. 14 is a schematic diagram of leaf vein point set of a plant phenotype parameter extraction method according to an embodiment of the present disclosure.

[0079] Figure 17 FIG. 1 is a leaf vein curve schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0080] Figure 18 FIG. 2 is a leaf width point set schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0081] Figure 19 FIG. 3 is a leaf width curve schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0082] Figure 20 FIG. 4 is a leaf grid surface schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0083] Figure 21 FIG. 5 is a color grade schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0084] Figure 22 FIG. 6 is a yellow-green color proportion in three color grades under two plant planting conditions schematic diagram of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0085] Figures 23a to 23f FIG. 7 is a waterfall diagram of the average value of each key quantitative trait of non-heading Chinese cabbage at four growth stages of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0086] Figure 24 FIG. 8 is a flowchart of a plant phenotype parameter extraction method provided by an embodiment of the present disclosure.

[0087] Figure 25 FIG. 9 is a structure block diagram of a plant phenotype parameter extraction device provided by another embodiment of the present disclosure.

[0088] Figure 26 FIG. 10 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0089] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, further detailed descriptions will be given to the present disclosure in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0090] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0091] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in the description herein is for describing the embodiments of the disclosure only and is not intended to be limiting of the disclosure.

[0092] First, several terms involved in the present disclosure are analyzed:

[0093] In the field of modern agriculture, the importance of plant phenotype acquisition technology is increasingly prominent. Through this technology, we can accurately and quickly obtain various parameters related to plant traits, thereby achieving monitoring of plant growth, selection of high-quality varieties, and in-depth study of molecular mechanisms of trait regulation, etc. During the growth and development of crops, the whole plant and local organs will continue to change in geometry and properties, so it is particularly important to continuously monitor crops. Improving the efficiency of plant phenotype parameter acquisition and achieving dynamic quantification of key phenotype parameters at various growth stages and growth monitoring are of great significance for crop breeding, cultivation and yield improvement.

[0094] The initial plant phenotype technology mainly measures plant phenotypes by naked eye and traditional tools to obtain plant phenotype parameters such as plant height, stem diameter and canopy width. Manual collection of plant phenotype parameters is inaccurate, time-consuming and labor-intensive, and cannot meet the needs of modern production experiments. With the development of digital phenotype technology, related technologies use sensors to collect plant point cloud data, including RGB images, spectral images, thermal infrared images and other types of raw data, and then analyze the raw data to obtain phenotype parameters. Part of them is to analyze two-dimensional images, but due to the existence of occlusion between plants or the lack of depth information, the accuracy is not high, and it is not suitable for measuring three-dimensional morphological parameters such as leaf length. Another part is to extract parameters based on three-dimensional data from spectral information. If accurate spectral information is to be obtained, a hyperspectral image sensor needs to be configured, which is costly. In addition, the phenotype measurement method in related technologies mainly analyzes the growth morphology of corn, cotton and sorghum, etc., and has problems such as high algorithm complexity and low automation of plant phenotype acquisition and processing. In addition, if the plant canopy sprouts new leaves close to each other, it is easy to cause skeleton extraction or organ segmentation errors. Therefore, it is necessary to propose a plant phenotype parameter extraction method with low cost, low complexity and high parameter acquisition accuracy.

[0095] Based on this, the embodiments of the present disclosure provide a plant phenotype parameter extraction method, device, equipment and storage medium. First, the point cloud data is segmented, so as to extract parameters for different plant parts, improve data processing efficiency and result accuracy, then use principal component analysis to obtain plant height, plant width, leaf length and leaf width, use similarity clustering to obtain leaf number, and then perform surface reconstruction to obtain leaf area, which can obtain rich phenotype parameters by using point cloud data.

[0096] The embodiment of the present disclosure provides a plant phenotype parameter extraction method, device and equipment and a storage medium. The embodiment is described as follows. First, the plant phenotype parameter extraction method in the embodiment of the present disclosure is described.

[0097] The plant phenotype parameter extraction method in the embodiment of the present disclosure is described as follows.

[0098] Figure 1 is an optional flowchart of the plant phenotype parameter extraction method provided by the embodiment of the present disclosure, Figure 1 The method in can include but is not limited to steps S110 to S150. It can be understood that the embodiment does not limit the order of steps S110 to S150 in Figure 1 The order of steps S110 to S150 can be adjusted or some steps can be reduced or added according to actual needs.

[0099] Step S110: obtaining plant point cloud data of a plant.

[0100] In an embodiment, the plant is analyzed by using pakchoi. The pakchoi is different from the general cabbage in that it does not form a tight spherical head, and the leaf of the pakchoi is relatively loose and the growth cycle is relatively short. It can be understood that the plant is only exemplified herein and the type of the plant is not limited.

[0101] In an embodiment, the process of obtaining the plant point cloud data of the plant can be as follows.

[0102] First, the plant initial point cloud data of the plant is obtained.

[0103] In this embodiment, the plant is planted in a cultivation container, for example, a flowerpot, and then the plant initial point cloud data of the plant is collected by using an image acquisition system as shown in Figure 2 .

[0104] Figure 2 The image acquisition system in includes a photographic studio 1, a power line 2, a motorized turntable 3, a telescopic camera support 4, an RGB camera 5 and an infrared remote controller 6. The telescopic camera support 4 can adjust the camera height and the shooting angle of the RGB camera 5.

[0105] When collecting, the flowerpot is placed on the motorized turntable 3, and then the infrared remote controller 6 is used to turn on the switch. When the motorized turntable 3 rotates, the 360° rotating video of the plant under two visual angles of the RGB camera 5 looking straight and 45° inclined angle downward looking is obtained. Then the rotating video is processed, and the video frame sequence in which the adjacent image overlap area meets the overlap threshold is extracted. Assuming that the frame number is 60, the extraction is performed every 10 frames. In this embodiment, the overlap threshold is 70% to 80%. Referring to Figure 3 ,Figure 3 A schematic diagram of the video frame sequence collected for the present embodiment.

[0106] Next, the video frame sequence is preliminarily denoised using a watershed algorithm to improve the point cloud reconstruction efficiency and data purity. The specific watershed algorithm denoising process includes: regarding the brightness of the grayscale image of the video frame as the surface of a mountain, and finding the "low land" and "high land" from it according to the brightness value, wherein the image background of the video frame is regarded as the "low land" and the plant part is regarded as the "high land", then taking the pixel point at the top left corner of the image of the video frame as the seed point to fill the entire image, and replacing all pixel points in the "low land" with white pixel points, thereby completing the background removal. The image of each video frame is removed from the background one by one. For reference Figure 4 , a schematic diagram of the video frame image denoising effect is shown in FIG. 6. Figure 3

[0107] Then, based on the video frame image after background removal, the three-dimensional reconstruction of the point cloud is performed. In an embodiment, the RealityCapture software is used to import the video frame image after background removal and perform sparse reconstruction and dense reconstruction, and then the model is texture mapped to obtain a color dense point cloud model of the plant as the initial point cloud data of the plant. For reference Figure 5 , a schematic diagram of the initial point cloud data of the plant is shown in FIG. 7, which can be understood as a black and white image.

[0108] Since there is a difference between the generated initial point cloud data of the plant and the actual size of the plant, the present embodiment also needs to restore the size of the plant.

[0109] In an embodiment, the cultivation container of the plant is fitted with multiple diameters to obtain an average diameter, and a point cloud recovery ratio is obtained according to the real diameter and the average diameter of the cultivation container. The diameter refers to the maximum cross-sectional length of the cultivation container. When the cultivation container is a flowerpot, the diameter is the diameter of the surface circle of the flowerpot. The real diameter d true can be obtained by prior knowledge or by image target recognition. The average diameter is obtained by fitting the maximum diameter of the cultivation container in the plant growth background of the video frame image using the random sample consensus algorithm, and the average value of the maximum diameters obtained by multiple fittings is the average diameter d MVS . For reference Figure 6 , a schematic diagram of the maximum diameter fitting process is shown in FIG. 8.

[0110] Next, the point cloud recovery ratio is obtained according to the real diameter and the average diameter of the cultivation container. The point cloud recovery ratio is represented as:

[0111]

[0112] ​Then, the coordinate of each point cloud in the plant initial point cloud data is multiplied by the point cloud recovery scale to recover the scale of the plant initial point cloud data, and plant point cloud size data is obtained.

[0113] The plant point cloud size data obtained through the above size recovery has at least centimeter-level precision. Then, a histogram of the plant point cloud size data is obtained by using an HSV model, and the plant point cloud size data is converted from RCG space to HSV space, and the histogram established is as shown in FIG. 4. Figure 7 The histogram includes H component and kernel density information, and the dotted line in the figure represents a critical value for separating the plant from the flowerpot and the soil, and the H value of the plant is constrained in the range of H∈(60, 120). Then, according to the H value constraint range of the plant in the histogram shown in FIG. 4, the video frame image of the plant point cloud size data is extracted, and irrelevant backgrounds such as red flowerpots and brown soil are removed, and a plant point cloud foreground data diagram as shown in FIG. 5 is obtained. Figure 7 Figure 8

[0114] Then, based on the type of noise points in the plant point cloud foreground data, a corresponding filter is selected for noise removal, and plant point cloud data is obtained. The type of noise points includes outliers and isolated points or invalid points.

[0115] For outliers, a statistical filter is used for noise removal. Specifically, the neighborhood of each point in the plant point cloud foreground data is statistically analyzed, and based on the distance distribution characteristics of the point to all neighboring points, outliers that are sparsely distributed in space are removed.

[0116] First, the plant point cloud foreground data is traversed, and the average distance d i between each point and multiple neighboring points is calculated. Second, the mean value μ of all average distances d i is calculated, and the global standard deviation σ is calculated, which is expressed as:

[0117] Finally, the point cloud is traversed again, and the points whose average distance d i do not satisfy d i ≤ μ + α * σ are marked as outliers and removed from the plant point cloud foreground data, where α represents a standard deviation coefficient, which can be valued according to an empirical value.

[0118] For the suspended isolated points or invalid points, the information amount contained therein is small and can be ignored, and therefore a radius filter is further used for removal. The specific process is: the number of neighboring points within a given radius neighborhood of each point is used as a judgment basis, and when the number is greater than a given value, the point is retained, otherwise the point is removed from the plant point cloud foreground data.

[0119] The plant point cloud data is obtained after noise removal. In an embodiment, the radius neighborhood is 0.8, and the fixed value is 100, and the generated plant point cloud data is as shown in FIG. 6. Figure 9 ​​The plant point cloud data shown.

[0120] Considering that the plant point cloud data is relatively dense, affecting the subsequent data processing speed, the embodiment of the present application performs a simplification processing with a simplification rate of 30% on the plant point cloud data under the condition of retaining rich detail features. Specifically, a uniform downsampling simplification method based on voxels is adopted, and the plant point cloud data is simplified by creating a three-dimensional voxel grid for the plant point cloud data and approximating the center of gravity of each voxel to represent all points in the voxel. Figure 10 The plant point cloud data after simplification is shown.

[0121] After the above steps, the real scale of the plant point cloud data is restored, the pure plant point cloud data is established, and the model data amount is simplified, thereby improving the plant phenotype parameter extraction accuracy and the subsequent processing speed of the plant point cloud data.

[0122] Next, the extraction process of the plant phenotype parameters is described.

[0123] Step S120: obtaining plant preliminary voxel data from the plant point cloud data, and performing voxel segmentation on the plant preliminary voxel data to obtain a stem-leaf segmentation result of the plant, the stem-leaf segmentation result including leaf point cloud data.

[0124] In an embodiment, the stem-leaf segmentation result includes stem point cloud data and leaf point cloud data. The process of obtaining plant preliminary voxel data from the plant point cloud data includes:

[0125] First, the plant point cloud data is voxelized based on an octree to obtain a plurality of voxel units. The octree is a data structure based on the extension of the binary tree, which is used to represent and organize objects in three-dimensional space. Each node of the octree represents a cubic region, and the root node represents the entire space. Then, the space is divided into smaller sub-regions by recursively dividing the cubic region until a certain termination condition is met. The characteristic of the octree is that each node has at most eight child nodes, corresponding to the cubic region being uniformly divided into eight sub-cubes. Each child node can represent whether there is an object in the sub-region, or further subdivide the sub-region. Voxelization is the process of converting continuous three-dimensional space into a discrete voxel grid. The embodiment of the present application uniformly divides the plant point cloud data using the octree method to obtain a plurality of cubic regions, each of which is defined as a voxel unit.

[0126] Then, the similarity value D between each adjacent voxel unit is calculated, and the similarity value D is represented as:

[0127]

[0128] wherein R secd represents the resolution of the voxel unit, and D c, D s , D n represents the color difference value, the spatial distance difference value and the normal vector difference value of two voxel units, W c , W s , W n represents the corresponding weight.

[0129] Then, the voxel units with high similarity values are merged to obtain new voxel units, the new voxel units are taken as super voxels, and the center voxels of the super voxels are obtained. The step is repeatedly executed until the positions of the obtained center voxels are unchanged, and a super voxel over-segmentation result of the plant point cloud is obtained, and the super voxel over-segmentation result is taken as plant preliminary voxel data.

[0130] Next, it is necessary to fuse the plant preliminary voxel data into a point cloud block with significant semantic information. The specific process includes:

[0131] First, the centroids and corresponding normal vectors of two adjacent voxel units p1 and p2 are obtained, and a voxel vector is generated according to the positions of the two centroids. The normal vector of a voxel unit can be obtained by the least square method, and the normal vector is represented as and

[0132] Next, the first concave-convexity of the two voxel units is generated according to the first included angle between the voxel vector and the two normal vectors. The voxel vector and the normal vector have an included angle of a1, and the voxel vector and the normal vector have an included angle of a2. When a1 Thresh is less than a2, the voxel units p1 and p2 will affect each other, and a threshold value b is introduced to represent the first concave-convexity CC, and the first concave-convexity is represented as:

[0133]

[0134] Where b represents the absolute value of the difference between a1 and a2, convex represents convexity, and concave represents concavity.

[0135] Since the first concave-convexity cannot judge the discontinuous structure of the surface, the second concave-convexity is introduced in this embodiment to identify invalid connections.

[0136] The cross product between the two normal vectors is calculated, and the voxel vector is obtained. The cross product The second included angle θ is used to determine whether two adjacent voxel units are truly connected and whether a separate surface exists. When the second included angle θ is close to 90°, the surfaces of the two voxel units are continuous; when the second included angle θ is close to 0°, the surfaces of the two voxel units are discontinuous. Obviously, the probability of discontinuity is higher when the value of the second included angle θ is small, therefore a preset threshold θ is used. Thresh , θ < θ Thresh Connections with similar normal vectors are considered invalid. For voxel units with similar normal vectors, the threshold condition is appropriately relaxed to compensate for potential errors in the algorithm. The preset threshold θ... Thresh Represented as:

[0137]

[0138] Where λ represents the normal vector and The angle between them The values ​​of a and b are 60°, 0.25°, and 25°, respectively.

[0139] Therefore, based on the second included angle θ and the preset threshold θ Thresh The relationship between the two voxel units generates the second concavity / convexity SC, which is represented as:

[0140]

[0141] Where θ = θ(p1, p2).

[0142] Next, based on the first and second concavity / convexity properties, the concavity / convexity conv of the two voxel units is obtained, and the concavity / convexity conv is represented as:

[0143] conv(p1,p2)=CC(p1,p2)∩SC(p1,p2)

[0144] In this embodiment, the unevenness includes both convexity and concavity;

[0145] After obtaining the concavity / convexity of each voxel and its adjacent voxels, any voxel is selected as a seed point. Regions are marked based on the concavity / convexity of the voxel, allowing only regions to grow across convex edges. Growth stops when both the seed point and its adjacent supervoxels are concave. Therefore, if a voxel is convex, it is marked as a region until both the seed point and its adjacent voxels are concave, at which point production stops. Other voxels are selected as new seed points, and this step is repeated until each voxel is marked. (Refer to...) Figure 11 The diagram shows the results of stem and leaf segmentation. It can be seen that due to the complexity of leaf features, there is still over-segmentation. However, the point cloud parts belonging to the stem and leaves are segmented correctly and have not adhered.

[0146] Then, each region is obtained according to the labeling result, and the stem-leaf segmentation result is obtained according to the region. The specific process includes:

[0147] First, a point cloud of a region is randomly selected as a sample subset. Since the stem region exhibits a cylindrical feature at different heights, a cylindrical model is obtained based on a least variance estimation algorithm. Then, the sample deviation of all sample points in the sample subset from the cylindrical model is calculated, and the sample points with a sample deviation less than a preset threshold DistanceThreshold (for example, 0.03) are regarded as inliers of the cylindrical model, and the number of inliers is counted. The region is randomly selected as a sample subset again, and the iteration process is repeated. In each iteration, the cylindrical model with the largest number of inliers is taken as the target model. In the iteration process, whether to end the iteration process is determined according to the expected error rate, the total number of samples, and the current iteration number.

[0148] After the iteration is completed, the inliers of the target model are taken as stem points. Then, the plant point cloud data corresponding to the region containing the stem points is selected as stem point cloud data based on the stem points, and the other plant point cloud data is taken as leaf point cloud data. Referring to Figure 12 , it is a schematic diagram of the stem point cloud data. Referring to Figure 13 , it is a schematic diagram of the leaf point cloud data. It can be seen that the segments belonging to the stem and the leaf in the segmentation result are correctly segmented and do not stick together.

[0149] The voxel segmentation process of the embodiment of the application does not need to set prior conditions or manual intervention, the degree of automation of the voxel segmentation process of the plant is high, and the segments belonging to the stem and the leaf in the segmentation result are correctly segmented and do not stick together.

[0150] Step S130: performing principal component analysis on the plant point cloud data to obtain the plant height and plant width, and performing similarity clustering on the leaf point cloud data to obtain the number of leaf blades of the plant.

[0151] In an embodiment, first, the centroid of the plant point cloud data is obtained. The centroid represents the geometric center position of the plant point cloud data. Each point in the plant point cloud data is traversed, and the coordinates of the point are added to the total sum of the corresponding axis. Then, the coordinate sum of each axis is divided by the total number of points in the plant point cloud data, and the coordinates of the centroid are obtained.

[0152] Then a first covariance matrix is established based on the centroid. The first covariance matrix represents the dispersion and correlation of the plant point cloud data in each direction. Since the plant point cloud data points are three-dimensional models, the covariance matrix is a 3x3 symmetric matrix. First, a 3x3 zero matrix is initialized as the covariance matrix, then for each point in the plant point cloud data, the difference vector between its coordinates and the centroid coordinates is denoted as v, and then the vector v is multiplied by its transpose to obtain a 3x3 matrix, which is added to the covariance matrix. After traversing all the points, divide the covariance matrix by the total number of points in the plant point cloud data. The centroid and covariance matrix of the plant point cloud data are obtained.

[0153] Solving the first covariance matrix obtained by the above process, three first eigenvalues and the first eigenvector of each first eigenvalue are obtained. Then the first eigenvalues are sorted in descending order. The first eigenvalue of the first eigenvalue is the first plant direction, the second eigenvalue of the first eigenvalue is the second plant direction, and the third eigenvalue of the first eigenvalue is the third plant direction.

[0154] The first eigenvector corresponding to the largest first eigenvalue is selected as the main direction of the plant point cloud data, so the rotation and translation matrix is established based on the first plant direction and the centroid.

[0155] The main direction represents the main direction of the data in the feature space, so the first eigenvector representing the main direction is taken as the column vector of the rotation matrix. For example, a feature vector is represented as a unit vector v = [v1, v2, v3], a 3x3 rotation matrix R can be constructed, where the first column is v, and the second and third columns can be obtained by cross-multiplying v. Then calculate the translation vector according to the centroid coordinates, assuming the centroid coordinates are [xm, ym, zm], then the translation vector t = [xm, ym, zm]. Finally, the rotation matrix R and the translation vector t are combined to form the rotation and translation matrix T. For example, the rotation and translation matrix T is a 4x4 matrix, where the first 3x3 part is the rotation matrix R, the last column is the translation vector t, and the last element is 1.

[0156] Next, the plant point cloud data is corrected using the rotation and translation matrix to obtain the corrected point cloud data. In this embodiment, the plant point cloud data is applied to the rotation and translation matrix to correct the coordinates to align with the reference coordinate system to obtain the corrected point cloud data. The reference coordinate system can be the geodetic coordinate system.

[0157] Then a rectangular bounding region is generated according to the corrected point cloud data and the reference coordinate system, similar to a rectangular box in the reference coordinate system, the corrected point cloud data is located in the rectangular bounding region, and the rectangular bounding region is the smallest bounding region. Then three edge lengths of the rectangular bounding region are calculated, the edge length in the first plant direction is taken as the plant height, and the larger value of the edge length in the second plant direction and the edge length in the third plant direction is taken as the plant width. Referring to Figure 14 A schematic diagram for calculating the plant height and the plant width is shown, in which the x-axis direction is the first plant direction, the y-axis is the second plant direction, and the z-axis is the third plant direction.

[0158] In an embodiment, the process of similarity clustering of the leaf point cloud data to obtain the number of leaves of the plant includes:

[0159] First, a set of neighboring points of each point in the leaf point cloud data is obtained, then a corresponding fitting plane is obtained by plane fitting according to the set of neighboring points, and the normal and curvature of the fitting plane are calculated.

[0160] Then, the point corresponding to the fitting plane with the smallest curvature is selected as the initial seed point, and the normal angle and the curvature between the initial seed point and each point in the neighborhood are taken as the constraint conditions based on the normal angle threshold and the curvature threshold. If the normal angle is within the normal angle threshold and the curvature is within the curvature threshold, the point is taken as a similar point in the neighborhood of the initial seed point, and is merged into the growth region where the initial seed point is located. By gradually merging similar points, a leaf region starts to grow and expand.

[0161] Since the growth of a leaf region is not limited to the vicinity of the initially selected seed point, it will continue to grow along the similar neighborhood points. If there is a point more suitable for being a seed (for example, smaller curvature) in the neighborhood of the current growth region, the point can be selected as a new seed point, and growth is continued with the new seed point as the center. Therefore, in the embodiment of the present application, the new seed point is reselected as the initial seed point by the constraint conditions of the normal angle threshold and the curvature threshold, and the similar points in the neighborhood of the new seed point are merged into the growth region. By repeating the step, adjacent leaf regions can be gradually connected until there is no point that meets the constraint conditions, the growth region iteration is completed, and a leaf region is obtained. In this way, regions with similar curvature characteristics can be segmented and connected into complete leaf regions.

[0162] The above steps are repeated until the clustering process of all leaves is completed, and a plurality of leaf regions are obtained. Referring to Figure 15 A schematic diagram of the plurality of leaf regions is shown.

[0163] Since there are noise points, when the number of points in the leaf region obtained by clustering is less than a certain amount, it is regarded as a leaf noise point for removal, so the embodiment obtains the number of points in the leaf region, and regards the leaf region with a number greater than a preset number threshold as a target leaf region, and then obtains the number of leaves according to the number of target leaf regions.

[0164] After obtaining the overall parameters such as plant height, plant width and leaf number, the detailed parameters such as leaf length, leaf width and leaf area are calculated.

[0165] Step S140: performing principal component analysis on the leaf point cloud data to obtain the leaf length and leaf width of the plant, and performing surface reconstruction on the leaf point cloud data to obtain the leaf area of the plant.

[0166] In an embodiment, the process of performing principal component analysis on the leaf point cloud data to obtain the leaf length and leaf width of the plant includes:

[0167] First, the centroid of the leaf point cloud data is obtained, a second covariance matrix is established based on the centroid, the second covariance matrix is solved to obtain three second eigenvalues and a second eigenvector of the second eigenvalues, and the second eigenvalues are sorted in descending order, and a first leaf direction, a second leaf direction and a third leaf direction are obtained according to the second eigenvector, and a reference coordinate system is constructed according to the first leaf direction, the second leaf direction and the third leaf direction. The first leaf direction is the first sorted second eigenvalue, the second leaf direction is the second sorted second eigenvalue, and the third leaf direction is the third sorted second eigenvalue.

[0168] Then, a curve mapping step is performed, in which, when the target leaf direction is the first leaf direction, the target curve is the vein curve, the target point set is the vein point set, and the target length is the leaf length; when the target leaf direction is the second leaf direction, the target curve is the leaf length curve, the target point set is the leaf length point set, and the target length is the leaf width.

[0169] In this embodiment, the curve mapping step includes:

[0170] The maximum point and the minimum point of the target leaf direction in the reference coordinate system are selected, the minimum point is taken as an initial candidate point, and the neighboring points of the initial candidate point are obtained.

[0171] For example, when the target leaf direction is the first leaf direction, the maximum point p max and the minimum point p min of the first leaf direction in the x-axis of the reference coordinate system are obtained. min The minimum point p i is taken as the initial candidate point p0, and then the neighborhood of the initial candidate point p0 is searched to obtain a plurality of neighboring points p .

[0172] Then, a first distance between each neighboring point and the initial candidate point and a second distance between each neighboring point and the maximum point are calculated, and a distance sum of the first distance and the second distance is calculated.

[0173] For example, when the target leaf direction is the first leaf direction, the first distance is a distance distance(p0, p i ), and the second distance is a distance distance(p i , p max ), and thus the distance sum D ist is expressed as: D ist = distance(p0, p i ) + distance(p i , p max ).

[0174] The neighboring point with the smallest distance sum is taken as a new candidate point. If a distance between the new candidate point and the initial candidate point satisfies a preset distance threshold (for example, 0.05 cm), the new candidate point is added to a target point set of the target curve. Otherwise, if the above condition is not satisfied, the searching process is repeated to continue searching for a neighboring point of the new candidate point. The new candidate point is taken as the initial candidate point, and iteration is continued until no new point is added to the target point set, and the point set extraction is completed.

[0175] Finally, a target curve is obtained according to a projection of the target point set, and a target length is calculated according to end points on the target curve. In an embodiment, when the target leaf direction is the first leaf direction, to reduce an error of the leaf vein point in the y-axis direction, the target point set is projected onto the xoz plane to obtain a leaf vein curve. When the target leaf direction is the second leaf direction, to reduce an error of the leaf width in the x-axis direction, the target point set is projected onto the yoz plane to obtain a leaf width curve.

[0176] Referring to Figure 16 , which is a schematic diagram of a leaf vein point set, and referring to Figure 17 , which is a schematic diagram of a leaf vein curve. Referring to Figure 18 , which is a schematic diagram of a leaf width point set, and referring to Figure 19 , which is a schematic diagram of a leaf width curve.

[0177] In an embodiment, a process of performing surface reconstruction on leaf point cloud data based on a greedy projection triangulation algorithm to obtain a leaf area of a plant includes:

[0178] First, the leaf point cloud data is projected onto a local two-dimensional coordinate plane based on the normal to obtain a two-dimensional projection plane. The projection points on the two-dimensional projection plane can be obtained by intersecting each point in the point cloud with its normal. Then, an initial triangular surface is selected from the two-dimensional projection plane as a starting surface. The initial surface is usually a surface formed by three points on the surface of the leaf. Next, the neighboring points of the starting surface are selected to expand the boundary of the surface. Specifically, starting from the initial surface, the boundary of the surface is expanded to gradually cover the entire leaf area. This process can be achieved by adding neighboring points, and it is ensured that there is no self-intersection or overlap during the expansion of the surface boundary.

[0179] After obtaining the surface boundary, the connection relationship between the three-dimensional points in the leaf point cloud data is determined according to the topological relationship of the projection points in the two-dimensional projection plane. The adjacent points that connect the surface boundary form a triangular surface. This step is repeated to obtain a plurality of triangular surfaces. The triangular surface forms a leaf grid surface, and the leaf grid surface is a planar triangular mesh. Referring to Figure 20 FIG. 4 is a schematic diagram of a leaf grid surface.

[0180] After obtaining the leaf grid surface, the area of the leaf is calculated based on the leaf grid surface. Specifically, the sum of the areas of all triangular surfaces is calculated. The area of a single triangular surface is calculated using the Heron formula and summed to obtain the area of the entire leaf.

[0181] Step S150: generating the phenotype parameters of the plant based on the plant height, plant width, leaf number, leaf length, leaf width, and leaf area.

[0182] At this point, the plant height, plant width, leaf number, leaf length, leaf width, and leaf area related phenotype parameters of the plant are obtained.

[0183] In an embodiment, the color parameter can also be obtained by quantifying the color grade of the plant to measure the color trait of the plant. The specific process is as follows: obtaining the histogram of the plant point cloud size data, converting the plant point cloud size data from the RCG space to the HSV space, and counting the H component related information. According to the H component, the pixel points are evenly divided into several bin intervals. Then, the pixel points belonging to the same bin interval are traversed, and the color of the pixel points is summed to obtain the mean value as the color of the corresponding bin interval. The mean color histogram is obtained. In the above calculation process, in order to improve the accuracy of the statistics, the range of the H component is doubled (0-360) as the reference. Finally, the intuitive expression of the proportion of different colors in the plant is obtained according to the mean color histogram, and the plant is divided into three color grades: yellow-green, green, and dark green. Referring to Figure 21 FIG. 5 is a schematic diagram of a color grade.

[0184] Since color traits as an important feature reflecting the nutritional status of plants, can be used for rapid classification of plants under different planting conditions, this embodiment takes the proportion of yellow-green color in the plant as the basis for classification to identify healthy plants and nitrogen-deficient plants.

[0185] For example, the color histogram is divided into 30 bins intervals, and the images of different plants are batch extracted and quantized from the video frames, and the distribution of plant color under normal planting conditions and nitrogen fertilizer test is counted.

[0186] Then, for example, the proportion of yellow-green color in the three color grades under the two planting conditions shown in Figure 22 The yellow-green color proportion of plants under normal planting conditions is set as [5%, 30%], and the yellow-green color proportion range under nitrogen fertilizer test is set as (30%, 80%].

[0187] In an embodiment, assuming that the plant is Brassica rapa, the key quantitative traits of Brassica rapa can be dynamically quantified in multiple time sequences, thereby providing data and technical support for key trait research and growth pattern analysis in Brassica rapa breeding process.

[0188] Firstly, the whole growth period of Brassica rapa is divided into seedling stage, growth stage, bolting stage and flowering stage, and then 5-10 Brassica rapa plants are randomly selected for three-dimensional reconstruction and point cloud model preprocessing such as background removal, noise removal and simplification to obtain plant point cloud data. The phenotypic parameters of the plant including plant height, plant width, leaf number, leaf length, leaf width and leaf area are calculated, and the leaf number is the average value of Brassica rapa in the growth process. Then, the Brassica rapa plants under different health conditions are classified by the eye-catching level in the color trait measurement.

[0189] Then, the obtained phenotypic parameters are visualized to intuitively describe the change of the average value of each key quantitative trait of Brassica rapa in the four growth stages. In this embodiment, referring to Figures 23a to 23f , the waterfall chart of the change of the average value of each key quantitative trait of Brassica rapa in the four growth stages is drawn. The three-dimensional waterfall chart is drawn by Origin software, the x-axis is the four growth stages of the whole growth period, the y-axis is the average value of the quantitative trait, and the z-axis can specify multiple Brassica rapa varieties. Among them, Figure 23a is the plant height growth time sequence waterfall chart generated by the multi-time sequence dynamic quantification module of the key traits of Brassica rapa; Figure 23b is the plant width growth time sequence waterfall chart generated by the multi-time sequence dynamic quantification module of the key traits of Brassica rapa; Figure 23c is the leaf length growth time sequence waterfall chart generated by the multi-time sequence dynamic quantification module of the key traits of Brassica rapa; Figure 23d is the leaf width growth time sequence waterfall chart generated by the multi-time sequence dynamic quantification module of the key traits of Brassica rapa; Figure 23eA leaf area growth time sequence waterfall chart generated by a multi-time dynamic quantification module for key traits of Chinese cabbage; Figure 23f A leaf number growth time sequence waterfall chart generated by a multi-time dynamic quantification module for key traits of Chinese cabbage.

[0190] Through the above process, the embodiments of the present application fill the gap in the visualization analysis of key phenotypic traits of Chinese cabbage during the whole growth period, promote the standardization process of multi-time monitoring of plant trait indicators, and provide technical support for the cultivation process of Chinese cabbage.

[0191] In an embodiment, referring to Figure 24 The overall flowchart of the embodiments of the present application.

[0192] First, a 360° rotating video of the plant is obtained.

[0193] Then, the 360° rotating video of the plant is preprocessed, including: preliminary denoising of the video frame sequence, three-dimensional reconstruction to obtain initial point cloud data of the plant, size recovery of the plant, selection of a filter for noise removal, and simplification of the plant point cloud data to obtain plant point cloud data.

[0194] Then, initial voxel data of the plant are obtained from the plant point cloud data, and voxel segmentation is performed on the initial voxel data of the plant to detect stem points and separate stems and leaves, to obtain a stem-leaf segmentation result of the plant.

[0195] Phenotypic parameters are extracted from the stem-leaf segmentation result, including: overall shape: plant height and plant width; organ traits: leaf number, leaf length, leaf width, and leaf area; color shape: HSV model and mean color histogram.

[0196] Finally, visualization is performed according to the extracted phenotypic parameters to obtain dynamic quantification results of multi-time phenotypic parameters.

[0197] The technical scheme provided by the embodiments of the present disclosure obtains initial voxel data of the plant from plant point cloud data, and performs voxel segmentation on the initial voxel data of the plant to obtain stem point cloud data and leaf point cloud data of the plant, then performs principal component analysis on the plant point cloud data to obtain plant height and plant width, and performs similarity clustering on the leaf point cloud data to obtain leaf number of the plant; further, performs principal component analysis on the leaf point cloud data to obtain leaf length and leaf width of the plant, and performs surface reconstruction on the leaf point cloud data to obtain leaf area of the plant, and finally obtains phenotypic parameters including plant height, plant width, leaf number, leaf length, leaf width, and leaf area. The embodiments of the present application first perform segmentation on point cloud data, thereby extracting parameters for different plant parts, improving data processing efficiency and result accuracy, then use principal component analysis to obtain plant height, plant width, leaf length, and leaf width, use similarity clustering to obtain leaf number, and then perform surface reconstruction to obtain leaf area, which can use point cloud data to obtain rich phenotypic parameters.

[0198] The embodiment of the present disclosure further provides a plant phenotype parameter extraction device, which can implement the plant phenotype parameter extraction method. Figure 25 The device comprises:

[0199] The point cloud data acquisition module 2510 is configured to acquire plant point cloud data of the plant.

[0200] The stem-leaf segmentation module 2520 is configured to obtain plant preliminary voxel data according to the plant point cloud data, and perform voxel segmentation on the plant preliminary voxel data to obtain a stem-leaf segmentation result of the plant, wherein the stem-leaf segmentation result comprises stem point cloud data and leaf point cloud data.

[0201] The first parameter acquisition module 2530 is configured to perform principal component analysis on the plant point cloud data to obtain plant height and plant width, and perform similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant.

[0202] The second parameter acquisition module 2540 is configured to perform principal component analysis on the leaf point cloud data to obtain leaf length and leaf width, and perform surface reconstruction on the leaf point cloud data to obtain leaf area of the plant.

[0203] The phenotype parameter acquisition module 2550 is configured to generate a plant phenotype parameter based on the plant height, the plant width, the number of leaves, the leaf length, the leaf width and the leaf area.

[0204] The specific implementation of the plant phenotype parameter extraction device of the embodiment is basically the same as the specific implementation of the plant phenotype parameter extraction method, and will not be repeated here.

[0205] The embodiment of the present disclosure further provides an electronic device, which comprises:

[0206] at least one memory;

[0207] at least one processor;

[0208] at least one program;

[0209] The program is stored in the memory, and the processor executes the at least one program to implement the plant phenotype parameter extraction method of the present disclosure. The electronic device can be any intelligent terminal, including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0210] Please refer to Figure 26 , Figure 26 which illustrates the hardware structure of the electronic device of another embodiment, and the electronic device comprises:

[0211] The processor 2601 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0212] The memory 2602 can be implemented by a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 2602 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 2602 and are called and executed by the processor 2601 to implement the plant phenotype parameter extraction method of the embodiments of the present disclosure.

[0213] The input / output interface 2603 is configured to implement information input and output.

[0214] The communication interface 2604 is configured to implement the communication interaction between the device and other devices. The communication can be implemented by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0215] The bus 2605 is configured to transmit information between various components (for example, the processor 2601, the memory 2602, the input / output interface 2603, and the communication interface 2604) of the device.

[0216] The processor 2601, the memory 2602, the input / output interface 2603, and the communication interface 2604 are connected to each other by the bus 2605 to realize the communication connection between them in the device.

[0217] The embodiments of the present disclosure further provide a storage medium, which is a computer readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, the above plant phenotype parameter extraction method is implemented.

[0218] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely relative to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0219] The plant phenotype parameter extraction method, plant phenotype parameter extraction device, electronic equipment and storage medium provided by the embodiments of the present disclosure obtain plant preliminary voxel data from plant point cloud data, perform voxel segmentation on the plant preliminary voxel data to obtain stem point cloud data and leaf point cloud data of the plant, then perform principal component analysis on the plant point cloud data to obtain plant height and plant width of the plant, and perform similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant; then perform principal component analysis on the leaf point cloud data to obtain leaf length and leaf width of the plant, and perform surface reconstruction on the leaf point cloud data to obtain leaf area of the plant, and finally obtain the phenotype parameters including the plant height, the plant width, the number of leaves, the leaf length, the leaf width and the leaf area. The embodiments of the present disclosure first perform segmentation on the point cloud data, so as to extract parameters for different plant parts, improve the data processing efficiency and the result accuracy, then use the principal component analysis method to obtain the plant height, the plant width, the leaf length and the leaf width, use the similarity clustering to obtain the number of leaves, and then perform surface reconstruction to obtain the leaf area, so that rich phenotype parameters can be obtained by using the point cloud data.

[0220] The embodiments described in the embodiments of the present disclosure are used to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0221] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present disclosure, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0222] The device embodiments described above are only schematic, and units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0223] It is understood by those skilled in the art that all or some of the steps in the methods disclosed above, the functional modules / units in the systems and devices can be implemented by software, firmware, hardware, and appropriate combinations thereof.

[0224] The terms "first", "second", "third", "fourth" etc. (if any) in the description of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0225] It should be understood that in the present disclosure, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0226] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0227] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0228] In addition, each functional unit in various embodiments of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0229] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present disclosure essentially or the part that contributes to the prior art, or all or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of various embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0230] The preferred embodiments of the embodiments of the present disclosure are described above with reference to the accompanying drawings, and the scope of the present disclosure is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present disclosure shall be within the scope of the embodiments of the present disclosure.

Claims

1. A method for extracting a plant phenotype parameter, characterized by, The method comprises the following steps: acquiring plant point cloud data of a plant; voxelizing the plant point cloud data based on an octree to obtain a plurality of voxel units; calculating similarity values between adjacent voxel units, merging the voxel units with high similarity values to obtain new voxel units, updating a central voxel, and repeating the step until the position of the central voxel is unchanged, thereby obtaining preliminary voxel data of the plant, obtaining the centroids and corresponding normal vectors of two adjacent voxel units, generating a voxel vector based on the positions of the two centroids, generating first convexities of the two voxel units based on a first included angle between the voxel vector and the two normal vectors, calculating a cross product between the two normal vectors and a second included angle between the voxel vector and the cross product, generating second convexities of the two voxel units based on the relationship between the second included angle and a preset threshold, and obtaining convexities of the two voxel units based on the first convexities and the second convexities, wherein the convexities include convexity and concavity; selecting any voxel unit as a seed point, labeling a region based on the convexities of the voxel units, labeling the region as convex if the voxel unit is convex, and selecting other voxel units to update the seed point until each voxel unit is labeled; obtaining each region based on the labeling result, obtaining stem-leaf segmentation results of the plant based on the regions, and wherein the stem-leaf segmentation results include leaf point cloud data; performing principal component analysis on the plant point cloud data to obtain plant height and plant width, and performing similarity clustering on the leaf point cloud data to obtain the number of leaves of the plant; 2. The plant phenotype parameter extraction method according to claim 1, characterized by, performing principal component analysis on the leaf point cloud data to obtain leaf length and leaf width, and performing surface reconstruction on the leaf point cloud data to obtain the area of the leaves of the plant; generating phenotype parameters of the plant based on the plant height, the plant width, the number of leaves, the leaf length, the leaf width, and the area of the leaves. The stem-leaf segmentation results also include stem point cloud data; obtaining each region based on the labeling result, and obtaining the stem-leaf segmentation results based on the regions, comprises the following steps: randomly selecting point cloud of a region as a sample subset; obtaining a cylindrical model based on minimum variance estimation, calculating sample deviation of sample points in the sample subset from the cylindrical model, and taking sample points with sample deviation less than a preset threshold as inliers of the cylindrical model, and counting the number of inliers; 3. The plant phenotype parameter extraction method according to claim 1, characterized by, repeating the iteration process to take the cylindrical model with the most inliers as a target model; taking the inliers of the target model as stem points, and taking the plant point cloud data corresponding to the region based on the stem points as the stem point cloud data, and taking the rest as the leaf point cloud data. The principal component analysis on the plant point cloud data to obtain the plant height and the plant width comprises the following steps: obtaining a centroid of the plant point cloud data, establishing a first covariance matrix based on the centroid, solving the first covariance matrix to obtain three first eigenvalues and a first eigenvector of the first eigenvalues, and sorting the first eigenvalues in descending order, and obtaining a first plant direction, a second plant direction and a third plant direction according to the first eigenvector; establishing a rotation and translation matrix based on the first plant direction and the centroid, and performing coordinate correction on the plant point cloud data by using the rotation and translation matrix to obtain corrected point cloud data; generating a rectangular enclosing region according to the corrected point cloud data and a reference coordinate system, and calculating the side length of the rectangular enclosing region; taking the side length in the first plant direction as the plant height; taking the larger value of the side length in the second plant direction and the side length in the third plant direction as the plant width.

4. The plant phenotype parameter extraction method according to claim 1, characterized by, the similarity clustering of the leaf point cloud data to obtain the number of leaves of the plant, comprising: obtaining a set of neighboring points of each point in the leaf point cloud data; obtaining a corresponding fitting plane according to the set of neighboring points, calculating the normal and curvature of the fitting plane, and selecting the point corresponding to the fitting plane with the minimum curvature as an initial seed point; based on the normal angle threshold and the curvature threshold, merging similar points in the neighborhood of the initial seed point into the growth region where the initial seed point is located; selecting a new seed point, merging similar points in the neighborhood of the new seed point into the growth region, repeating the step until the iteration of the growth region is completed to obtain a leaf region; obtaining the number of points in the leaf region, taking the leaf region with a number greater than a preset number threshold as a target leaf region, and obtaining the number of leaves according to the number of target leaf regions.

5. The plant phenotype parameter extraction method according to claim 1, characterized by, the principal component analysis of the leaf point cloud data to obtain the leaf length and leaf width of the plant, comprising: obtaining a centroid of the leaf point cloud data, establishing a second covariance matrix based on the centroid, solving the second covariance matrix to obtain three second eigenvalues and a second eigenvector of the second eigenvalues, and sorting the second eigenvalues in descending order, and obtaining a first leaf direction and a second leaf direction according to the second eigenvector; performing a curve mapping step, the curve mapping step comprising: selecting a maximum point and a minimum point of the target leaf direction in the reference coordinate system, taking the minimum point as an initial candidate point, and obtaining neighboring points of the initial candidate point; calculating a first distance between each of the neighboring points and the initial candidate point and a second distance between each of the neighboring points and the maximum point, respectively calculating the distance sum of the first distance and the second distance; taking the neighboring point with the minimum distance sum as a new candidate point, if the distance between the new candidate point and the initial candidate point satisfies a preset distance threshold, adding the new candidate point to a target point set of a target curve, otherwise, reselecting the new candidate point; Continue iteration with the new candidate point as the initial candidate point until no new point is added in the target point set, obtain the target curve according to the projection of the target point set, and calculate the target length according to the target curve; When the target leaf direction is the first leaf direction, the target curve is a vein curve, the target point set is a vein point set, and the target length is a leaf length; when the target leaf direction is the second leaf direction, the target curve is a leaf length curve, the target point set is a leaf length point set, and the target length is a leaf width.

6. The plant phenotype parameter extraction method according to claim 1, characterized by, The method further comprises: projecting the leaf point cloud data based on a normal to obtain a two-dimensional projection surface; selecting an initial triangular surface as a starting surface from the two-dimensional projection surface, and selecting adjacent points of the starting surface to perform boundary expansion to obtain a surface boundary, and connecting adjacent points of the surface boundary to form a triangular surface; repeat the step until a leaf grid surface composed of the triangular surface is obtained; calculating the leaf area based on the leaf grid surface.

7. The method according to any one of claims 1 to 6, further comprising: obtaining plant initial point cloud data of the plant; performing multiple diameter fitting on a cultivation container of the plant to obtain an average diameter, and obtaining a point cloud recovery ratio based on an actual diameter of the cultivation container and the average diameter; recovering the scale of the plant initial point cloud data based on the point cloud recovery ratio to obtain plant point cloud size data; obtaining a histogram of the plant point cloud size data, removing the background based on the histogram to obtain plant point cloud foreground data; selecting a corresponding filter based on the type of noise points in the plant point cloud foreground data to remove the noise points, and obtaining the plant point cloud data.

8. A plant phenotype parameter extraction apparatus characterized by comprising: The method further comprises: a point cloud data acquisition module configured to obtain plant point cloud data of the plant; a stem-leaf segmentation module configured to voxelize the plant point cloud data based on an octree to obtain a plurality of voxel units; The similarity values between each adjacent voxel unit are calculated, the voxel units with high similarity values are merged to obtain new voxel units, and the center voxel is updated, and the step is repeatedly executed until the position of the center voxel is unchanged, preliminary voxel data of the plant is obtained, the centroids of two adjacent voxel units and corresponding normal vectors are obtained, and a voxel vector is generated according to the positions of the two centroids; a first convexity of the two voxel units is generated according to a first included angle between the voxel vector and two normal vectors; a cross product between the two normal vectors is calculated, and a second included angle between the voxel vector and the cross product is calculated, a second convexity of the two voxel units is generated according to a relationship between the second included angle and a preset threshold, and a convexity of the two voxel units is obtained based on the first convexity and the second convexity, the convexity includes convexity and concavity; any voxel unit is selected as a seed point, and the region is labeled based on the convexity of the voxel unit, if the voxel unit is convex, the voxel unit is labeled as the region, until the seed point and the adjacent voxel unit are both concave, other voxel units are selected to update the seed point, the step is repeatedly executed until each voxel unit is labeled; each region is obtained according to the labeling result, and stem and leaf segmentation results of the plant are obtained according to the regions, and the stem and leaf segmentation results include leaf point cloud data; The first parameter acquisition module is configured to perform principal component analysis on the plant point cloud data to obtain a plant height and a plant width, and perform similarity clustering on the leaf point cloud data to obtain a leaf number of the plant; The second parameter acquisition module is configured to perform principal component analysis on the leaf point cloud data to obtain a leaf length and a leaf width, and perform surface reconstruction on the leaf point cloud data to obtain a leaf area of the plant; The phenotype parameter acquisition module is configured to generate a phenotype parameter of the plant based on the plant height, the plant width, the leaf number, the leaf length, the leaf width, and the leaf area.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the plant phenotype parameter extraction method of any one of claims 1 to 7 when executing the computer program.

10. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the plant phenotype parameter extraction method of any one of claims 1 to 7.

Images