A tobacco phenotype trait extraction method based on three-dimensional point cloud

Abstract

The present application relates to a kind of tobacco phenotypic character extraction methods based on three-dimensional point cloud, comprising the following steps: step one, data preprocessing: carry out voxel downsampling processing, denoising processing and coordinate correction, obtain corrected tobacco point cloud data;Step two, plant organ segmentation: first, according to the stem feature, the stem is segmented out, then complete leaf rough segmentation, then carry out the differentiation of overlapping leaf, obtain the composition point of stem and leaf;Step three, detection and repair of leaf hole;Step four, phenotype data extraction: the extraction of plant height adopts the way of approximate measurement plant height of segmented type;The extraction of leaf area adopts poisson surface reconstruction algorithm to carry out the meshing of leaf;When extracting leaf angle, first extract skeleton point, obtain the direct angle with stem, then obtain leaf angle by calculation method.The present application not only realizes the extraction work of tobacco plant height, leaf number, leaf area, leaf angle and other phenotypic characters, but also improves the accuracy of phenotypic character extraction.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of plant phenotype data measurement and extraction, and particularly relates to a tobacco phenotype extraction method based on three-dimensional point cloud. BACKGROUND

[0002] Traditional plant phenotype data is mostly collected manually, which is not only low in efficiency, time-consuming, high in cost and difficult, but also low in precision, seriously interfered by manual intervention and strong in subjective consciousness, and even may cause damage to plants during the collection process. With the advent of computer vision technology, plant phenotype extraction technology based on images has developed rapidly, which mainly includes two categories: two-dimensional image-based phenotype extraction technology and three-dimensional point cloud image-based phenotype extraction technology.

[0003] Among them, the two-dimensional image-based phenotype extraction technology has more applications in the extraction of plant height, plant type and canopy area determination due to its simple acquisition and low cost. However, due to the lack of spatial depth information characteristics of two-dimensional images, in actual application, in order to solve the problem of mutual occlusion of objects in actual three-dimensional space, a lossy measurement method is often adopted for measuring plant leaf area or leaf angle, that is, the leaves or part of the stems are taken off from the plant and placed on a two-dimensional plane for image acquisition. This method cannot meet the needs of high-throughput and dynamic detection of plant development phenotypes, and the problem of scale change between the actual object size and the control object may exist due to the problem of two-dimensional image shooting angle. Therefore, this technology is restricted by many factors in application and is only applied in specific scenarios for the measurement of certain plant phenotypes. Compared with the two-dimensional image-based phenotype extraction technology, the three-dimensional point cloud image-based phenotype extraction technology solves the problems of lack of spatial depth information and mutual occlusion of objects, can well describe the three-dimensional spatial information, shape characteristics and various three-dimensional morphological parameters of plants, and the extracted phenotype data has higher precision and better robustness, which makes it possible to measure plant phenotype characteristics with high throughput, and has become a research hotspot in the study of Arabidopsis, corn, wheat and rice phenomics. As one of the important model plants and economic crops, tobacco has a huge biomass, and unlike corn, wheat and rice, which mainly harvest seeds, leaves are an important economic organ for harvest. Therefore, it is more important to study the plant height, leaf number, leaf area, stem-leaf angle and other characteristics of tobacco. Due to the wide and whorled leaves of tobacco, it is more challenging to collect and process three-dimensional point cloud images compared with other plants, and there is still no three-dimensional point cloud plant extraction method that can accurately extract the phenotype parameters such as plant height, leaf number, leaf area and leaf angle of tobacco.

[0004] Therefore, there is an urgent need to develop a three-dimensional point cloud-based tobacco phenotype extraction method that can accurately extract the leaf area, stem-leaf angle and other characteristics of tobacco plants, in order to provide support for tobacco germplasm resource identification and simple and easy-to-use tobacco variety cultivation, and provide a reference for similar research on other plants. SUMMARY

[0005] The present application aims to overcome the defects of the existing plant phenotype extraction technology, develop a tobacco phenotype trait extraction method based on three-dimensional point cloud, which can accurately obtain the phenotype parameters such as tobacco plant height, leaf number, leaf area, and leaf angle, and realize the rapid and efficient acquisition of the phenotype characteristics of tobacco plants.

[0006] To achieve the above-mentioned purpose, the technical solutions adopted by the present application are as follows:

[0007] A tobacco phenotype trait extraction method based on three-dimensional point cloud, comprising the following steps:

[0008] Step one, data preprocessing: sequentially performing voxel downsampling processing, denoising processing and coordinate correction on the original point cloud image data to obtain corrected tobacco point cloud data;

[0009] Step two, plant organ segmentation: first, performing stem segmentation according to stem characteristics, and then performing leaf segmentation (including non-overlapping leaf and overlapping leaf segmentation) to obtain the constituent points of stems and leaves;

[0010] Among them, when the stem is segmented, the stem is first preliminarily screened in a step-by-step screening manner, and then the leaf points adhered to the stem are removed to obtain the constituent points of the stem;

[0011] When the leaf is segmented, first, based on the region growing algorithm of the normal vector, different leaves are preliminarily segmented, and then the overlapping leaves are segmented by iteratively updating the cluster center and surface fitting, and finally the constituent points of each leaf are obtained;

[0012] Step three, detection and repair of leaf holes

[0013] First, radius filtering is used to preliminarily screen the leaf to screen out points with few neighboring points; then, it is judged whether the screened points with few neighboring points are boundary points or internal points, and all boundary points of the leaf are screened out; then, it is judged whether the screened boundary points are peripheral boundary points or hole boundary points, and all hole boundary points are screened out, thereby completing the detection of holes; then, a cubic surface equation is used to interpolate and fit the leaf to complete the hole repair, and the final tobacco point cloud data is obtained;

[0014] Step four, phenotype data extraction: using the final tobacco point cloud data obtained after hole repair to perform phenotype extraction to obtain the phenotype information of the plant, wherein the phenotype includes one or more of plant height, leaf number, leaf area, and leaf angle; and the phenotype extraction includes one or more of plant height extraction, leaf number extraction, leaf area extraction, and leaf angle extraction.

[0015] As a further technical solution, in step one, the original point cloud image data is obtained by scanning tobacco plants using a 3D scanning device;

[0016] In step one, the original point cloud image data is sequentially subjected to voxel downsampling, denoising, and coordinate correction to obtain corrected tobacco point cloud data, specifically including:

[0017] First, voxel downsampling is used to reduce the number of points in the original point cloud image data. Then, radius filtering and statistical filtering are used to filter out outliers and irregularities in the downsampled point cloud image data. After that, normal vector estimation is performed, and finally coordinate correction is performed to obtain the corrected tobacco point cloud data.

[0018] As a further technical solution, the normal vector estimation is specifically performed as follows: A point is randomly selected from the downsampled point cloud image data. Find the point using a kd-tree of Find the nearest neighbor points, and then find the... Nearest neighbor points and points The principal component analysis (PCA) algorithm is used to obtain the results. Nearest neighbor points and points The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by itself is used as the eigenvector of that point. Estimation of normal vector

[0019] The coordinate correction includes two steps: First, the stalk orientation is estimated by minimizing the magnitude of the sum of the outer products of the normal vectors of all points in the point cloud image data of the entire tobacco plant, based on the downsampled point cloud image data. This yields the initial estimated stalk orientation (denoted as...). Then, the initial orientation of the stem is estimated. Perform iterative corrections to determine the final estimated direction of the stem (denoted as...). Finally, rotate the z-axis of the coordinate system to the final estimated direction of the stem. The position is used to project all points of the downsampled point cloud image data onto the rotated image. On the plane, calculate the coordinates of the centroid of all projected points. Then, translate the origin. The coordinates are corrected at the centroid point to obtain the corrected tobacco point cloud data.

[0020] As a further technical solution, during coordinate correction, the initial estimated direction of the stalk is obtained by minimizing the magnitude of the sum of the outer products of the normal vectors of all points in the point cloud image data of the entire tobacco plant. Specifically, it includes the following steps:

[0021] Let the number of points in the down-sampled point cloud image data be , and the normal vector of each point be .

[0022] Let the initial estimated direction of the stem be , and solve the following optimization problem:

[0023]

[0024] Let , and convert the target problem to:

[0025]

[0026]

[0027] The initial estimated direction of the stem is obtained as ;

[0028] As a further technical solution, when performing coordinate correction, the initial estimated direction of the stem is iteratively corrected to determine the final estimated direction of the stem , specifically including the following steps:

[0029] Step (1), set ;

[0030] Step (2), extract the cylindrical body with as the generatrix and as the radius;

[0031] Step (3), use the PCA algorithm to extract the first principal component direction of the cylindrical body ;

[0032] Step (4), if , end the loop; otherwise, set and reduce to , repeat steps (2)-(4) until the loop ends, i.e., the final estimated direction of the stem is determined.

[0033] As a further technical solution, in step two, when segmenting the stem, a step-by-step screening method is used to preliminarily screen the stem, and then the leaf points adhered to the stem are removed to obtain the constituent points of the stem, specifically including the following steps:

[0034] Step 1), based on the corrected tobacco point cloud data obtained after coordinate correction, use pass-through filtering in the direction, filtering out points near the origin;

[0035] Step 2), based on the re-estimated normal vector (denoted as ), the normal vector is unitized, and the data points of are filtered out;

[0036] Step 3), using statistical filtering to filter out abnormal points and outliers in the filtered data points of , completing the preliminary screening and obtaining the point cloud data after preliminary screening;

[0037] Step 4), clustering the point cloud data after preliminary screening using a density clustering algorithm based on Euclidean distance, and filtering out the class with the most points;

[0038] Step 5), using a region growing algorithm based on the normal vector to cluster the class with the most points filtered out in step 4), and continue to filter out the class with the most points, all points in the class with the most points filtered out this time are the constituent points of the final stem.

[0039] As a further technical solution, in step two, an iterative updating of the clustering center and surface fitting is used to segment the overlapping leaves, and finally the constituent points of each leaf are obtained. The specific steps are as follows:

[0040] Step 1, select the points close to the stem DBSCAN density clustering, obtain clusters and the initial clustering center of each cluster; record each cluster in the clusters as , and the clustering center of the cluster is , wherein the number of clusters is the number of overlapping leaves; if no leaf segmentation is performed, if , steps 2-5 are sequentially executed for leaf segmentation;

[0041] Step 2, for each point in the clusters, calculate the nearest clustering center to the point in the axial direction, and reclassify the point to the cluster to which the point belongs; after all points are reclassified, update the clustering center ;

[0042] Step 3, for each point in the ​​, calculate the cluster to which the point belongs in the whole coordinate space the nearest cluster center , re-divide the point to the cluster to which the point belongs ; after all points are re-divided, update the cluster center ;

[0043] Step 4, set , for the cluster , use the DBSCAN density clustering algorithm to obtain sub-clusters , arrange all sub-clusters in the cluster in descending order of the number of points, and update to , add the sub-cluster to the cluster ; update to , and repeat step 4 until ;

[0044] Step 5, re-calculate the cluster to which each point belongs in each cluster, and the specific steps are as follows: for each cluster , fit a cubic surface ; randomly select a point , calculate , and then the point belongs to the cluster ; after re-calculation of the cluster of all points, the points in each cluster form a leaf, and each leaf or each point is clearly attributed to a leaf.

[0045] As a further technical solution, in step three, the method for judging whether the point with few near neighbor points is a boundary point or an internal point includes:

[0046] for any point in the set of points with few near neighbor points , use the RANSAC algorithm to fit the point and its near neighbor points into a plane, then put the point and its near neighbor points on the plane, and connect the projections of the near neighbor points of the point on the plane to form vectors, which are arranged in clockwise order, and each two adjacent vectors form an angle, and there are angles; calculate the maximum value of the angles, and if the maximum value is greater than a preset angle threshold value, the point is a boundary point; otherwise, the point is an internal point. Greater than a given threshold Then it is determined to be a point. It is a boundary point; otherwise, it is determined to be a point. It is an interior point;

[0047] Step three, the method for determining whether the selected boundary points are outer boundary points or hole boundary points, includes:

[0048] Project all points on the blade onto On the plane, the points among the selected boundary points Connect the vectors to their nearest neighbors on the leaf, calculate the angle between adjacent vectors after arranging all vectors clockwise, and find the maximum value of the angle. ;like Greater than a given threshold Then the point is considered It is an outer boundary point; otherwise, it is considered a point. It is the boundary point of the hole;

[0049] In step three, the equation of the cubic surface obtained through fitting is used. Interpolation is performed to repair the holes and obtain the final tobacco point cloud data, specifically including:

[0050] Find the respective boundary points of the hole Then, a cubic surface is obtained by fitting all the hole boundary points and their respective nearest neighbors using a nonlinear least squares method. Then iterate through the holes to find the points inside. Coordinates, using the fitted cubic surface at the found points inside the hole. Interpolation is performed to complete the hole repair.

[0051] As a further technical solution, step four, the method for extracting plant height, includes: firstly, in... The stems are cut evenly along the axial direction. The stalk is then segmented, and the RANSAC algorithm is used to fit a cylindrical model for each segment, and the height of the cylinder in each cylindrical model is calculated. Finally, the sum of the heights of all cylinders is calculated as the length of the stem, i.e., the plant height.

[0052] Leaf number extraction method: The number of leaves in the final tobacco point cloud data obtained after hole repair can be directly counted;

[0053] The method for extracting leaf area includes: first, processing the leaf using the Poisson surface reconstruction algorithm to generate a triangular mesh on the leaf surface; then, calculating the area of ​​the triangles using Heron's formula; and finally, using the sum of the areas of all triangles as the surface area of ​​the leaf, i.e., the leaf area.

[0054] The extraction method of the leaf angle comprises the following steps: firstly, extracting the skeleton points of the leaf blade; secondly, selecting the skeleton points close to the stem , and setting the skeleton point closest to the stem as point ; thirdly, connecting each of the rest of the -1 skeleton points with point to form a straight line, and calculating the angle between the -1 straight lines and the stem; fourthly, removing the abnormal values in the angles between the -1 straight lines and the stem by using the box plot method, and calculating the average value of all the angles after removing the abnormal values, i.e. the leaf angle.

[0055] As a further technical scheme, in step four, the method for extracting the skeleton points of the leaf blade comprises the following steps:

[0056] Step a, calculating the first principal component direction of the leaf blade by using the PCA algorithm

[0057] Step b, evenly dividing the leaf blade into a plurality of segments in the first principal component direction ;

[0058] Step c, performing DBSCAN density clustering on each segment to divide the segment into a plurality of classes;

[0059] Step d, calculating the cluster center of each class, and taking the cluster center as a skeleton point.

[0060] Compared with the prior art, the present application has the following beneficial effects:

[0061] 1. When the point cloud image data of the plant is preprocessed, the present application performs adaptive transformation of the coordinates, which does not need to fit a plane, does not require a flat ground, and does not need to fix the camera position. It directly starts from the point cloud image data, and for the scattered data set with non-uniform coordinate system, the minimum of the modulus of the outer product sum of all the point normal vectors of the tobacco point cloud image is used to obtain the initial estimation direction of the stem, and the first principal component direction is calculated in the cylindrical range as the final estimation direction of the stem. A unified method is used to complete the translation and rotation operation of the coordinates, which has good universality and will not be affected by many restrictions such as the flatness of the ground and the position of the camera during data acquisition.

[0062] ​2、The plant organ segmentation includes semantic segmentation and instance segmentation, when plant organ segmentation is carried out based on point cloud image data, the stem is first segmented according to the stem characteristics (normal vector, shape and spatial feature), then the leaf rough segmentation is completed, and then the overlapping leaf is distinguished, the plant organ segmentation process set can be well applied to tobacco point cloud image data, the stem-leaf segmentation and overlapping leaf segmentation are completed, and the accuracy of subsequent phenotype extraction is improved.

[0063] 3、The holes in the leaf are detected and repaired after organ segmentation in the application, and the accuracy of subsequent phenotype extraction is further improved, especially the accuracy of leaf area extraction data.

[0064] 4、In the plant phenotype trait extraction, the plant height extraction adopts the segmented approximate measurement of plant height, which is closer to the actual length of the stem, the leaf area extraction adopts the smoother Poisson surface reconstruction algorithm for leaf meshing, and the requirement for point cloud density is reduced compared with the traditional technology in the calculation of leaf area, and in the extraction of leaf angle, the skeleton points are extracted by segmenting and clustering in the first principal component direction, and the direct angle with the stem is calculated, and then the leaf angle is obtained by calculation, the method is not only suitable for short petiole tobacco, but also has good performance effect on tobacco with slender petiole, and the adaptability to different tobacco plant types is improved.

[0065] In summary, the data preprocessing method, plant organ segmentation method and phenotype trait extraction method are improved, and the detection and repair steps of the leaf hole are added, so that the extraction of the phenotype characteristics such as plant height, leaf number, leaf area and leaf angle of tobacco plant can be realized, and the extraction accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 The flow chart of one embodiment of the application;

[0067] Figure 2 The tobacco plant image before and after coordinate correction in one embodiment of the application;

[0068] Figure 3 The schematic diagram of the leaf segmentation result in one embodiment of the application;

[0069] Figure 4 The identification process diagram of the hole boundary point in the hole detection in one embodiment of the application;

[0070] Figure 5 The schematic diagram of the projection of the peripheral boundary point and the hole boundary point to the plane in one embodiment of the application.

[0071] Figure 6 ​A solving diagram of the height of the cylinder in one embodiment of the present application is shown in the figure; DETAILED DESCRIPTION

[0072] The technical solutions of the present application will be described clearly and completely in combination with specific embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0073] Embodiment 1

[0074] As Figure 1 A tobacco phenotype trait extraction method based on three-dimensional point cloud is shown, which comprises the following steps:

[0075] Step 0, data acquisition:

[0076] A 3D scanning device (F6sr 3D scanner of Mantis vision company in Israel) is used to scan the tobacco plant to obtain the original point cloud image data of the tobacco plant; the original point cloud image data includes the spatial coordinates of each point on the tobacco plant color information rough point cloud normal vector ;

[0077] Step 1, data preprocessing:

[0078] Step 1.1, downsampling: the original point cloud image data obtained by the 3D scanning device is processed and reduced in point number by using voxel downsampling method;

[0079] The original point cloud data is unevenly distributed in density, and there may be 1-200 points in a unit voxel cube;

[0080] Voxel downsampling is to voxelize the three-dimensional space, and then sample a point in each voxel, usually the center point or the point closest to the center is used as the sampling point, so as to reduce the number of points in the point cloud. The voxel downsampling method can take into account the sampling speed and effect. In this embodiment, the side length of each voxel cube is set to 4mm, which can reduce the number of points to 1 / 50 of the original data, and basically retains the shape and other characteristics, which will not affect the subsequent work;

[0081] Step 1.2, point cloud denoising: using radius filtering and statistical filtering to filter out abnormal points and outliers in the point cloud image data obtained by downsampling, and then estimating the normal vector;

[0082] Each point in the downsampled point cloud image data should have a corresponding normal vector. Since the original point cloud image data may not contain this information or the normal vector may be inaccurate, normal vector estimation is required.

[0083] Specifically, when performing normal vector estimation, any point is randomly selected from the downsampled point cloud image data as a point. Find the point using a kd-tree of Find the nearest neighbor points, and then find the... Nearest neighbor points and points The principal component analysis (PCA) algorithm is used to obtain the results. Nearest neighbor points and points The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by itself is used as the eigenvector of that point. Estimation of the normal vector;

[0084] Step 1.3, Coordinate Correction: Perform coordinate correction to obtain the corrected tobacco point cloud data;

[0085] Step 1.3.1: First, based on minimizing the magnitude of the sum of the outer products of the normal vectors of all points in the tobacco stalk point cloud image data, the initial estimated direction of the stalk is obtained. Specifically, the steps include the following:

[0086] Let the number of points in the downsampled point cloud image data be... The normal vector of each point is .

[0087] Let the initial estimated direction of the stem be... Solve the following optimization problem:

[0088]

[0089] remember The target problem is transformed into:

[0090]

[0091]

[0092] Obtain the initial estimated direction of the stem .

[0093] Step 1.3.2: Initial orientation of the stem Perform iterative corrections to determine the final estimated direction of the stem. ;

[0094] The method for iteratively correcting the stem orientation is as follows:

[0095] Step (1), set ;

[0096] Step (2), extract the cylinder with the origin as the center and the radius as the radius. ; ;

[0097] Step (3), extract the first principal component direction of the cylinder using the PCA algorithm. ;

[0098] The PCA algorithm used, i.e., the principal component analysis algorithm, is an unsupervised algorithm that does not rely on any class label information, and is a commonly used linear dimensionality reduction method. The goal of this algorithm is to map the data distribution characteristics of the original high-dimensional space to a lower-dimensional space through a certain linear projection, and to satisfy the following conditions in the projected dimensions: 1) preserve the variation information of the original data as much as possible; 2) the variables in the new dimensions are not correlated with each other.

[0099] Step (4), if , end the loop; otherwise, set and reduce the radius to , repeat steps (2)-(4) until the loop ends, i.e., the final estimated direction of the stem is determined. ;

[0100] Step 1.3.3, rotate the z-axis of the coordinate system to the position of the final estimated direction of the stem , project the point set of the down-sampled point cloud image data onto the rotated plane, calculate the centroid of all projected points , and translate the origin to the centroid to complete the coordinate correction, obtaining the corrected tobacco point cloud data. The images before and after coordinate correction are shown in Figure 2 .

[0101] Step 2, segment the stem: first, use a step-by-step screening method to preliminarily screen the stem, then remove the leaf points adhering to the stem to obtain the constituent points of the stem.

[0102] Step 2.1, based on the tobacco point cloud data obtained after coordinate correction, use straight-through filtering in the direction of to screen out points near the origin.

[0103] Step 2.2, based on the re-estimated normal vector (denoted as ), unitize the normal vector and screen out data points of .

[0104] Step 2.3, filter out abnormal points and outliers in the point set screened out in step 2.2 using statistical filtering, complete the preliminary screening, and obtain the point cloud data after preliminary screening;

[0105] Step 2.4, cluster the point cloud data after preliminary screening using a density clustering algorithm based on Euclidean distance to screen out the class with the most points;

[0106] Step 2.5, use a region growing algorithm based on the normal vector to cluster the class with the most points screened out in step 2.4 to continue screening out the class with the most points, and all points in the class with the most points screened out this time are the constituent points of the final stem; the angle threshold is 15° and the curvature threshold is 0.9 in this step of the embodiment when screening the class with the most points.

[0107] Among them, the purpose of steps 2.1-2.2 is mainly to preliminarily screen the stem, and steps 2.3-2.5 aim to remove the leaf points adhered to the stem.

[0108] Step 3, segment the leaves: when segmenting the leaves, first, use the region growing algorithm based on the normal vector to preliminarily segment different leaves, and then use the method of iteratively updating the cluster center and surface fitting to segment the overlapping leaves to obtain the constituent points of each leaf, which specifically includes:

[0109] Step 3.1, first, use the region growing algorithm based on the normal vector to preliminarily segment different leaves by taking the angle threshold as 4.85 and the curvature threshold as 0.9;

[0110] Step 3.2, distinguish the overlapping leaves by clustering;

[0111] Since part of the leaves are overlapped together in the result of the coarse segmentation and are classified as the same leaf, some operations are needed to distinguish the overlapping leaves;

[0112] When distinguishing the overlapping leaves, use the method of iteratively updating the cluster center and surface fitting to segment the leaves to achieve the purpose of distinguishing the overlapping leaves, which specifically includes:

[0113] Step 3.2.1, select the points close to the stem DBSCAN density clustering to obtain clusters and the initial cluster center of each cluster; let each cluster in the clusters be , and the cluster center of the cluster be , wherein the number of clusters is the number of overlapping leaves; if do not segment the leaves, if , sequentially execute steps 2-5 to segment the leaves; ​

[0114] Step 3.2.2, for each point in the clusters , , calculate the nearest cluster center of the point in the axis direction ; , , re-divide the point into the cluster it belongs to ; ; after all points are re-divided, update the cluster center ;

[0115] Step 3.2.3, for each point in the clusters , , calculate the nearest cluster center of the point in the entire coordinate space ; , re-divide the point into the cluster it belongs to ; ; after all points are re-divided, update the cluster center ;

[0116] Step 3.2.4, set , for the cluster , use the DBSCAN density clustering algorithm to obtain sub-clusters , arrange all sub-clusters in in descending order of the number of points, update to , add the sub-clusters to ; update to , repeat step 4 until ;

[0117] Step 3.2.5, re-calculate the cluster to which each point belongs for all points in each cluster, the specific steps are as follows: for each cluster , fit a cubic surface ; , calculate , then the cluster to which the point belongs is ; after completing the re-calculation of the cluster of all points, the points in each cluster form a leaf, that is, it can be clear that each leaf includes points or that each point belongs to a leaf; the view after leaf segmentation is shown in Figure 3 ; the point number statistics after segmentation is shown in Table 1.

[0118] ​​​Due to the clustering result obtained in step 3.2.4, the segmentation result at the intersection of two blades is still poor. Therefore, further judgment needs to be made on each point to determine which surface it belongs to, so as to obtain the final clustering result and determine the points included in each blade or the blade to which each point belongs.

[0119] Table 1. Point number statistics of all leaf segmentation results (arranged in ascending order of leaf height)

[0120]

[0121] Step 4, detection and repair of leaf holes: first, use radius filtering to preliminarily screen the leaf to screen out points with few neighboring points; then determine whether the screened points with few neighboring points are boundary points or internal points to screen out all boundary points of the leaf; then determine whether the screened boundary points are peripheral boundary points or hole boundary points to screen out all hole boundary points, thereby completing the detection of holes; then, complete the repair of holes by fitting a cubic surface B for interpolation, and obtain the final tobacco point cloud data;

[0122] Step 4.1, hole detection:

[0123] Step 4.1.1, use radius filtering to preliminarily screen the leaf to screen out points with few neighboring points, as shown in Figure 4 (a), the red points are the screened points with few neighboring points, and the yellow points are other points on the leaf;

[0124] Step 4.1.2, determine whether all the screened points with few neighboring points are boundary points or internal points to screen out all boundary points; the screening result is shown in Figure 4 (b);

[0125] The boundary point and internal point determination method comprises:

[0126] For any point in the screened point set with few neighboring points , use the RANSAC algorithm to fit the point and its neighboring points into a plane, then project the point and its neighboring points onto the plane, connect the point and the projections of all neighboring points on the plane to form vectors, arrange them in clockwise order, and form an angle between every two adjacent vectors, a total of angles; calculate the maximum value of the angles, if is greater than a given threshold , it is determined that the point is a boundary point, otherwise, it is determined as a point is an interior point;

[0127] Step 4.1.3, determining all boundary points, confirming whether the screened boundary points are peripheral boundary points or hole boundary points, screening all hole boundary points, and the screening result is as shown in Figure 4 (c);

[0128] The method for determining whether the screened boundary points are peripheral boundary points or hole boundary points comprises:

[0129] Projecting all points on the leaf onto the plane as shown in , connecting the point Figure 5 in the screened boundary points with other neighboring points on the leaf, calculating the included angle between adjacent vectors after clockwise arrangement of all vectors, and finding the maximum value of the included angle ; if the maximum value of the included angle is greater than a given threshold value , it is considered that the point is a peripheral boundary point, otherwise, it is considered that the point is a hole boundary point. Step 4.1.4, counting and removing those relatively scattered points (scattered points far away from the hole boundary), as shown in (d);

[0130] Figure 4 Step 4.1.5, performing DBSCAN density clustering on the hole boundary points after removing the scattered points, and distinguishing different holes, wherein each class obtained by clustering is a hole;

[0131] Step 4.2, hole repair: using a hole repair method to repair all holes to obtain final tobacco point cloud data;

[0132] The hole repair method comprises:

[0133] First, find the respective neighbor points for each boundary point of the hole; then use a non-linear least square method to fit a cubic surface

[0134] (or a B-spline surface) for all hole boundary points and their respective neighbor points, and then traverse to find the coordinates of the interior points of the hole, and use the fitted cubic surface to interpolate at the found interior points of the hole to complete the hole repair. The cubic surface is:

[0135]

[0136] ​​​Step 5, phenotype extraction: using the final tobacco point cloud data obtained after hole repair, phenotype extraction is performed to obtain the phenotype information of the plant. The plant phenotype data extracted in this embodiment is shown in Table 2; the phenotype extraction includes plant height extraction, leaf number extraction, leaf area extraction, leaf angle extraction, etc.; the phenotype includes plant height, leaf number, leaf area, leaf angle, etc.

[0137] A, plant height extraction

[0138] After the stem is extracted (after all the points constituting the stem are determined), the stem is evenly cut into segments in the axial direction, and for each cut stem segment, a cylindrical model is fitted respectively, and the height of the cylinder in each cylindrical model is calculated ; finally, the sum of the heights of all the cylinders is calculated Figure 6 as the length of the stem, i.e. the plant height. RANSAC is the abbreviation of RANdom SAmple Consensus, which can estimate the parameters of a mathematical model from a set of observation data containing "outliers" through iterative method;

[0139] The above method of calculating the plant height has higher accuracy in detecting curved or inclined stems compared to the method of directly calculating the difference between the maximum and minimum values in the axial direction as the plant height used in the traditional technology.

[0140] The above method of calculating the plant height has higher accuracy in detecting curved or inclined stems compared to the method of directly calculating the difference between the maximum and minimum values in the axial direction as the plant height used in the traditional technology. B, leaf number extraction

[0141] Since the segmentation of the leaves has been performed in step 3 and the constituent points of the leaves are clear, the number of leaves is counted here, i.e. the number of classes contained in the final clustering result obtained in step 3, i.e. the number of leaves.

[0142] C, leaf area extraction: the leaf area is calculated by point cloud surface reconstruction (i.e. point cloud gridding), which specifically includes:

[0143] For each leaf, first use the Poisson surface reconstruction algorithm to generate a triangular mesh of the leaf surface, calculate the area of the triangle using the Heron formula, and the sum of the areas of all triangles is taken as the surface area of the leaf.

[0144] Poisson surface reconstruction is a triangular mesh reconstruction algorithm based on implicit function, which obtains the approximate surface we want by optimizing the interpolation of point cloud;

[0145] Poisson surface reconstruction is a triangular mesh reconstruction algorithm based on implicit function, which obtains the approximate surface we want by optimizing the interpolation of point cloud;

[0146] Poisson surface reconstruction is a triangular mesh reconstruction algorithm based on implicit function, which obtains the approximate surface we want by optimizing the interpolation of point cloud;Heron's formula is also known as Heron formula, Helen formula, Heron formula, Helen Qin Jiushao formula. It is a formula for calculating the area of a triangle directly using the lengths of its three sides. Specifically, ; wherein, is the area of the triangle, is the semi-perimeter of the triangle, are the three sides of the triangle respectively.

[0147] C, leaf angle extraction: first extract the skeleton points of the leaf, then select K-1 skeleton points close to the stem, and set the skeleton point closest to the stem as point , and the remaining K-1 skeleton points form straight lines with point respectively, calculate the angles between the K-1 straight lines and the stem; then detect and remove outliers in the angles between the K-1 straight lines and the stem using the box plot method, and calculate the average value of all angles after removing outliers, which is the leaf angle.

[0148] Wherein, the method for extracting the skeleton points of the leaf comprises:

[0149] Step a, using PCA algorithm to calculate the first principal component direction of the leaf ;

[0150] Step b, evenly divide the leaf into several segments in the first principal component direction ;

[0151] Step c, perform DBSCAN density clustering on each segment to divide the segment into several classes;

[0152] Step d, calculate the cluster center of each class, and take the obtained cluster center as the skeleton point;

[0153] In this embodiment, the extraction results of the phenotypic traits of the tobacco plant.

[0154] Table 2. Extraction results of phenotypic traits of tobacco plants

[0155]

[0156] The above-described embodiments are only preferred embodiments of the present application, and are not an exhaustive list of the feasible implementations of the present application. Any obvious modifications made by those skilled in the art without departing from the principles and spirit of the present application should be considered to be included in the protection scope of the claims of the present application.​​​​

Claims

1. A method for extracting tobacco phenotypic traits based on three-dimensional point cloud, characterized in that, The method comprises the following steps: Step one, data preprocessing: sequentially performing voxel downsampling processing, denoising processing and coordinate correction on the original point cloud image data to obtain corrected tobacco point cloud data; In step one, the original point cloud image data is obtained by scanning the tobacco plant using a 3D scanning device; In step one, the original point cloud image data is sequentially subjected to voxel downsampling processing, denoising processing and coordinate correction to obtain corrected tobacco point cloud data, which specifically comprises: First, the number of points of the original point cloud image data is reduced by using voxel downsampling, then the abnormal points and outliers in the point cloud image data obtained by downsampling are filtered by using radius filtering and statistical filtering, then the normal vector is estimated, and finally the coordinate is corrected to obtain the corrected tobacco point cloud data; The normal vector estimation is specifically performed as follows: A point is randomly selected from the downsampled point cloud image data. Find the point using a kd-tree of Find the nearest neighbor points, and then find the... Nearest neighbor points and points The principal component analysis algorithm is used to obtain the result. Nearest neighbor points and points The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by itself is used as the eigenvector of that point. Estimation of the normal vector; The coordinate correction includes two steps: First, the stalk orientation is estimated: based on minimizing the magnitude of the sum of the outer products of the normal vectors of all points in the point cloud image data of the entire tobacco plant, the initial estimated stalk orientation is obtained, denoted as... Then, the initial estimated direction of the stem is determined. Perform iterative corrections to determine the final estimated direction of the stem, denoted as . Finally, rotate the z-axis of the coordinate system to the final estimated direction of the stem. The position is used to project all points of the downsampled point cloud image data onto the rotated image. On the plane, calculate the coordinates of the centroid of all projected points. Then, translate the origin. The coordinates are corrected at the centroid point to obtain the corrected tobacco point cloud data. When the coordinate correction is performed, the initial estimation direction of the stem is obtained based on minimization of a modulus of a cross product sum of normal vectors of all points of the point cloud of the whole tobacco in the point cloud image data obtained by the downsampling and specifically includes the following steps: Let the number of points in the down-sampled point cloud image data be the normal vector of each point is ； Let the initial stem estimate direction be Solve the following optimization problem: ； Recall The target problem is transformed into: ； ； Obtaining a stem initial estimated direction ; performing a coordinate correction to the initial estimate of the stem direction performing an iterative correction to determine a final estimate of the stem direction comprising the following steps: Step (1), placing ; Step (2), with as the generatrix, as the radius, extracts a cylinder ; Step (3), extracting the first principal component direction of the cylinder using the PCA algorithm ;​ Step (4), if , end the loop; otherwise, set and reduce the radius to , repeat Step (2)-Step (4) until the loop ends, i.e. the final estimated direction of the stem is determined ; Step two, plant organ segmentation: first, the stem is segmented according to the stem characteristics, and then the leaf is segmented to obtain the composition points of the stem and the leaf; In the stem segmentation, the stem is initially screened by using a step-by-step screening method, and then the leaf points adhered to the stem are removed to obtain the composition points of the stem; In the leaf segmentation, different leaves are initially segmented based on the region growing algorithm of the normal vector, and then the overlapping leaves are segmented by using the iterative updating of the clustering center and the curve fitting, and finally the composition points of each leaf are obtained; Step three, detection and repair of leaf holes First, radius filtering is used to preliminarily screen the leaf to screen out points with few neighboring points; then it is judged whether the screened points with few neighboring points are boundary points or internal points to screen out all boundary points of the leaf; then it is judged whether the screened boundary points are peripheral boundary points or hole boundary points to screen out all hole boundary points, thereby completing the detection of holes; then, the leaf is interpolated and fitted by using a cubic surface equation to complete the hole repair and obtain the final tobacco point cloud data; Step four, phenotype data extraction: using the final tobacco point cloud data obtained after hole repair to extract the phenotype to obtain the phenotype information of the plant, the phenotype including plant height, leaf number, leaf area and leaf angle; the phenotype extraction including plant height extraction, leaf number extraction, leaf area extraction and leaf angle extraction.

2. The tobacco phenotype trait extraction method based on three-dimensional point cloud according to claim 1, wherein in step two, in the stem segmentation, the stem is initially screened by using a step-by-step screening method, and then the leaf points adhered to the stem are removed to obtain the composition points of the stem, which specifically comprises the following steps: Step 4), the point cloud data screened in step 3) is clustered by using a density clustering algorithm based on Euclidean distance to screen out the class with the most points; Step 1), based on the corrected tobacco point cloud data obtained after coordinate correction, using Straight-through filtering in the direction, screening out points near the origin; Step 2), based on the re-estimated normal vectors, denoted as ; normal vectors are unitized, and data points with are filtered out; Step 3) Use statistical filtering to filter out the selected items. The outliers and anomalies in the data points are identified to complete the initial screening and obtain the initial screening point cloud data. Step 5), the class with the most points screened in step 4) is clustered by using a region growing algorithm based on the normal vector to continue screening out the class with the most points, and all the points in the class with the most points screened this time are the composition points of the final stem.

3. The tobacco phenotype trait extraction method based on three-dimensional point cloud according to claim 1, wherein in step two, the overlapping leaves are segmented by using the iterative updating of the clustering center and the curve fitting to finally obtain the composition points of each leaf, and the specific steps are as follows: ​ ​ Step 1, selecting a point close to the stem DBSCAN density clustering, obtaining clusters and the initial cluster center of each cluster; record each cluster in the cluster as , wherein ; the cluster center of the cluster is , wherein the number of clusters , that is, the number of overlapping leaves; if , do not perform leaf segmentation, if , sequentially perform steps 2-5 to perform leaf segmentation; Step 2, regarding the above Each point in each cluster ,exist Calculate the relationship with the point along the axial direction. Nearest cluster center , will point Reassigned to Cluster to which it belongs After all points have been re-divided, update the cluster centers. ; Step 3, regarding the above Each point in each cluster Calculate the relationship between the point and the coordinate space. Recent cluster center , will the point Reassigned to Cluster to which it belongs After all points have been re-divided, update the cluster centers. ; Step 4, set For clustering , use DBSCAN density clustering algorithm to get sub-clusters , where, ; arrange all sub-clusters in by point number in descending order, update to , add sub-cluster to ; update to , repeat step 4 until . Step 5, recompute the cluster that each point belongs to for all points in each cluster, the specific steps are as follows: for each cluster , fit a cubic surface ; take a point , calculate , then the cluster that the point belongs to is ; after the cluster re-computation of all points is completed, the points in each cluster form a leaf, that is, the points included in each leaf or the leaf to which each point belongs can be determined.

4. The method of claim 1, wherein the method of extracting tobacco phenotypic traits based on a three-dimensional point cloud is characterized in that, The method of determining whether the screened point with few neighboring points is a boundary point or an internal point in step three comprises the following steps: For any point in the point set with less neighbors , use RANSAC algorithm to fit the point and its neighbors into a plane, then project the point and its neighbors onto the plane, connect the point and the projection of all neighbors on the plane to form vectors, arrange them in clockwise order, each two adjacent vectors form an angle, there are angles in total; calculate the maximum value of these angles , if is greater than a given threshold , then determine that the point is a boundary point, otherwise, determine that the point is an interior point; The method of determining whether the screened boundary point is a peripheral boundary point or a hole boundary point in step three comprises the following steps: Project all points on the blade onto On the plane, the points among the selected boundary points Connect the vectors to their nearest neighbors on the leaf, calculate the angle between adjacent vectors after arranging all vectors clockwise, and find the maximum value of the angle. ;like Greater than a given threshold Then the point is considered It is an outer boundary point; otherwise, it is considered a point. It is the boundary point of the hole; In step three, the cubic surface equation obtained by fitting is used Interpolation is performed to complete the hole repair and obtain the final tobacco point cloud data, specifically including: find the respective nearest neighbor point for each boundary point of the hole, then fit a cubic surface to all the hole boundary points and their respective nearest neighbor points using non-linear least square method then traverse to find the coordinates of the hole interior points at which the fitted cubic surface is used for interpolation to complete the hole repair then traverse to find the coordinates of the hole interior points at which the fitted cubic surface is used for interpolation to complete the hole repair​​ 5. The method of claim 1, wherein the method of extracting tobacco phenotypic traits based on a three-dimensional point cloud is characterized in that, Step four, the method for extracting plant height, includes: firstly, in The stems are cut evenly along the axial direction. The stalk is then segmented, and the RANSAC algorithm is used to fit a cylindrical model for each segment, and the height of the cylinder in each cylindrical model is calculated. ,in, Finally, calculate the sum of the heights of all cylinders, which will be the length of the stem, i.e., the plant height. The method of extracting the number of leaves comprises directly counting the number of leaves in the final tobacco point cloud data obtained after hole repair; The method of extracting the leaf area comprises the following steps: first, using a Poisson surface reconstruction algorithm to process the leaf to generate a triangular mesh of the leaf surface, then calculating the area of each triangle using the Heron formula, and finally summing the areas of all the triangles to obtain the surface area of the leaf, i.e., the leaf area; The method for extracting leaf angles includes: first, extracting the skeletal points of the leaf, and then selecting points close to the stem. Let the skeleton point be the one closest to the stem. , will point The rest -1 skeleton point, each connected to point Connect the lines to form a straight line, calculate. -1. The angle between the straight line and the stem; then use the box plot method to remove... -1 outliers in the angle between the line and the stem, calculate the average of all angles after removing outliers, which is the leaf angle.

6. The method for extracting tobacco phenotypic traits based on three-dimensional point cloud according to claim 5, characterized in that, The method of extracting the skeleton points of the leaf in step four comprises the following steps: Step a, calculating the first principal component direction of the leaf using the PCA algorithm ; Step b, in the first principal direction The leaves are evenly divided into several segments; Step c: performing DBSCAN density clustering on each segment to divide the segment into several classes; Step d: calculating the cluster center of each class, and taking the cluster center as a skeleton point.

Images