Wheat population multi-dimensional feature extraction method based on three-dimensional point cloud and image processing

Abstract

The invention discloses a wheat population multi-dimensional feature extraction method based on three-dimensional point cloud and image processing, and relates to the technical field of image feature extraction. The method comprises the following steps: acquiring multi-view video data of a target wheat population and performing frame processing to generate three-dimensional point cloud data of the wheat population; the three-dimensional point cloud data is preprocessed and then spatially layered, and the average height feature, the canopy coverage feature, the tillering layer volume ratio feature and the tillering density feature of the wheat population are calculated; the method has the advantages that multilevel features such as the form, the structure and the density of wheat are effectively extracted, the comprehensiveness and the data robustness of feature characterization are remarkably improved, and a high-precision and low-cost monitoring scheme is provided for wheat growth state evaluation, precise fertilization and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for extracting multi-dimensional features of wheat population based on three-dimensional point cloud and image processing. Background Art

[0002] In recent years, the rapid development of remote sensing technology, computer vision, and three-dimensional point cloud technology has promoted the progress of non-contact agricultural monitoring technology. Among them, three-dimensional point cloud technology has become an important technical means for wheat growth monitoring because it can completely obtain the three-dimensional spatial information of wheat plants, including morphological characteristics (such as plant height, biomass) and structural characteristics (such as canopy distribution, leaf inclination). Research shows that wheat under different nutritional states will present different canopy structural characteristics. For example, the leaf erectness of nitrogen-deficient plants increases, and the canopy openness of phosphorus-deficient plants decreases, etc. This provides a theoretical basis for wheat nutrition diagnosis based on three-dimensional features.

[0003] However, the current wheat monitoring method based on three-dimensional point cloud still faces three challenges: First, existing technologies mostly focus on the extraction of morphological characteristics and fail to effectively fuse structural characteristic parameters such as the spatial distribution of stems and leaves and canopy light transmittance, resulting in a single dimension for growth state assessment; Second, the nutrition diagnosis model generally relies on a single key characteristic index and fails to fully explore the co-representation ability of multi-dimensional features in three-dimensional point cloud data; Third, existing methods rely too much on the quality of point cloud data, and lack robust feature extraction methods for point cloud data obtained under complex field lighting conditions and low-precision scanning equipment, which severely restricts the field application and promotion of the technology. Summary of the Invention

[0004] In order to solve the technical problems of single feature extraction dimension, dependence on high-precision point cloud data, and neglect of canopy structural characteristics in existing wheat monitoring methods, the present invention provides a multi-level feature extraction method for wheat crops based on three-dimensional point cloud and image processing, which realizes the extraction of features such as plant height, canopy coverage, the proportion of tillering layer volume, and density by fusing point cloud downsampling, spatial stratification, and multi-angle projection techniques.

[0005] In order to achieve the above invention purpose, the present invention provides a method for extracting multi-dimensional features of wheat population based on three-dimensional point cloud and image processing. The extraction method includes the following steps: Step S1: Obtain multi-view video data of the target wheat population; Step S2: Perform frame processing on the multi-view video data to generate three-dimensional point cloud data of the wheat population through NeRF technology; Step S3: Preprocess the three-dimensional point cloud data by methods of voxel downsampling, ground point segmentation, and outlier removal; Step S4: Extract the average height feature of the wheat population based on the preprocessed three-dimensional point cloud data; Step S5: Perform spatial stratification on the preprocessed three-dimensional point cloud data to obtain tillering layer point cloud data; Step S6: Based on the tillering layer point cloud data, calculate the canopy coverage feature through multi-angle projection combined with the color mask method; Step S7: Use the Quickhull algorithm to construct three-dimensional convex hull models of the whole wheat and the tillering layer, and use the tetrahedron decomposition method to calculate the volume ratio feature of the tillering layer; Step S8: Calculate the density feature at the tillering position based on the analysis of the tillering layer baseline.

[0006] Among them, in Step S1, obtaining the multi-view video data of the target wheat population specifically means using a portable device to shoot three videos around the target wheat population. The first video focuses on the bottom area of the target wheat population, the second video focuses on the middle area of the target wheat population, and the third video focuses on the top area of the population at a 45° elevation angle.

[0007] Step S2 specifically includes: extracting key frames from each video sequence, removing blurred frames, screening a set number of clear images from each spatial partition, and fusing multi-view image frames through the NeRF technology to generate three-dimensional point cloud data of the wheat population.

[0008] The preprocessing in Step S3 specifically includes: Step S301: Calculate the boundary range of the three-dimensional point cloud data of the wheat population, divide the point cloud data into uniform cubic grids with a fixed side length, calculate the centroid of the points in each voxel as the representative point, and generate downsampled point cloud data; Step S302: Use the RANSAC algorithm in the three-dimensional point cloud data to randomly select three sample points to construct a plane model, calculate the Euclidean distance from the remaining points in the downsampled point cloud data to the plane model, and screen the points with a distance less than the set threshold as the ground point set and remove them from the point cloud; Step S303: Perform statistical filtering on the three-dimensional point cloud after removing the ground, calculate the average Euclidean distance and standard deviation within the neighborhood of each point, mark and remove outliers through the set standard deviation multiple threshold, and output the preprocessed point cloud data.

[0009] In Step S4, extracting the average height feature of the wheat population specifically includes: Step S401: Use the rotation matrix to perform rotation and translation operations on the preprocessed point cloud data to align the wheat boundary with the coordinate axes, and define the z-axis coordinate as the point cloud height; Step S402: Set a height ratio threshold, and use the product of the z value of the highest point of the point cloud and the height ratio threshold as the threshold height; Step S403: Screen the point cloud data with the z - coordinate greater than the threshold height, remove the point cloud with the top proportion being the height proportion threshold, and output the effective - height point cloud; Step S403: Calculate the average height feature of the wheat population based on the effective - height point cloud.

[0010] In step S5, the pre - processed three - dimensional point cloud data is spatially stratified to obtain the tillering - layer point cloud data. Specifically: the pre - processed point cloud data is divided from top to bottom by vertical height into the ear layer, the straw - leaf layer, and the tillering layer, and the point cloud data corresponding to each stratified level is output; among them, the height proportion of each layer is determined by the preset stratification proportion parameter, and can be: the upper ear layer accounts for 30%, the middle straw - leaf layer accounts for 45%, and the lower tillering layer accounts for 25%.

[0011] The method for calculating the canopy coverage feature in step S6 is specifically as follows: Step S601: Project the overall point cloud data and the tillering - layer point cloud data onto a mapping plane at a specified angle to generate a projected point cloud; Step S602: Draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud into the HSV color space, and calculate the number of non - zero pixels in the mask as the canopy coverage area; Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage feature.

[0012] The method for calculating the tillering - layer volume proportion feature in step S7 is specifically as follows: Step S701: Based on the stratified tillering - layer point cloud data and the pre - processed overall point cloud data, use the Quickhull algorithm to construct three - dimensional convex hull models respectively. Specifically: select the point with the minimum x - coordinate P min and the point with the maximum x - coordinate P max in the overall point cloud to construct an initial segmentation line; calculate the distance from each point to the initial segmentation line, and screen the point with the maximum distance P for as the new convex - hull vertex; recursively segment the point set and repeat the vertex screening until the construction of the overall convex hull and the tillering - layer convex hull is completed; Step S702: Calculate the volume of each convex hull and the total volume through the tetrahedron decomposition method. Specifically, select a base point D from the vertices of the tillering - layer convex hull; traverse each triangular face of the tillering - layer convex hull ( A , B , C ), and form a tetrahedron with the base point D ( A , B , C, D ) Calculate the volume of each tetrahedron; accumulate the volumes of all tetrahedrons to obtain the total volume of the tillering layer convex hull; calculate the total volume of the overall convex hull in the same way; Step S703: Take the ratio of the total volume of the tillering layer convex hull to the total volume of the overall convex hull as the tillering layer volume ratio feature.

[0013] The method for calculating the tillering density feature in step S8 is specifically as follows: Step S801: Generate a front view projection scatter plot based on the point cloud data of the tillering layer after stratification; Step S802: Start scanning row by row from the lower end of the scatter plot, and locate the first row containing the preset color value as the root baseline , set an offset and adjust the baseline to the middle area of the tillering layer according to the offset to obtain the position of the tillering layer baseline. The calculation formula of the tillering layer baseline is:

[0014] where is the height of the scatter plot, usually set to 0.15; Step S803: Detect all continuous pixel segments on the tillering layer baseline , record the starting column and the ending column of each segment, and calculate the single-segment coverage length, the total coverage length, and the total length of the tillering layer baseline; Step S804: Statistically calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the tillering density feature.

[0015] The beneficial effects of the present invention are as follows: The present invention reduces the interference of point cloud noise through voxel downsampling and statistical filtering, combines hierarchical mapping and convex hull volume decomposition technology, effectively extracts multi-level features such as the morphology, structure, and density of wheat, significantly improves the comprehensiveness of feature representation and the data robustness, and provides a high-precision and low-cost monitoring solution for wheat growth status evaluation and precise fertilization, etc. Description of the Drawings

[0016] Figure 1 is the flowchart of Embodiment 2 of the present invention.

[0017] Figure 2 is the schematic diagram of data acquisition in Embodiment 3 of the present invention.

[0018] Figure 3 is the schematic diagram of the three-dimensional point cloud preprocessing result in Embodiment 3 of the present invention.

[0019] Figure 4It is a schematic diagram of average height extraction in Example 4 of the present invention.

[0020] Figure 5 It is a schematic diagram of the overall canopy and tillering layer of wheat in Example 5 of the present invention.

[0021] Figure 6 It is a schematic diagram of the coverage of the wheat point cloud at 30 degrees in Example 5 of the present invention.

[0022] Figure 7 It is a schematic diagram of the bottom density baseline in Example 6 of the present invention.

[0023] Figure 8 It is a schematic diagram of the convex hull of the tillering layer in Example 7 of the present invention.

[0024] Figure 9 It is a schematic diagram of the convex hull of the whole wheat canopy in Example 7 of the present invention. DETAILED DESCRIPTION

[0025] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation method.

[0026] Embodiment 1: The embodiment of the present invention provides a method for extracting multi-dimensional features of wheat populations based on three-dimensional point cloud and image processing, and the extraction method comprises the following steps: Step S1: Obtain multi-view video data of the target wheat group; specifically, use a portable device to shoot three videos around the target wheat group, the first video focuses on the bottom area of ​​the target wheat group, the second video focuses on the middle area of ​​the target wheat group, and the third video focuses on the top area of ​​the group at a 45° elevation angle.

[0027] Step S2: Frame processing is performed on the multi-view video data, and three-dimensional point cloud data of the wheat group is generated through NeRF technology; specifically: key frames are extracted from each video sequence, and blurred frames are removed, a set number of clear images are screened from each spatial partition, and multi-view image frames are fused through NeRF technology to generate three-dimensional point cloud data of the wheat group.

[0028] Step S3: preprocessing the three-dimensional point cloud data by voxel downsampling, ground point segmentation and outlier removal methods; specifically comprising the following steps: Step S301: Calculate the boundary range of the three-dimensional point cloud data of the wheat population, divide the point cloud data into uniform cubic grids with fixed side lengths, calculate the centroid of the points in each voxel as a representative point, and generate downsampled point cloud data; Step S302: Use the RANSAC algorithm to randomly select three sample points in the 3D point cloud data to construct a plane model, calculate the Euclidean distance from the remaining points in the downsampled point cloud data to the plane model, and filter out the points with a distance less than the set threshold as the ground point set and remove them from the point cloud; Step S303: Perform statistical filtering on the 3D point cloud after removing the ground, calculate the average Euclidean distance and standard deviation within the neighborhood of each point, mark and remove the outlier points by setting the standard deviation multiple threshold, and output the preprocessed point cloud data.

[0029] Step S4: Based on the preprocessed 3D point cloud data, extract the average height feature of the wheat population; specifically including the following steps: Step S401: Use the rotation matrix to perform rotation and translation operations on the preprocessed point cloud data to align the wheat boundary with the coordinate axes, and define the z-axis coordinate as the point cloud height; Step S402: Set the height ratio threshold, and use the product of the z value of the highest point in the point cloud and the height ratio threshold as the threshold height; Step S403: Filter out the point cloud data with a z coordinate greater than the threshold height, and remove the point cloud with the top proportion of the height ratio threshold, and output the effective height point cloud; Step S403: Calculate the average height feature of the wheat population based on the effective height point cloud.

[0030] Step S5: Perform spatial stratification on the preprocessed 3D point cloud data to obtain the tillering layer point cloud data; specifically: divide the preprocessed point cloud data into the ear layer, the straw and leaf layer, and the tillering layer from top to bottom according to the vertical height, and output the point cloud data corresponding to each level after stratification. Among them, the height proportion of each layer is determined by the preset stratification ratio parameter, and can be: the upper ear layer accounts for 30%, the middle straw and leaf layer accounts for 45%, and the lower tillering layer accounts for 25%.

[0031] Step S6: Based on the tillering layer point cloud data, calculate the canopy coverage feature through multi-angle projection and combined with the color mask method; specifically: Step S601: Project the overall point cloud data and the tillering layer point cloud data onto the mapping plane at a specified angle to generate the projected point cloud; Step S602: Draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud to the HSV color space, and calculate the number of non-zero pixels in the mask as the canopy coverage area; Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage feature.

[0032] Step S7: Use the Quickhull algorithm to construct a three-dimensional convex hull model of the whole wheat and the tillering layer, and use the tetrahedron decomposition method to calculate the volume ratio feature of the tillering layer; specifically: Step S701: Based on the point cloud data of the tillering layer after stratification and the preprocessed point cloud data of the whole, use the Quickhull algorithm to construct three-dimensional convex hull models respectively; specifically: Select the point with the smallest x coordinate in the point cloud of the whole P min and the largest point P max , construct an initial segmentation line; calculate the distance from each point to the initial segmentation line, and screen the point with the largest distance P for as a new convex hull vertex; recursively divide the point set and repeat vertex screening until the construction of the whole convex hull and the tillering layer convex hull is completed; Step S702: Calculate the volume of each convex hull and the total volume through the tetrahedron decomposition method; specifically, select a base point from the vertices of the tillering layer convex hull D ; traverse each triangular patch of the tillering layer convex hull ( A , B , C ), and form a tetrahedron with the base point D ( A , B , C , D ), calculate the volume of each tetrahedron; accumulate the volumes of all tetrahedrons to obtain the total volume of the tillering layer convex hull; calculate the total volume of the whole convex hull in the same way; Step S703: Take the ratio of the total volume of the tillering layer convex hull to the total volume of the whole convex hull as the volume ratio feature of the tillering layer.

[0033] Step S8: Based on the analysis of the tillering layer baseline, calculate the density feature at the tillering position; specifically: Step S801: Generate a front projection scatter plot based on the point cloud data of the tillering layer after stratification; Step S802: Start scanning row by row from the bottom of the scatter plot, and locate the first row containing the preset color value as the root baseline , set an offset and adjust the baseline to the middle area of the tillering layer according to the offset to obtain the position of the tillering layer baseline. The calculation formula of the tillering layer baseline is:

[0034] where, is the height of the scatter plot, is usually set to 0.15; Step S803: At the tillering layer baseline Detect all continuous pixel segments above, and record the starting column of each segment and the ending column , and calculate the single-segment coverage length, total coverage length, and total length of the tillering layer baseline; Step S804: Statistically calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the tillering density characteristics.

[0035] Example 2: Refer to Figure 1 , this embodiment of the present invention provides a method for extracting wheat population characteristics based on three-dimensional point cloud and image processing, which has realized extracting a variety of effective and high-value feature information on the basis of low-precision wheat three-dimensional point cloud, and provides a basis for the real-time monitoring and nutritional diagnosis of the growth state of field wheat. The extraction method specifically includes the following steps: I. Data collection. At present, the construction of three-dimensional point cloud often uses high-end equipment. In this invention, a portable video recording device (such as a mobile phone, camera) is used to take pictures of field wheat to collect wheat data. The specific method is: take three circles of pictures around the wheat population. The first circle's perspective focuses on the bottom of the wheat population, the second circle focuses on the middle of the wheat population, and the third circle focuses on the upper part of the wheat population, forming an angle of about 45° with the ground (which is convenient for clearly photographing the top and some internal structures of the wheat population), to obtain the original video data set of the stratified wheat population.

[0036] II. Data processing to obtain three-dimensional point cloud. Perform frame extraction on the video data set obtained in step I, remove blurred and cluttered target frames, extract several clear images from each layer, and then use the NeRF technology to convert the wheat images into three-dimensional point cloud data.

[0037] III. Data preprocessing. The three-dimensional point cloud data of the wheat population obtained in step II has noise points and also contains the background information of the wheat population. Noise points and background information are one of the reasons for the difficulty in extracting features from low-precision three-dimensional point cloud. The data preprocessing method in this embodiment is specifically as follows: Step 3.1: Remove the outlier points on the periphery of the wheat three-dimensional point cloud. Use Voxel Grid Downsampling to divide the wheat three-dimensional space into voxels of a fixed size, and retain a small number of representative points (such as the nearest point, centroid point, or maximum distance point) in each voxel, so as to reduce the density of the original wheat three-dimensional point cloud and remove noise points, which can effectively retain the overall geometric structure of the point cloud and significantly reduce the computational complexity. Then use RANSAC to detect and segment to remove the ground points, and then use the statistical filtering algorithm to remove the remaining sparsely distributed outlier points.

[0038] First, divide the preprocessed three-dimensional space into small cubes (voxels) of a fixed size. The side length of each voxel is preset as Voxel_size, which can be 0.1 m. Accurately calculate the boundary range of the point cloud. This boundary range defines the distribution quadrants of the point cloud data in the three-dimensional space. According to the calculated boundary range of the point cloud, divide the three-dimensional space into several small cubes, which are the voxels. Assign the points to the voxels. For each point Calculate the voxel index where it is located:

[0039] Among them, Represents rounding down, Is the point 's coordinates, 、 、 Represent the minimum boundary values of the point cloud data on the three axes, v Is the side length of the voxel.

[0040] Secondly, for each divided voxel, use the centroid method to determine its representative point. Conduct a statistical analysis on the coordinates of all points within the voxel, and use the centroid method to calculate the geometric center of the points within the voxel :

[0041] Among them, n Is the total number of points within the voxel, i Represents the serial number index of the points within the voxel.

[0042] Finally, collect the representative points selected for all voxels according to the above method. These representative points together constitute the downsampled three-dimensional point cloud of wheat. After downsampling, the density of the original three-dimensional point cloud of wheat is reduced, and at the same time, the overall set structure of the point cloud is better retained.

[0043] Step 3.2. Since the ground points usually occupy most of the point cloud data and do not contribute to the feature extraction of the wheat population, use the RANSAC plane segmentation algorithm to detect the ground model. Specifically: First, find the optimal ground model through random sampling and iterative fitting. Randomly select the minimum sample set in the three-dimensional model point data of wheat. This sample set is the minimum number of samples required to initialize the model parameters. At least three points are required as sample points Ransac_n to fit a plane in the three-dimensional space. Use the selected minimum sample set to fit a plane model, which is the ground of the wheat crop. The mathematical expression of the plane model is usually:

[0044] Among them, A 、B , C , D are planar parameters, x , y , z are the coordinates of the wheat crop point cloud.

[0045] Secondly, substitute the remaining points into the fitted plane model, calculate the distance from the point to the plane. If it is less than a set threshold Distance_threshold, it is considered a ground point; otherwise, it is an outlier. The formula for calculating the distance from a point to the plane is as follows:

[0046] , , represents the coordinates of the point to be judged.

[0047] Thirdly, count the number of inliers. The model with more inliers is more likely to be the correct model. Repeat the above steps (random sampling, model fitting, inlier determination, model evaluation) to select the best model as the final model, which is the wheat crop ground. Record it in an array, which contains the indices or actual point data of the points identified as inliers in the plane ( x , y , z ). Then remove the points in the array from the overall wheat crop three-dimensional model point cloud data to complete the removal of the wheat crop ground.

[0048] Step 3.3: Since the fitted ground models may be different, some new noise points will be generated near the ground after removing the ground from the wheat crop point cloud. At this time, use Statistical Outlier Removal; specifically: First, select K nearest neighbors as the local neighborhood and perform calculations in the local neighborhood. For the point P in the point cloud, calculate its distance from each point in the local neighborhood:

[0049] where, is the th nearest neighbor of point i , is the Euclidean distance between point and point .

[0050] Calculate the average distance of the local neighborhood:

[0051] Calculate the standard deviation of the local neighborhood :

[0052] Secondly, set a parameter std_ratio to control the judgment criterion for noise points. Calculate the judgment threshold for noise points:

[0053] For point P, if its average distance μ exceeds the threshold, then mark it as a noise point:

[0054] Traverse all points in the point cloud, mark and remove all noise points, and finally obtain the denoised wheat point cloud data.

[0055] IV. Extract multi-angle and multi-level features from the preprocessed three-dimensional wheat point cloud. There are differences in the growth status of wheat under different nutrient element contents. This embodiment provides methods for extracting features such as canopy coverage, canopy volume ratio, tillering density, and average canopy height of the wheat population. The extracted features can be used to diagnose and analyze the growth status of wheat. Specifically: Step 4.1: Extract the canopy height feature. Rotate and translate the preprocessed three-dimensional wheat point cloud using a rotation matrix to align the boundary of the wheat field with the coordinate axes. The rotation matrix calculation formula (taking rotation around the X-axis as an example):

[0056] The x , y , z coordinates of the rotated three-dimensional wheat model can approximately start from 0, where the z coordinate can be regarded as the height of the point; set Height_ratio as the height threshold parameter, which can take 15%. The product of the highest point in the three-dimensional wheat point cloud and the height threshold parameter Height_ratio is used as the threshold height .

[0057]

[0058] Take out z points with coordinates greater than the threshold height. Considering the limitations of the three-dimensional point cloud, there will still be noise points at the top of the three-dimensional wheat point cloud. At this time, the data with a proportion of Height_ratio at the top can be further removed. Calculate the average value of the z coordinates of the remaining points, which is denoted as the average canopy height of the wheat .

[0059]

[0060] Among them, n is the number of remaining points, is the z - coordinate of the i th point among the remaining points, is the maximum Z - coordinate of the wheat point cloud after aligning the coordinate axes.

[0061] Step 4.2: Extract the canopy coverage of the wheat tillering layer and the whole at various angles. Given that the wheat field has a clear layered structure in the vertical direction, after multiple inspections and measurements of the sample field, the wheat field is divided into three layers: the upper ear layer, the middle straw - leaf layer, and the lower tillering layer, and their corresponding proportions are approximately 30%, 45%, and 25%. According to the calculated average height of the wheat canopy, the wheat three - dimensional point cloud is divided into three layers according to the height thresholds divided by the above - mentioned proportions. In the process of generating the wheat three - dimensional point cloud, the details in the middle part will fade and blur, and in the upper part, due to the inability to fix the ears during data collection, some noises that are difficult to remove will be generated, and the point cloud effect of the lower tillering layer is the best.

[0062] Step 4.2.1: First, map the wheat three - dimensional model onto a plane at a specified angle. Assume a projection is made from an angle θ (the angle formed with the z = 0 plane). Define the discovery vector n of the inclined plane:

[0063]

[0064] Among them, r is the radian value of the angle θ .

[0065] The general form of the projection plane is:

[0066] Among them, A =sin( r ), B =0, C =cos( r ), D is a constant.

[0067] Select the origin P =[0,0,0] to calculate D : D =-(n· P ) Therefore, the projection plane at this angle is:

[0068] Step 4.2.3. For any point P = [x, y, z] in the point cloud, calculate its perpendicular distance to the plane d :

[0069] Calculate the projection point:

[0070] After expansion:

[0071] Project all the point cloud data onto the plane to obtain θ the projected point cloud at the

[0072] Step 4.2.4. Draw a scatter plot of the wheat projected point cloud, set the scatter size S (to make the scatter points touch each other to form a surface), color (such as blue), and transparency (0.9). Convert the drawn RGB image to the HSV color space and define the color range lower_color = [100, 50, 50], upper_color = [130, 255, 255]. Create a binary mask:

[0073] Calculate the number of non-zero pixels in the mask:

[0074] Calculate the total area of the sample field Total_area (calculated through the bounding box) of the canopy coverage:

[0075] Similarly, this method can be used to extract the canopy coverage of the wheat three-dimensional point cloud at various angles and levels.

[0076] Step 4.3. Calculate the proportion of the canopy volume of the tillering layer in the overall canopy. Use the Quickhull algorithm to construct the convex hull of the wheat canopy point cloud. Select the point with the minimum x coordinate and the maximum point from the point set. These two points must be part of the convex hull. Pass through and to form a line: part is above the line and part is below the line.

[0077] Step 4.3.1. For each point P , calculate its distance d to the line:

[0078] Among them, Represents the cross product of vectors.

[0079] Step 4.3.2: Find the point farthest from the line , which is also part of the convex hull. Use , and to define a new triangular face and further divide the point set into two subsets. Recursively repeat the above steps for these two subsets until there are no points to process.

[0080] Step 4.3.3: Calculate the volume of the convex hull using the tetrahedron decomposition method. Select a fixed point D from all the vertices of the convex hull as the base point, and traverse the faces of the convex hull. Each face is denoted as ( A , B , C ). For each triangular face and the base point, form a tetrahedron. Traverse all the faces and add the volume of each tetrahedron to the total volume. Use the same method to calculate the canopy volume of the tillering layer and the ratio, which is the tillering layer volume ratio.

[0081]

[0082] Step 4.4: Calculate the density characteristics at the tillering layer. The density of wheat tillers is different under different nutrient element contents. A baseline needs to be defined at the wheat tillers. The main idea is to calculate the proportion of the wheat tillers to the baseline. Use the scatter plot drawing method provided in Step 4.2 to draw the front view scatter plot, convert the RGB color space to the HSV color space, and define the specified color mask mask (blue) low_color and upper_ color used when drawing the scatter plot. Check the mask row by row from one of the top and bottom segments of the image until the first row containing the specified color is found (i.e., the baseline first reaches the root position of the crop in the front view of the wheat crop). Displace towards the tillering part and set the offset Offset so that the baseline moves to the middle of the tillering and ensure it does not exceed the image boundary.

[0083] Step 4.4.1: Find all consecutive specific color segments on the baseline. For each consecutive region, calculate its distance Object_distance, and then calculate the distance between the leftmost and rightmost sides of the baseline Length .

[0084] Step 4.4.2: Calculate the ratio of the distance of each consecutive region to the length of the baseline using the values calculated above: .

[0085] The wheat population characteristics extracted by the method provided in this embodiment can be further processed and a diagnostic model can be constructed.

[0086] Example 3: Refer to Figure 2 and Figure 3 , the present invention provides a preprocessing method based on the three-dimensional point cloud of wheat population, which specifically includes the following steps: Step S1: Obtain the original wheat population dataset; in this embodiment, partitions of the same kind of wheat at different times and different nitrogen and phosphorus application levels are collected. The experiment set 4 nitrogen levels: N-0%, N-30%, N-60%, N-CK, and 4 phosphorus levels: P-0%, P-30%, P-60%, P-CK. Different blocks are isolated by 1m, and different fertilization blocks are isolated by 1m. There are a total of 24 partitions, and the area of a single partition . Wheat at the green-reverting stage, jointing stage, and heading stage is selected.

[0087] Use a portable video recording device (such as a mobile phone, camera) to take pictures of the wheat in the farmland to collect wheat data. The specific method is to take three circles around the wheat population. The first circle focuses on the bottom of the wheat population, the second circle focuses on the middle of the wheat population, and the third circle focuses on the upper part of the wheat population, forming an angle of about 45° with the ground (to facilitate taking clear pictures of the top and some internal structures of the wheat population), and obtain the original video dataset of the stratified wheat population.

[0088] Step S2: Use NeRF to process the original dataset to generate three-dimensional point cloud data of the wheat population; perform frame extraction on the original video dataset obtained in step S1, remove blurred and cluttered target frames, extract several clear images at each level, and then use NeRF technology to convert the wheat images into three-dimensional point cloud data and export it.

[0089] Step S3: Preprocess the three-dimensional point cloud data. Since the generated wheat point cloud will have a background environment and a large number of noise points, it is necessary to preprocess the wheat point cloud to remove the wheat background environment and most of the noise points. Specifically: Step S301: Use the Voxel Grid Downsampling technology to preprocess the three-dimensional point cloud data. Divide the wheat three-dimensional point cloud space into voxels of a fixed size, set the voxel side length Voxel_size = 0.01m, accurately calculate the boundary range of the point cloud of the point cloud, and this boundary range defines the distribution quadrant of the point cloud data in the three-dimensional space. These small cubes are voxels, and the points are assigned to the voxels. For each point calculate the voxel index where it is located.

[0090]

[0091] Among them, Denotes rounding down, is the point coordinates, , , represent the minimum boundary values of the point cloud data on three axes, v is the voxel side length.

[0092] For each divided voxel, the central method is used to determine its representative point, and the geometric center of the point where the voxel is located is calculated using the central point method:

[0093] Among them, n is the total number of points in this voxel, i represents the serial number index of the points in the voxel. These representative points constitute the downsampled three-dimensional wheat point cloud.

[0094] Step S302: Use the improved RANSAC algorithm to detect and remove the ground from the three-dimensional wheat point cloud data. Set Ransac_n = 3, that is, randomly select 3 points each time in three-dimensional space for plane fitting, and the mathematical expression of the fitted plane is:

[0095] Substitute the remaining points into the fitted plane model, calculate the distance from the point to the plane, if it is less than the set threshold Distance_threshold, it is considered a ground point, that is, an inlier, otherwise it is an outlier. Repeat the above steps to select the plane model containing the most inliers, and then remove the ground points in the plane model from the overall three-dimensional wheat crop point cloud data.

[0096] Step S303: For the remaining noise points after ground removal, use the Statistical Outlier Removal algorithm to remove discrete points. Search for K nearest neighbor points in the neighborhood for each point in the three-dimensional point cloud data, and calculate P the average distance from μ to the neighborhood points .

[0097] Distance between two points:

[0098] Among them, is the point the i th nearest neighbor point, is the point and the point the Euclidean distance between.

[0099] Average distance:

[0100] Standard deviation:

[0101] Set a std_ratio as the judgment criterion for noise points, the threshold , and the points exceeding this threshold are regarded as noise. If the average distance from a certain point to its neighborhood exceeds the threshold, it is determined as a noise point, and the noise is removed from the point cloud while retaining the valid points that meet the requirements.

[0102] Example 4: Refer to Figure 4 , this embodiment of the present invention provides a method for extracting the average height of a wheat population canopy based on three-dimensional point clouds. The content of nitrogen and phosphorus elements in wheat fields has a direct impact on the height of wheat. As key nutrient elements, nitrogen and phosphorus play a significant promoting role in the growth and development of wheat. When the content of nitrogen and phosphorus elements in the soil is maintained within an appropriate range, the higher their content, the stronger the positive promotion effect on the development of wheat roots, the elongation of stems, and the expansion of leaves. Therefore, it can be inferred that in wheat fields with relatively high soil nitrogen and phosphorus content, the average height of wheat is often higher. Based on Example 3, the method of this embodiment of the present invention includes the following steps: Step S1: Align the boundary of the wheat three-dimensional point cloud with the three-dimensional coordinate axes. Specifically: For the preprocessed wheat three-dimensional point cloud data, use the rotation matrix for rotation and translation to align the boundary of the wheat field with the coordinate axes. The calculation formula of the rotation matrix (taking rotation around the X-axis as an example):

[0103] Step S2: Select the top wheat three-dimensional point cloud. Specifically: For the x , y , z coordinates of the rotated and translated wheat three-dimensional model, they can be approximately started from 0, where the z coordinate can be regarded as the height of the point. Set Height_ratio as the height threshold parameter, which can take 15%. Multiply the highest point in the wheat three-dimensional point cloud by Height_ratio as the threshold height.

[0104]

[0105] Select the points with the z coordinate greater than the threshold height. Considering the limitations of the three-dimensional point cloud, there may be noise points at the top of the wheat three-dimensional point cloud. At this time, the top points selected can be further removed by the data with a top proportion of Height_ratio .

[0106] Step S3: Calculate the average height of the qualified points. Specifically: Calculate the average value of the z - coordinates of the remaining points, which is denoted as the average height of the wheat canopy. :

[0107] Among them, n is the number of remaining points, is the i th point among the remaining points, z coordinate, is the maximum Z - coordinate of the wheat point cloud after aligning the coordinate axes.

[0108] Example 5: Refer to Figure 5 and Figure 6 , the embodiment of the present invention provides a method for extracting the multi - level and multi - angle wheat coverage based on three - dimensional point clouds. In field crops, there are significant morphological structure differences in the vertical direction, forming a clear layered structure. Nitrogen and phosphorus elements have a significant promoting effect on the growth of wheat leaves, enabling them to expand and thicken, making the leaves wider and thicker, and the degree of outward expansion also increases. The growth state of wheat crops in the field is roughly divided into three levels: the upper layer is the ear layer, the middle layer is the straw - leaf layer, and the bottom layer is the tillering layer. The three - dimensional model of wheat crops is sliced, and thus the sliced images of the wheat canopy are obtained. Subsequently, the coverage characteristics of the whole wheat three - dimensional point cloud and the levels are extracted. Specifically: Step S1: Slice and layer the wheat three - dimensional point cloud. Since the growth of wheat crop fields varies, only a general range can be determined. First, project the three - dimensional model of wheat crops orthogonally. After pre - processing, the three - dimensional model of wheat crops has no ground information, and only tillers and their leaves remain at the bottom. The wheat tillers and their leaves expand radially outward. The bottom cut - off is set at the highest point where the tiller leaves expand outward, which is approximately at 25% - 30%. And the wheat straw leaves also expand radially outward. The upper cut - off is set at the highest point where the straw leaves expand outward, which is approximately at 25% - 30%. Specifically: Obtain the average height of the wheat crop through the above - mentioned canopy height algorithm, cut this average height and set the cut - off threshold. Set the upper cut - off point at 30% of the top, and the bottom cut - off layer at 30%. The upper cut - off threshold is the average height multiplied by 70%, and the lower cut - off threshold is the average height multiplied by 30%. Sort the z - coordinate array Z_coords of the three - dimensional model of wheat crops in descending order. For those with z - coordinates greater than or equal to the upper cut - off threshold, the z - coordinate indices are moved into the index array top, and for those less than or equal to the lower cut - off threshold, the z - coordinate indices are moved into the index array bottom. At this time, select and save the data of the three - dimensional model of wheat crops corresponding to the top, Z_coords, and botttom arrays, and then the three - layer canopy slices of the wheat crop can be obtained.

[0109] Step S2: Project the 3D point cloud of wheat at each angle; During the generation of the 3D point cloud of the wheat population, the details in the middle layer will fade and blur, and for the upper layer, due to the inability to fix the wheat ears during data collection, some noises that are difficult to remove will be generated. The point cloud effect of the lower tillering layer is the best. Specifically: Step S201: Map the overall 3D wheat model and the slices of the lower tillering layer onto planes at each angle. Assume projection is carried out from the angle θ (the angle formed with the z = 0 plane). Define the discovery vector of the inclined plane n :

[0110]

[0111] Among them, r is the radian value of the angle θ .

[0112] And the general form of the projection plane is:

[0113] Among them, A =sin(r), B =0, C =cos(r), D is a constant.

[0114] Select the origin P =[0, 0, 0] to calculate D : D =-( n · P ) Therefore, the projection plane at this angle is obtained as:

[0115] Step S202: For any point P =[x, y, z] in the point cloud, calculate its perpendicular distance d to the plane:

[0116] Calculate the projection point:

[0117] After expansion:

[0118] Project all the point cloud data onto the plane to obtain θProjected point cloud at an angle.

[0119] Step S203: Draw a scatter plot of the wheat projected point cloud, set the scatter size S (so that the scatter points touch each other to form a surface), color (such as blue), transparency, etc., convert the drawn RGB image to the HSV color space, and define the color range lower_color =[100,50,50], upper_color =[130,255,255] (the color range here is set according to the color of the drawn scatter plot), create a binary mask:

[0120] Calculate the number of non-zero pixels in the mask:

[0121] Step S3: Draw and extract the coverage of the scattered points after mapping projection. Specifically: Calculate the total area of the sample field Total_area Canopy coverage (calculated through the bounding box): .

[0122] This method can extract the canopy coverage characteristics of the wheat three-dimensional point cloud at various angles and levels.

[0123] Example 6: Refer to Figure 7 , this embodiment of the present invention provides a method for extracting the density of the bottom tillering layer of a wheat population based on three-dimensional point clouds. In the application of the algorithm for extracting the canopy coverage of wheat crops, this process will generate a series of mapping images based on different angles. It is found that there are significant differences in the tillering situation between columns and the density of leaves of wheat crops under different nitrogen and phosphorus content conditions. The specific method of this embodiment is as follows: Step S1: Set a reference line in the frontal view mapping of the wheat point cloud and move it to the root position of the wheat. Use the scatter plot generated by the frontal projection of the three-dimensional point cloud to scan row by row from the bottom of the image to find the first row containing a specific color (the color set when drawing the scatter plot), and record it as the root position , move an offset along the tillering direction (vertically upward) , so that the reference line moves to the middle of the tillering:

[0124] Among them, is the image height, usually set to 0.15.

[0125] Step S2: Detect and calculate the density of the tillering area; specifically: Step S201: On the reference line Detect all continuous pixel segments above and record the starting column of each segment and the ending column .

[0126] Step S202: Calculate the single - end distance, total coverage length, total length of the reference line and calculate the results: Single - segment distance:

[0127] Total coverage length:

[0128] Total length of the reference line:

[0129] Then the bottom density is the ratio of the coverage length of the tillering area to the total length of the reference line: .

[0130] Example 7: Refer to Figure 8 、 Figure 9 , the present invention provides a method for extracting the volume ratio characteristics of each layer of a wheat population based on three - dimensional point clouds. When wheat crops face environments with different nitrogen and phosphorus contents, obvious growth differences will be shown. Specifically, the size, length, and degree of expansion of the leaves, as well as the thickness of the stems, will all be different. This growth difference directly leads to different occupied ranges of each wheat individual in space. Generally, the higher the nitrogen and phosphorus content, the more lush the growth of wheat, and the larger the space it occupies, that is, the overall volume is more significant. The volume ratio of the tillering layer can be extracted to judge the nutritional characteristics of wheat. The method of this example is specifically as follows: Step 1: Use the Quickhull algorithm to construct the convex hull. Find the two points with the smallest and the largest x - coordinates in the point set as the initial sides of the convex hull. And divide the remaining points into two groups through the straight line : the point set above the straight line , the point set below the straight line .

[0131] Find the point farthest from the straight line for each group of point sets Q (calculate the distance through the vector cross - product) to Q define the triangular patch and divide the point set into left and right subsets. Recursively process the subsets until all points are included in the convex hull.

[0132] Among them, the formula for the distance from a point to a straight line is:

[0133] : point Q to the end point of the line P vector; : line direction vector; d : point Q to the perpendicular distance of the line.

[0134] Step 2, use the tetrahedron decomposition method to calculate the volume of the whole wheat canopy and the volume of the tillering layer and calculate the proportion of the tillering layer volume. Optionally select the convex hull vertex D as the common base point. For each triangular patch (vertices are A , B , C ), form a tetrahedron D with the base point ABCD , and calculate the volume of each tetrahedron.

[0135]

[0136] , , : vector from the base point D to other vertices.

[0137] : vector cross product, : vector point set.

[0138] Sum up the volumes of all tetrahedrons:

[0139] M is the number of tetrahedrons.

[0140] Take the ratio of the tillering layer volume to the whole wheat canopy volume as the proportion of the tillering layer volume:

[0141] Among them, is the tillering layer volume, is the whole wheat canopy volume.

[0142] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting multi-dimensional features of wheat population based on three-dimensional point cloud and image processing, characterized in that The extraction method includes the following steps: Step S1: Obtain multi-view video data of the target wheat population; Step S2: Perform frame processing on the multi-view video data, and generate three-dimensional point cloud data of the wheat population through NeRF technology; Step S3: Preprocess the three-dimensional point cloud data by voxel downsampling, ground point segmentation, and outlier removal methods; Step S4: Extract the average height feature of the wheat population based on the preprocessed three-dimensional point cloud data; Step S5: Perform spatial stratification on the preprocessed three-dimensional point cloud data to obtain tiller layer point cloud data; Step S6: Based on the tiller layer point cloud data, calculate the canopy coverage feature by multi-angle projection and combined with the color mask method; Step S7: Use the Quickhull algorithm to construct three-dimensional convex hull models of the whole wheat and the tiller layer, and use the tetrahedron decomposition method to calculate the tiller layer volume ratio feature; Step S8: Calculate the tiller density feature based on the tiller layer baseline analysis.

2. The extraction method according to claim 1, wherein In step S1, obtaining the multi-view video data of the target wheat population specifically means using a portable device to shoot three videos around the target wheat population. The first video focuses on the bottom area of the target wheat population, the second video focuses on the middle area of the target wheat population, and the third video focuses on the top area of the population at a 45° elevation angle.

3. The extraction method according to claim 2, wherein Step S2 specifically means: Extract key frames from each video sequence, remove blurred frames, select a set number of clear images from each spatial partition, and fuse multi-view image frames through NeRF technology to generate three-dimensional point cloud data of the wheat population.

4. The extraction method according to claim 1, wherein The preprocessing in step S3 specifically means: Step S301: Calculate the boundary range of the three-dimensional point cloud data of the wheat population, divide the point cloud data into a uniform cubic grid with a fixed side length, calculate the centroid of the points in each voxel as the representative point, and generate the downsampled point cloud data; Step S302: Use the RANSAC algorithm in the three-dimensional point cloud data to randomly select three sample points to construct a plane model, calculate the Euclidean distance from the remaining points in the downsampled point cloud data to the plane model, and select the points with a distance less than the set threshold as the ground point set and remove them from the point cloud; Step S303: Perform statistical filtering on the three-dimensional point cloud after removing the ground, calculate the average Euclidean distance and standard deviation within the neighborhood of each point, mark and remove outliers by setting a standard deviation multiple threshold, and output the preprocessed point cloud data.

5. The extraction method according to claim 1, characterized in that, In step S4, extracting the average height feature of the wheat population specifically means: Step S401: Use the rotation matrix to perform rotation and translation operations on the preprocessed point cloud data to align the wheat boundary with the coordinate axes, and define the z-axis coordinate as the point cloud height; Step S402: Set a height ratio threshold, and use the product of the z value of the highest point of the point cloud and the height ratio threshold as the threshold height; Step S403: Screen the point cloud data with a z coordinate greater than the threshold height, and remove the point cloud with a top proportion of the height ratio threshold, and output the effective height point cloud; Step S403: Calculate the average height feature of the wheat population based on the effective height point cloud.

6. The extraction method according to claim 1, wherein In step S5, the preprocessed three-dimensional point cloud data is spatially stratified to obtain tiller layer point cloud data, specifically: the preprocessed point cloud data is divided from top to bottom by vertical height into an ear layer, a straw and leaf layer, and a tiller layer, and the point cloud data corresponding to each stratified level is output.

7. The extraction method according to claim 1, wherein The method for calculating the canopy coverage feature in step S6 is specifically as follows: Step S601: Project the overall point cloud data and the tiller layer point cloud data onto a mapping plane at a specified angle to generate a projected point cloud; Step S602: Draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud into the HSV color space, and calculate the number of non-zero pixels in the mask as the canopy coverage area; Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage feature.

8. The extraction method according to claim 1, characterized in that The method for calculating the tiller layer volume ratio feature in step S7 is specifically as follows: Step S701: Based on the stratified tiller layer point cloud data and the preprocessed overall point cloud data, use the Quickhull algorithm to construct three-dimensional convex hull models respectively; specifically: select the point with the smallest x coordinate in the overall point cloud P min and the point with the largest x coordinate P max , and construct an initial segmentation line; Calculate the distances from each point to the initial dividing line, and select the point with the maximum distance P for As a new convex hull vertex; recursively divide the point set and repeat the vertex selection until the construction of the overall convex hull and the tillering layer convex hull is completed; Step S702: Calculate the volume of each convex hull and the total volume through the tetrahedron decomposition method; specifically, select a base point from the vertices of the tillering layer convex hull D ; Traverse each triangular patch of the tillering layer convex hull ( A , B , C ), and form a tetrahedron with the base point D ( A , B , C , D ), calculate the volume of each tetrahedron; accumulate the volumes of all tetrahedrons to obtain the total volume of the tillering layer convex hull; Calculate the total volume of the overall convex hull in the same way; Step S703: Take the ratio of the total volume of the tiller layer convex hull to the total volume of the overall convex hull as the tiller layer volume ratio feature.

9. The extraction method according to claim 1, wherein The method for calculating the tiller density feature in step S8 is specifically as follows: Step S801: Generate a front view projection scatter plot based on the stratified tiller layer point cloud data; Step S802: Start scanning line by line from the lower end of the scatter plot, and locate the first line containing the preset color value as the root baseline , set the offset and according to the offset Adjust the baseline to the middle area of the tillering layer to obtain the position of the tillering layer baseline. The calculation formula of the tillering layer baseline is as follows: Among them, is the height of the scatter plot, usually set to 0.15; Step S803: Detect all continuous pixel segments on the tillering layer reference line and record the starting column and the ending column of each segment, and calculate the single-segment coverage length, the total coverage length, and the total length of the tillering layer reference line; Step S804: Statistically calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the tiller density feature.

Images