Multi-dimensional feature extraction method of wheat population based on 3D point cloud and image processing

Abstract

The present invention discloses a method for extracting multidimensional features of wheat populations based on three-dimensional point clouds and image processing, which relates to the technical field of image feature extraction. The method comprises the following steps: obtaining multi-perspective video data of a target wheat population and performing frame processing to generate three-dimensional point cloud data of the wheat population; pre-processing the three-dimensional point cloud data and then spatially stratifying it to calculate the average height characteristics, canopy coverage characteristics, tiller layer volume ratio characteristics, and tiller density characteristics of the wheat population. The present invention has the beneficial effects of effectively extracting multi-level features such as wheat morphology, structure, and density, significantly improving the comprehensiveness of feature representation and data robustness, and providing a high-precision, low-cost monitoring solution for wheat growth status assessment and precision fertilization.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for extracting multi-dimensional features of a wheat population based on three-dimensional point cloud and image processing. Background Art

[0002] In recent years, the rapid development of remote sensing, computer vision, and 3D point cloud technologies has driven advancements in non-contact agricultural monitoring. 3D point cloud technology, in particular, has become a crucial tool for wheat growth monitoring due to its ability to capture comprehensive 3D spatial information about wheat plants, including morphological characteristics (such as plant height and biomass) and structural features (such as canopy distribution and leaf inclination). Research has shown that wheat exhibits distinct canopy structural characteristics under different nutritional states, such as increased leaf uprightness in nitrogen-deficient plants and decreased canopy openness in phosphorus-deficient plants. This provides a theoretical basis for 3D feature-based wheat nutritional diagnosis.

[0003] However, the current wheat monitoring methods based on three-dimensional point clouds still face three challenges: First, existing technologies mostly focus on the extraction of morphological features, and fail to effectively integrate structural characteristic parameters such as the spatial distribution of stems and leaves and canopy transmittance, resulting in a single dimension of growth status assessment; second, nutritional diagnosis models generally rely on a single key feature indicator, and fail to fully tap the collaborative representation capabilities of multidimensional features in three-dimensional point cloud data; third, existing methods are too dependent on the quality of point cloud data. In point cloud data obtained under complex field lighting conditions and low-precision scanning equipment, there is a lack of robust feature extraction methods, which seriously restricts the application and promotion of the technology in large-scale fields. Summary of the Invention

[0004] In order to solve the technical problems in existing wheat monitoring methods that the feature extraction dimension is single, relies on high-precision point cloud data and ignores the canopy structure characteristics, the present invention provides a multi-level feature extraction method for wheat crops based on three-dimensional point cloud and image processing. By integrating point cloud downsampling, spatial stratification and multi-angle projection technology, the feature extraction of plant height, canopy coverage, tiller layer volume ratio and density is achieved.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides a method for extracting multidimensional features of wheat populations based on three-dimensional point cloud and image processing, the extraction method comprising the following steps:

[0006] Step S1: Acquire multi-view video data of the target wheat population;

[0007] Step S2: performing frame processing on the multi-view video data and generating three-dimensional point cloud data of the wheat population using NeRF technology;

[0008] Step S3: pre-processing the three-dimensional point cloud data by voxel downsampling, ground point segmentation and outlier removal methods;

[0009] Step S4: extracting the average height feature of the wheat population based on the preprocessed three-dimensional point cloud data;

[0010] Step S5: spatially stratifying the pre-processed three-dimensional point cloud data to obtain tillering layer point cloud data;

[0011] Step S6: Based on the tiller layer point cloud data, the canopy cover characteristics are calculated by multi-angle projection combined with the color mask method;

[0012] Step S7: using the Quickhull algorithm to construct a three-dimensional convex hull model of the wheat as a whole and the tiller layer, and using the tetrahedron decomposition method to calculate the volume ratio characteristics of the tiller layer;

[0013] Step S8: Based on the tiller layer baseline analysis, calculate the density characteristics at the tillering location.

[0014] Among them, step S1 obtains multi-perspective video data of the target wheat group, specifically, using a portable device to shoot three videos around the target wheat group, the first video focuses on the bottom area of the target wheat group, the second video focuses on the middle area of the target wheat group, and the third video focuses on the top area of the group at a 45° elevation angle.

[0015] The step S2 specifically includes: extracting key frames from each video sequence, removing blurred frames, screening a set number of clear images from each spatial partition, and fusing multi-view image frames using NeRF technology to generate three-dimensional point cloud data of the wheat population.

[0016] The pre-processing in step S3 is specifically as follows:

[0017] Step S301: Calculating the boundary range of the three-dimensional point cloud data of the wheat population, dividing the point cloud data into a uniform cubic grid with a fixed side length, calculating the centroid of each point within each voxel as a representative point, and generating downsampled point cloud data;

[0018] Step S302: Using the RANSAC algorithm, three sample points are randomly selected from the 3D point cloud data to construct a plane model. The Euclidean distance between the remaining points in the downsampled point cloud data and the plane model is calculated. Points with a distance less than a set threshold are selected as ground point sets and removed from the point cloud.

[0019] Step S303: Statistical filtering is performed on the three-dimensional point cloud after removing the ground, and the average Euclidean distance and standard deviation in the neighborhood of each point are calculated. Outliers are marked and removed by setting a threshold value of the standard deviation multiple, and the pre-processed point cloud data is output.

[0020] The average height feature of the wheat population is extracted in step S4, specifically:

[0021] Step S401: Using a rotation matrix, the pre-processed point cloud data is rotated and translated so that the boundary of the wheat is aligned with the coordinate axis, and the z-axis coordinate is defined as the point cloud height;

[0022] Step S402: setting a height ratio threshold, with the product of the z value of the highest point of the point cloud and the height ratio threshold being used as the threshold height;

[0023] Step S403: filtering point cloud data with z coordinates greater than a threshold height, removing point clouds with a top ratio equal to the height ratio threshold, and outputting a valid height point cloud;

[0024] Step S403: Calculate the average height characteristics of the wheat population based on the effective height point cloud.

[0025] In the step S5, the preprocessed three-dimensional point cloud data is spatially layered to obtain tiller layer point cloud data, specifically: the preprocessed point cloud data is divided into a wheat ear layer, a wheat straw and leaf layer, and a tiller layer from top to bottom according to the vertical height, and the point cloud data corresponding to each layer after layering is output; wherein the height proportion of each layer is determined by a preset layering ratio parameter, which can be selected as: the upper wheat ear layer accounts for 30%, the middle wheat straw and leaf layer accounts for 45% and the lower tiller layer accounts for 25%.

[0026] The method for calculating the canopy coverage characteristics in step S6 is specifically as follows:

[0027] Step S601: Projecting the overall point cloud data and the tiller layer point cloud data onto a mapping plane of a specified angle to generate a projected point cloud;

[0028] Step S602: draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud into the HSV color space, and calculate the number of non-zero pixels in the mask as the canopy coverage area;

[0029] Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage characteristics.

[0030] The method for calculating the volume ratio characteristics of the tillering layer in step S7 is specifically as follows:

[0031] Step S701: Based on the stratified tillering layer point cloud data and the pre-processed overall point cloud data, a three-dimensional convex hull model is constructed using the Quickhull algorithm; specifically, the point with the minimum x coordinate in the overall point cloud is selected. P min and maximum point P max , construct the initial segmentation line; calculate the distance between each point and the initial segmentation line, and select the point with the largest distance P forAs a new convex hull vertex; recursively split the point set and repeat vertex screening until the overall convex hull and tiller layer convex hull construction are completed;

[0032] Step S702: Calculate the volume of each convex hull and the total volume by tetrahedron decomposition method; specifically, select the base point from the vertices of the convex hull of the tiller layer D ; Traverse each triangle patch of the convex hull of the tiller layer ( A , B , C ), and with the base point D Tetrahedron ( A , B , C , D ), calculate the volume of each tetrahedron; add up the volumes of all tetrahedrons to get the total volume of the tillering layer convex hull; calculate the total volume of the overall convex hull in the same way;

[0033] Step S703: The ratio of the total volume of the tillering layer convex hull to the total volume of the entire convex hull is used as the tillering layer volume ratio feature.

[0034] The method for calculating the density characteristics at the tillering position in step S8 is specifically as follows:

[0035] Step S801: generating an orthographic projection scatter plot based on the layered tiller layer point cloud data;

[0036] Step S802: Scan row by row along the bottom of the scatter plot and locate the first row containing the preset color value as the root reference line , set the offset and adjust the value according to the offset Adjust the baseline to the middle area of the tillering layer to obtain the position of the tillering layer baseline. The calculation formula is:

[0037]

[0038] in, is the height of the scatter plot, Usually set to 0.15;

[0039] Step S803: At the tillering layer baseline Detect all continuous pixel segments and record the starting column of each segment and the termination column , and calculate the single-segment coverage length, total coverage length and total length of the tillering layer baseline;

[0040] Step S804: Calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the density feature of the tillering.

[0041] The beneficial effects of the present invention are: the present invention reduces point cloud noise interference through voxel downsampling and statistical filtering, combines layered mapping and convex hull volume decomposition technology, effectively extracts multi-level features such as wheat morphology, structure and density, significantly improves the comprehensiveness of feature representation and data robustness, and provides a high-precision, low-cost monitoring solution for wheat growth status assessment and precise fertilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of Example 2 of the present invention.

[0043] Figure 2 This is a schematic diagram of data acquisition in Example 3 of the present invention.

[0044] Figure 3 3D point cloud preprocessing results in Example 3 of the present invention.

[0045] Figure 4 Schematic diagram of average height extraction in Example 4 of the present invention.

[0046] Figure 5 Schematic diagram of the overall canopy and tillering layer of wheat in Example 5 of the present invention.

[0047] Figure 6 This is a schematic diagram of the coverage of the wheat point cloud at 30 degrees in Example 5 of the present invention.

[0048] Figure 7 This is a schematic diagram of the bottom density baseline in Example 6 of the present invention.

[0049] Figure 8 It is a schematic diagram of the convex hull of the tillering layer in Example 7 of the present invention.

[0050] Figure 9 This is a schematic diagram of the convex hull of the wheat canopy in Example 7 of the present invention. DETAILED DESCRIPTION

[0051] In order to clearly illustrate the technical features of this solution, this solution is described below through specific implementation methods.

[0052] Example 1: The present invention provides a method for extracting multidimensional features of wheat populations based on three-dimensional point cloud and image processing. The extraction method comprises the following steps:

[0053] Step S1: Obtain multi-perspective video data of the target wheat group; specifically, use a portable device to shoot three videos around the target wheat group, the first video focuses on the bottom area of the target wheat group, the second video focuses on the middle area of the target wheat group, and the third video focuses on the top area of the group at a 45° elevation angle.

[0054] Step S2: Frame processing is performed on the multi-view video data, and three-dimensional point cloud data of the wheat group is generated using NeRF technology. Specifically, key frames are extracted from each video sequence, and blurred frames are removed. A set number of clear images are screened from each spatial partition, and multi-view image frames are fused using NeRF technology to generate three-dimensional point cloud data of the wheat group.

[0055] Step S3: Preprocessing the three-dimensional point cloud data by voxel downsampling, ground point segmentation, and outlier removal methods; specifically, the following steps are included:

[0056] Step S301: Calculating the boundary range of the three-dimensional point cloud data of the wheat population, dividing the point cloud data into a uniform cubic grid with a fixed side length, calculating the centroid of each point within each voxel as a representative point, and generating downsampled point cloud data;

[0057] Step S302: Using the RANSAC algorithm, three sample points are randomly selected from the 3D point cloud data to construct a plane model. The Euclidean distance between the remaining points in the downsampled point cloud data and the plane model is calculated. Points with a distance less than a set threshold are selected as ground point sets and removed from the point cloud.

[0058] Step S303: Statistical filtering is performed on the three-dimensional point cloud after removing the ground, and the average Euclidean distance and standard deviation in the neighborhood of each point are calculated. Outliers are marked and removed by setting a threshold value of the standard deviation multiple, and the pre-processed point cloud data is output.

[0059] Step S4: extracting the average height feature of the wheat population based on the pre-processed three-dimensional point cloud data; specifically comprising the following steps:

[0060] Step S401: Using a rotation matrix, the pre-processed point cloud data is rotated and translated so that the boundary of the wheat is aligned with the coordinate axis, and the z-axis coordinate is defined as the point cloud height;

[0061] Step S402: setting a height ratio threshold, with the product of the z value of the highest point of the point cloud and the height ratio threshold being used as the threshold height;

[0062] Step S403: filtering point cloud data with z coordinates greater than a threshold height, removing point clouds with a top ratio equal to the height ratio threshold, and outputting a valid height point cloud;

[0063] Step S403: Calculate the average height characteristics of the wheat population based on the effective height point cloud.

[0064] Step S5: Spatially stratify the pre-processed three-dimensional point cloud data to obtain tiller layer point cloud data. Specifically, the pre-processed point cloud data is divided into the wheat ear layer, the wheat straw leaf layer, and the tiller layer from top to bottom according to vertical height, and the point cloud data corresponding to each layer after stratification is output. The height ratio of each layer is determined by a preset layer ratio parameter, which can be selected as: 30% for the upper wheat ear layer, 45% for the middle wheat straw leaf layer, and 25% for the lower tiller layer.

[0065] Step S6: Based on the tiller layer point cloud data, the canopy coverage characteristics are calculated by multi-angle projection combined with the color mask method; specifically:

[0066] Step S601: Projecting the overall point cloud data and the tiller layer point cloud data onto a mapping plane of a specified angle to generate a projected point cloud;

[0067] Step S602: draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud into the HSV color space, and calculate the number of non-zero pixels in the mask as the canopy coverage area;

[0068] Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage characteristics.

[0069] Step S7: Use the Quickhull algorithm to construct a three-dimensional convex hull model of the wheat as a whole and the tiller layer, and use the tetrahedron decomposition method to calculate the volume ratio characteristics of the tiller layer; specifically:

[0070] Step S701: Based on the stratified tillering layer point cloud data and the pre-processed overall point cloud data, a three-dimensional convex hull model is constructed using the Quickhull algorithm; specifically, the point with the minimum x coordinate in the overall point cloud is selected. P min and maximum point P max , construct the initial segmentation line; calculate the distance between each point and the initial segmentation line, and select the point with the largest distance P for As a new convex hull vertex; recursively split the point set and repeat vertex screening until the overall convex hull and tiller layer convex hull construction are completed;

[0071] Step S702: Calculate the volume of each convex hull and the total volume by tetrahedron decomposition method; specifically, select the base point from the vertices of the convex hull of the tiller layer D ; Traverse each triangle patch of the convex hull of the tiller layer ( A , B , C ), and with the base point D Tetrahedron ( A , B , C, D ), calculate the volume of each tetrahedron; add up the volumes of all tetrahedrons to get the total volume of the tillering layer convex hull; calculate the total volume of the overall convex hull in the same way;

[0072] Step S703: The ratio of the total volume of the tillering layer convex hull to the total volume of the entire convex hull is used as the tillering layer volume ratio feature.

[0073] Step S8: Based on the tillering layer baseline analysis, calculate the density characteristics at the tillering location; specifically:

[0074] Step S801: generating an orthographic projection scatter plot based on the layered tiller layer point cloud data;

[0075] Step S802: Scan row by row along the bottom of the scatter plot and locate the first row containing the preset color value as the root reference line , set the offset and adjust the value according to the offset Adjust the baseline to the middle area of the tillering layer to obtain the position of the tillering layer baseline. The calculation formula is:

[0076]

[0077] in, is the height of the scatter plot, Usually set to 0.15;

[0078] Step S803: At the tillering layer baseline Detect all continuous pixel segments and record the starting column of each segment and the termination column , and calculate the single-segment coverage length, total coverage length and total length of the tillering layer baseline;

[0079] Step S804: Calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the density feature of the tillering.

[0080] Example 2: See Figure 1 The present invention provides a wheat population feature extraction method based on 3D point cloud and image processing. This method extracts a variety of effective and high-value feature information from low-precision 3D wheat point clouds, providing a basis for real-time monitoring of wheat growth and nutritional diagnosis in the field. The extraction method specifically includes the following steps:

[0081] 1. Data Collection. Currently, high-end equipment is often used to construct 3D point clouds. This method uses portable recording equipment (such as mobile phones or cameras) to capture wheat data from a field. Specifically, the method involves capturing three circles around a wheat field. The first circle focuses on the base of the field, the second on the middle, and the third on the upper portion, forming an angle of approximately 45° to the ground (enabling the top and internal structures of the field to be clearly captured). This yields a raw video dataset of a layered wheat field.

[0082] 2. Data Processing to Obtain 3D Point Clouds: Frame processing is performed on the video dataset obtained in step 1 to remove blurry and cluttered target frames. Several clear images are extracted at each level, and the wheat images are converted into 3D point cloud data using NeRF technology.

[0083] 3. Data preprocessing. The three-dimensional point cloud data of the wheat group obtained in step 2 contains noise points and also contains background information of the wheat group. Noise points and background information are one of the reasons why it is difficult to extract features from low-precision three-dimensional point clouds. The data preprocessing method of this embodiment is specifically as follows:

[0084] Step 3.1: Remove outliers from the 3D wheat point cloud. Using Voxel Grid Downsampling, the 3D wheat space is divided into fixed-size voxels (voxels). Within each voxel, a small number of representative points (such as the closest point, centroid, or maximum distance point) are retained. This reduces the density of the original 3D wheat point cloud and removes noise points. This effectively preserves the overall geometric structure of the point cloud while significantly reducing computational complexity. After using RANSAC to detect and segment the ground points, a statistical filtering algorithm is used to remove the remaining sparsely distributed outliers.

[0085] First, the pre-processed three-dimensional space is divided into small cubes (voxels) of fixed size. The side length of each voxel is pre-set as Voxel_size, which can be selected as 0.1m. The point cloud boundary range of the point cloud is accurately calculated. This boundary range defines the distribution quadrant of the point cloud data in the three-dimensional space. According to the calculated point cloud boundary range, the three-dimensional space is divided into several small cubes. These small cubes are voxels. Points are assigned to voxels. For each point Calculate the voxel index:

[0086]

[0087] in, Indicates rounding down. for point The coordinates of 、 、 Represents the minimum boundary value of the point cloud data on the three axes, v is the voxel side length.

[0088] Secondly, for each divided voxel, the center method is used to determine its representative point, the coordinates of all points in the voxel are statistically analyzed, and the center point method is used to calculate the geometric center of the point in the voxel. :

[0089]

[0090] in, n is the total number of points in the voxel, i Represents the ordinal index of the point inside the voxel.

[0091] Finally, the representative points selected from all voxels using the above method are collected together to form the downsampled 3D wheat point cloud. The downsampling process reduces the density of the original 3D wheat point cloud while better preserving the overall structure of the point cloud.

[0092] Step 3.2: Given that ground points usually occupy the majority of point cloud data and do not contribute to the feature extraction of wheat populations, the RANSAC plane segmentation algorithm is used to detect the ground model. Specifically:

[0093] First, the optimal ground model is found through random sampling and iterative fitting. A minimum sample set is randomly selected from the three-dimensional wheat model point data. This sample set is the minimum number of samples required to initialize the model parameters. Fitting a plane in three-dimensional space requires at least three sample points (Ransac_n). Using this minimum sample set, a plane model is fitted, representing the wheat crop ground. The mathematical expression for the plane model is typically:

[0094]

[0095] in, A 、 B 、 C 、 D is the plane parameter, x 、 y 、 z are the coordinates of the wheat crop point cloud.

[0096] Secondly, the remaining points are brought into the fitted plane model and the distance from the point to the plane is calculated. If it is less than a set threshold Distance_threshold, it is considered a ground point, otherwise it is an external point. The expression is:

[0097]

[0098] , , Represents the coordinates of the point to be determined.

[0099] Again, count the number of inliers. The model with more inliers is more likely to be the correct model. Repeat the above steps (random sampling, model fitting, inlier determination, model evaluation) to select the best model as the final model, i.e., the wheat crop ground. It is recorded in an array, which contains the index of the points identified as inliers or the data of the actual points ( x 、 y 、 z ) Then the points in the array are removed from the overall wheat crop 3D model point cloud data to complete the removal of the wheat crop ground.

[0100] Step 3.3: Because the fitted ground model may be different, some noise points will be generated near the ground after the wheat crop point cloud is removed. In this case, use Statistical Outlier Removal. Specifically:

[0101] First, select K The nearest neighboring points are used as local neighborhoods and calculations are performed in the local neighborhoods. P Compute its distance to each point in its local neighborhood:

[0102]

[0103] in, Yes No. i The nearest point, Yes with dot The Euclidean distance between .

[0104] Calculate the average distance of the local neighborhood :

[0105]

[0106] Calculate the standard deviation of a local neighborhood :

[0107]

[0108] Next, set a parameter std_ratio To control the judgment standard of noise points. Calculate the judgment threshold of noise points:

[0109]

[0110] For a point P, if its average distance μIf it exceeds the threshold, it will be marked as a noise point:

[0111]

[0112] Traverse all points in the point cloud, mark and remove all noise points, and finally obtain the denoised wheat point cloud data.

[0113] 4. Perform multi-angle and multi-level feature extraction on the pre-processed wheat 3D point cloud. The growth conditions of wheat vary under different nutrient content. This embodiment provides a method for extracting features such as canopy coverage, canopy volume ratio, tiller density, and average canopy height of a wheat population. The extracted features can be used to diagnose and analyze the growth status of wheat. Specifically:

[0114] Step 4.1: Extract canopy height features. Use the rotation matrix to rotate and translate the pre-processed wheat 3D point cloud so that the boundary of the wheat field is aligned with the coordinate axis. The rotation matrix calculation formula (taking rotation around the X axis as an example) is:

[0115]

[0116] The rotated 3D model of wheat x 、 y 、 z The coordinates can be approximately started from 0, where the z coordinate can be regarded as the height of the point; set Height_ratio As the height threshold parameter, 15% can be taken, and the highest point in the wheat 3D point cloud is With the height threshold parameter Height_ratio The product of .

[0117]

[0118] take out z Points with coordinates greater than the threshold height. Considering the limitations of 3D point clouds, there will still be noise points at the top of the wheat 3D point cloud. At this time, the top portion can be further removed. Height_ratio For the remaining points z The coordinates are averaged and recorded as the average height of the wheat canopy .

[0119]

[0120] in, n is the number of remaining points, The remaining points i The z coordinate of the point, The maximum Z coordinate of the wheat point cloud after aligning the coordinate axes.

[0121] Step 4.2: Extract the wheat tillering layer and the overall canopy coverage at various angles. Given that wheat fields have a clear vertical layered structure, after multiple surveys and measurements of sample fields, the wheat fields were divided into three layers: the upper ear layer, the middle straw and leaf layer, and the lower tiller layer, with corresponding proportions of approximately 30%, 45%, and 25%. Based on the calculated average height of the wheat canopy, the wheat 3D point cloud was divided into three layers according to the height thresholds divided by the above proportions. During the generation process of the wheat 3D point cloud, the details of the middle layer will fade and blur, and the upper layer will produce some difficult-to-remove noise because the wheat ears cannot be fixed during data collection. The point cloud of the lower tiller layer has the best effect.

[0122] Step 4.2.1, now map the wheat 3D model to the plane at the specified angle. Assume that θ (the angle formed with the z=0 plane) for projection. Define the found vector n of the tilted plane:

[0123]

[0124]

[0125] in, r is the angle θ The radian value.

[0126] The general form of the projection plane is:

[0127]

[0128] in, A =sin( r ), B =0, C =cos( r ), D is a constant.

[0129] Select Origin P =[0,0,0] calculation D :

[0130] D =-(n· P )

[0131] Therefore, the projection plane at this angle is:

[0132]

[0133] Step 4.2.3, any point in the point cloud P =[x,y,z], calculate its vertical distance to the plane d :

[0134]

[0135] Calculate the projection point:

[0136]

[0137] After expansion:

[0138]

[0139] Project all point cloud data onto the plane and get θ Projected point cloud under angle.

[0140] Step 4.2.4: Draw a scatter plot of the wheat projection point cloud, set the scatter point size S (so that the scatter points touch each other to form a surface), color (such as blue), and transparency (0.9), convert the drawn RGB image to HSV color space, and define the color range. lower_color =[100,50,50], upper_color =[130,255,255]. Create a binary mask:

[0141]

[0142] Count the number of non-zero pixels in the mask:

[0143]

[0144] Calculate the total area of the sample plot Total_area Canopy cover (calculated from bounding box):

[0145]

[0146] Similarly, this method can be used to extract the canopy coverage of wheat three-dimensional point clouds at various angles and levels.

[0147] Step 4.3: Calculate the ratio of the tillering layer canopy volume to the total canopy volume. Use the Quickhull algorithm to construct the convex hull of the wheat canopy point cloud. Select the point with the minimum x coordinate from the point set. and the largest point , these two points must be part of the convex hull. and Form a line: one part is above the line and the other part is below.

[0148] Step 4.3.1, for each point P , calculate its distance to the straight line d :

[0149]

[0150] in, Represents the cross product of vectors.

[0151] Step 4.3.2: Find the point farthest from the line , which is also part of the convex hull. 、 and Define a new triangular face and further divide the point set into two subsets. Recursively repeat the above steps for these two subsets until there are no points to be processed.

[0152] Step 4.3.3, calculate the volume of the convex hull using the tetrahedron decomposition method. Select a fixed point from all the vertices of the convex hull D As the base point, traverse the convex hull surface, each surface is recorded as ( A , B , C ), for each triangular face and the base point, a tetrahedron is formed. Traverse all the faces and add the volume of each tetrahedron to the total volume. Use the same method to calculate the canopy volume of the tillering layer and the ratio is the tillering layer volume ratio.

[0153]

[0154] Step 4.4: Calculate the density characteristics of the tiller layer. The density of wheat tillers varies under different nutrient content. It is necessary to define a baseline at the wheat tiller. The main idea is to calculate the proportion of the wheat tiller to the baseline. Use the scatter plot drawing method provided in step 4.2 to draw the front view scatter plot, convert the RGB color space to the HSV color space, and define the color mask (blue) used when drawing the scatter plot. low_color and upper_ color The mask is checked row by row from one of the upper and lower sections of the image until the first row containing the specified color is found (i.e., the first time the baseline reaches the root of the wheat plant in the front view). The baseline is shifted toward the tiller, with an offset value set to Offset, which moves the baseline to the middle of the tiller without exceeding the image boundary.

[0155] Step 4.4.1. Find all continuous segments of a specific color on the baseline. For each continuous area, calculate its distance to Object_distance, and then calculate the distance between the leftmost and rightmost sides of the baseline. Length .

[0156] Step 4.4.2: Use the values calculated above to calculate the ratio of the distance between each continuous area and the length of the baseline:

[0157] .

[0158] The wheat population characteristics extracted by the method provided in this embodiment can be further processed to construct a diagnostic model.

[0159] Example 3: See Figure 2 and Figure 3 The present invention provides a preprocessing method based on a three-dimensional point cloud of a wheat population, which specifically comprises the following steps:

[0160] Step S1: Obtain the original data set of the wheat population; this example collects partitions of the same wheat species, at different periods, and with different nitrogen and phosphorus application levels. The experiment sets four nitrogen levels: N-0%, N-30%, N-60%, and N-CK, and four phosphorus levels: P-0%, P-30%, P-60%, and P-CK. Different block groups are isolated by 1m, and different fertilization block groups are isolated by 1m. There are a total of 24 partitions, and the area of each partition is 2.5m. Wheat was selected in three stages: greening stage, jointing stage and heading stage.

[0161] Wheat data was collected by filming wheat fields using portable video recording equipment (e.g., mobile phones or cameras). Specifically, the video was taken three times around the wheat field. The first circle focused on the base of the field, the second on the middle, and the third on the upper part of the field, forming an angle of approximately 45° to the ground (to clearly capture the top of the field and some internal structures). This yielded a raw video dataset of a stratified wheat field.

[0162] Step S2: Use NeRF to process the original data set to generate three-dimensional point cloud data of the wheat population; perform frame processing on the original video data set obtained in step S1, remove the blurred and cluttered target frames, extract several clear images at each level, and then use NeRF technology to convert the wheat images into three-dimensional point cloud data and export them.

[0163] Step S3: Preprocess the 3D point cloud data. Since the generated wheat point cloud will contain background environment and a large number of noise points, it is necessary to preprocess the wheat point cloud to remove the wheat background environment and most of the noise points. Specifically:

[0164] Step S301: Use Voxel Grid Downsampling technology to pre-process the 3D point cloud data. Divide the wheat 3D point cloud space into fixed-size voxels, set the voxel side length Voxel_size = 0.01m, and accurately calculate the point cloud boundary range. This boundary range defines the distribution quadrant of the point cloud data in the 3D space. These small cubes are voxels, and points are assigned to voxels. For each point The voxel index at which the calculation is performed.

[0165]

[0166] in, Indicates rounding down. for point The coordinates of 、 、 Represents the minimum boundary value of the point cloud data on the three axes, v is the voxel side length.

[0167] For each divided voxel, the center method is used to determine its representative point, and the center point method is used to calculate the geometric center of the voxel:

[0168]

[0169] in, n is the total number of points in the voxel, i The index of the points in the voxel. These representative points constitute the downsampled 3D point cloud of wheat.

[0170] Step S302: Use the improved RANSAC algorithm to perform ground detection and removal on the wheat 3D point cloud data. Set Ransac_n=3, that is, randomly select 3 points in the 3D space each time to perform plane fitting. The mathematical expression of the fitted plane is:

[0171]

[0172] The remaining points are inserted into the fitted plane model and the distance from the point to the plane is calculated. If the distance is less than the set threshold, Distance_threshold, the point is considered an inlier. Otherwise, it is an outlier. Repeat the above steps to select the plane model containing the most inliers. Then, remove the ground points in the plane model from the entire wheat crop 3D point cloud data.

[0173] Step S303: For the noise points remaining after ground removal, use the Statistical Outlier Removal algorithm to remove the discrete points. K Neighborhood nearest neighbor, calculation P Average distance to neighboring points μ and standard deviation .

[0174] Distance between two points:

[0175]

[0176] in, Yes No. i The nearest point, Yes with dot The Euclidean distance between .

[0177] Average distance:

[0178]

[0179] Standard Deviation:

[0180]

[0181] Set a std_ratio Used as the judgment standard for noise points, threshold , points exceeding this threshold are considered noise. If the average distance from a point to its neighborhood exceeds the threshold, it is considered a noise point and the noise is removed from the point cloud to retain valid points that meet the requirements.

[0182] Example 4: See Figure 4 , an embodiment of the present invention provides a method for extracting the average height of the canopy of a wheat population based on a three-dimensional point cloud. The content of nitrogen and phosphorus in a wheat field has a direct impact on the height of wheat. As key nutrients, nitrogen and phosphorus play a significant role in promoting the growth and development of wheat. When the nitrogen and phosphorus content in the soil is maintained within an appropriate range, the higher the content, the stronger the positive driving effect on the development of the wheat root system, the elongation of the stem, and the expansion of the leaves. Therefore, it can be inferred that in wheat fields with higher nitrogen and phosphorus content in the soil, the average height of wheat is often higher. Based on Example 3, the method of the embodiment of the present invention includes the following steps:

[0183] Step S1: Align the boundary of the wheat 3D point cloud with the 3D coordinate axis. Specifically: Use the rotation matrix to rotate and translate the pre-processed wheat 3D point cloud data so that the boundary of the wheat field is aligned with the coordinate axis. The rotation matrix calculation formula (taking rotation around the X axis as an example) is:

[0184]

[0185] Step S2: Select the top wheat 3D point cloud. Specifically: the 3D model of the wheat after rotation and translation x 、 y 、 z The coordinates can be approximately started from 0, where z The coordinates can be regarded as the height of the point. Height_ratio As the height threshold parameter, 15% can be taken, and the highest point in the wheat 3D point cloud is and Height_ratio The product of is used as the threshold height.

[0186]

[0187] Select the points whose z coordinate is greater than the threshold height. Considering the limitations of the 3D point cloud, there will be noise points at the top of the wheat 3D point cloud. At this time, the top points can be further removed to reduce the top ratio. Height_ratio data.

[0188] Step S3: Calculate the average height of the points that meet the conditions. Specifically, calculate the average z coordinate of the remaining points and record it as the average height of the wheat canopy. :

[0189]

[0190] in, n is the number of remaining points, The remaining points i points z coordinate, The maximum Z coordinate of the wheat point cloud after aligning the coordinate axes.

[0191] Example 5: See Figure 5 and Figure 6 , an embodiment of the present invention provides a method for extracting multi-level and multi-angle wheat coverage of a wheat population based on a three-dimensional point cloud. In field crops, significant morphological and structural differences are shown in the vertical direction, forming a clear layered structure. Nitrogen and phosphorus elements have a significant promoting effect on the growth of wheat leaves, which can cause them to expand and thicken, so that the leaves become wider and thicker, and the degree of outward expansion also increases accordingly. The growth state of wheat crops in the field is roughly divided into three levels: the upper layer is the wheat ear layer, the middle layer is the wheat straw leaf layer, and the bottom layer is the tillering layer. The three-dimensional model of the wheat crop was layered to obtain a slice image of the wheat canopy. Subsequently, the coverage features of the wheat three-dimensional point cloud as a whole and at the layer level were extracted. Specifically:

[0192] Step S1, slice and layer the wheat three-dimensional point cloud. Since the growth of wheat crops in farmland varies, only the approximate range can be determined. First, perform an orthographic projection on the three-dimensional model of the wheat crop. After preprocessing, there is no ground information on the three-dimensional model of the wheat crop. Only tillers and their leaves are left at the bottom. The wheat tillers and their leaves extend radially outward, and the bottom cut is set at the highest point of the tiller leaves extending outward, which is approximately 25%-30%. The wheat straw and leaves also extend radially outward, and the upper cut is set at the highest point of the wheat straw and leaves extending outward, which is approximately 25%-30%. Specifically:

[0193] The average height of the wheat crop is obtained through the above-mentioned canopy height algorithm. The average height is segmented and the segmentation threshold is set. The upper segmentation point is set at the top 30%, and the bottom segmentation point is set at 30%. The upper segmentation threshold is the average height multiplied by 70%, and the lower segmentation threshold is the average height multiplied by 30%. The z coordinate array Z_coords of the wheat crop three-dimensional model is sorted in descending order. If the z coordinate is greater than or equal to the upper segmentation threshold, the z coordinate index is moved to the index array top. If it is less than or equal to the lower segmentation threshold, the z coordinate index is moved to the index array bottom. At this time, the three-dimensional wheat crop model data corresponding to the top, Z_coords, and bottom arrays are selected and saved to obtain the three-layer canopy slice of the wheat crop.

[0194] Step S2: Project the wheat 3D point cloud from various angles. During the generation of the wheat group 3D point cloud, the details in the middle layer will be blurred, and the upper layer will produce some difficult-to-remove noise because the wheat ears cannot be fixed during data collection. The point cloud of the lower tiller layer has the best effect. Specifically:

[0195] Step S201: Map the whole wheat 3D model and the lower tiller layer slices to planes at various angles. θ (the angle formed with the z=0 plane) for projection. Define the found vector of the tilted plane n :

[0196]

[0197]

[0198] in, r is the angle θ The radian value.

[0199] The general form of the projection plane is:

[0200]

[0201] in, A =sin(r), B =0, C =cos(r), D is a constant.

[0202] Select Origin P =[0,0,0] calculation D :

[0203] D =-( n · P )

[0204] Therefore, the projection plane at this angle is:

[0205]

[0206] Step S202: any point in the point cloud P =[x,y,z], calculate its vertical distance to the plane d :

[0207]

[0208] Calculate the projection point:

[0209]

[0210] After expansion:

[0211]

[0212] Project all point cloud data onto the plane and get θ Projected point cloud under angle.

[0213] Step S203: Draw a scatter plot of the wheat projection point cloud, set the scatter point size S (so that the scatter points touch each other to form a surface), color (such as blue), and transparency, etc., convert the drawn RGB image to HSV color space, and define the color range. lower_color =[100,50,50], upper_color =[130,255,255], (the color range here is set according to the color of the scatter plot) to create a binary mask:

[0214]

[0215] Count the number of non-zero pixels in the mask:

[0216]

[0217] Step S3: Draw and extract coverage of the scattered points after mapping projection, specifically: calculate the total area of the sample field Total_area Canopy cover (calculated using bounding box):

[0218] .

[0219] This method can be used to extract the canopy coverage characteristics of wheat three-dimensional point clouds at various angles and levels.

[0220] Example 6: See Figure 7, an embodiment of the present invention provides a method for extracting the density of the bottom tiller layer of a wheat population based on a three-dimensional point cloud. In the application of the algorithm for extracting the canopy cover of wheat crops, the process generates a series of mapping images based on different angles. It was found that under different nitrogen and phosphorus content conditions, there are significant differences in the tillering conditions and leaf density between rows of wheat crops. The method of this embodiment is specifically as follows:

[0221] Step S1: Set the baseline in the wheat point cloud orthographic mapping image and move it to the root position of the wheat. Use the scatter plot generated by the 3D point cloud orthographic projection to scan from the bottom of the image line by line, and find the first line containing a specific color (the color is the color set when drawing the scatter plot), and record it as the root position. , move the offset along the tillering direction (vertically upward) , so that the baseline Move to the middle of the tiller:

[0222]

[0223] in, is the image height, Typically set to 0.15.

[0224] Step S2: Detecting the tillering area and calculating the density; specifically:

[0225] Step S201: On the baseline Detect all continuous pixel segments and record the starting column of each segment and the termination column .

[0226] Step S202: Calculate the single-end distance, total coverage length, and total baseline length and calculate the result:

[0227] Single segment distance:

[0228]

[0229] Total coverage length:

[0230]

[0231] Total length of baseline:

[0232]

[0233] The bottom density is the ratio of the tillering area coverage length to the total baseline length:

[0234] .

[0235] Example 7: See Figure 8 、 Figure 9, the present invention provides a method for extracting the volume ratio features of each layer of a wheat population based on a three-dimensional point cloud. When wheat crops are faced with different nitrogen and phosphorus content environments, they will show obvious growth differences. Specifically, the size, length and extension of the leaves, as well as the thickness of the stems will be different. This growth difference directly leads to the fact that the occupied range of each wheat individual in space is also different. Generally, the higher the nitrogen and phosphorus content, the more luxuriant the wheat grows, and the larger the space it occupies, that is, the overall volume is more significant. The volume ratio of the tillering layer can be extracted to judge the nutritional characteristics of wheat. The method of this embodiment is specifically:

[0236] Step 1: Use the Quickhull algorithm to construct the convex hull. Find the point with the minimum x coordinate in the set. and maximum The two points of are used as the initial edges of the convex hull. And through the straight line Divide the remaining points into two groups: the set of points above the line , the set of points below the line .

[0237] For each set of points, find the point farthest from the line Q (Calculate distance by vector cross product) Q Define a triangular patch and divide the point set into left and right subsets. Recursively process the subsets until all points are included in the convex hull.

[0238] The formula for the distance from a point to a straight line is:

[0239]

[0240] :point Q To the end point of the line P vector of : Line direction vector; d :point Q The perpendicular distance to the line.

[0241] Step 2: Use tetrahedron decomposition method to calculate the overall canopy volume and tiller layer volume of wheat and calculate the tiller layer volume ratio. D As a common base point. For each triangle patch (vertices are A 、 B 、 C ) and base points D Tetrahedron ABCD , calculate the volume of each tetrahedron.

[0242]

[0243] , , :From the base point D Vectors to other vertices.

[0244] : vector cross product, : vector point set.

[0245] Cumulative sum over all tetrahedra:

[0246]

[0247] M is the number of tetrahedrons.

[0248] The ratio of the tillering layer volume to the total wheat canopy volume is taken as the tillering layer volume ratio:

[0249]

[0250] in, is the tillering layer volume, is the overall canopy volume of wheat.

[0251] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-dimensional feature extraction method for wheat populations based on three-dimensional point cloud and image processing, characterized in that: The extraction method comprises the following steps: Step S1: Acquire multi-view video data of the target wheat population; Step S2: performing frame processing on the multi-view video data and generating three-dimensional point cloud data of the wheat population using NeRF technology; Step S3: pre-processing the three-dimensional point cloud data by voxel downsampling, ground point segmentation and outlier removal methods; Step S4: extracting the average height feature of the wheat population based on the preprocessed three-dimensional point cloud data; Step S5: spatially stratifying the pre-processed three-dimensional point cloud data to obtain tillering layer point cloud data; Step S6: Based on the tiller layer point cloud data, the canopy cover characteristics are calculated by multi-angle projection combined with the color mask method; Step S7: using the Quickhull algorithm to construct a three-dimensional convex hull model of the wheat as a whole and the tiller layer, and using the tetrahedron decomposition method to calculate the volume ratio characteristics of the tiller layer; Step S8: Based on the tiller layer baseline analysis, calculate the density characteristics at the tillering location.

2. The extraction method according to claim 1, wherein The step S1 obtains multi-perspective video data of the target wheat group. Specifically, three videos are shot around the target wheat group using a portable device, the first video focuses on the bottom area of the target wheat group, the second video focuses on the middle area of the target wheat group, and the third video focuses on the top area of the group at a 45° elevation angle.

3. The extraction method according to claim 2, characterized in that The step S2 specifically includes: extracting key frames from each video sequence, removing blurred frames, screening a set number of clear images from each spatial partition, and fusing multi-view image frames using NeRF technology to generate three-dimensional point cloud data of the wheat population.

4. The extraction method according to claim 1, wherein The pre-processing in step S3 is specifically as follows: Step S301: Calculating the boundary range of the three-dimensional point cloud data of the wheat population, dividing the point cloud data into a uniform cubic grid with a fixed side length, calculating the centroid of each point within each voxel as a representative point, and generating downsampled point cloud data; Step S302: Using the RANSAC algorithm, three sample points are randomly selected from the 3D point cloud data to construct a plane model. The Euclidean distance between the remaining points in the downsampled point cloud data and the plane model is calculated. Points with a distance less than a set threshold are selected as ground point sets and removed from the point cloud. Step S303: Statistical filtering is performed on the three-dimensional point cloud after removing the ground, and the average Euclidean distance and standard deviation in the neighborhood of each point are calculated. Outliers are marked and removed by setting a threshold value of the standard deviation multiple, and the pre-processed point cloud data is output.

5. The extraction method according to claim 1, wherein The average height feature of the wheat population is extracted in step S4, specifically: Step S401: Using a rotation matrix, the pre-processed point cloud data is rotated and translated so that the boundary of the wheat is aligned with the coordinate axis, and the z-axis coordinate is defined as the point cloud height; Step S402: setting a height ratio threshold, with the product of the z value of the highest point of the point cloud and the height ratio threshold being used as the threshold height; Step S403: filtering point cloud data with z coordinates greater than a threshold height, removing point clouds with a top ratio equal to the height ratio threshold, and outputting a valid height point cloud; Step S403: Calculate the average height characteristics of the wheat population based on the effective height point cloud.

6. The extraction method according to claim 1, characterized in that In step S5, the preprocessed three-dimensional point cloud data is spatially layered to obtain tiller layer point cloud data, specifically: the preprocessed point cloud data is divided into wheat ear layer, wheat straw leaf layer and tiller layer from top to bottom according to vertical height, and the point cloud data corresponding to each layer after layering is output.

7. The extraction method according to claim 1, characterized in that The method for calculating the canopy coverage characteristics in step S6 is specifically as follows: Step S601: Projecting the overall point cloud data and the tiller layer point cloud data onto a mapping plane of a specified angle to generate a projected point cloud; Step S602: draw a scatter plot of the projected point cloud, convert the scatter plot of the projected point cloud into the HSV color space, and calculate the number of non-zero pixels in the mask as the canopy coverage area; Step S603: Calculate the total projected area according to the circumscribed rectangle of the mapping plane, and calculate the ratio of the canopy coverage area to the total projected area to obtain the canopy coverage characteristics.

8. The extraction method according to claim 1, characterized in that The method for calculating the volume ratio characteristics of the tillering layer in step S7 is specifically as follows: Step S701: Based on the stratified tillering layer point cloud data and the pre-processed overall point cloud data, a three-dimensional convex hull model is constructed using the Quickhull algorithm; specifically, the point with the minimum x coordinate in the overall point cloud is selected. P min and maximum point P max , construct the initial segmentation line; Calculate the distance from each point to the initial segmentation line and select the point with the largest distance P for As a new convex hull vertex; recursively split the point set and repeat vertex screening until the overall convex hull and tiller layer convex hull construction are completed; Step S702: Calculate the volume of each convex hull and the total volume by tetrahedron decomposition method; specifically, select the base point from the vertices of the convex hull of the tiller layer D ; Traverse each triangle patch of the convex hull of the tiller layer ( A , B , C ), and with the base point D Tetrahedron ( A , B , C , D ), calculate the volume of each tetrahedron; add up the volumes of all tetrahedrons to get the total volume of the convex hull of the tillering layer; Calculate the total volume of the convex hull in the same way; Step S703: The ratio of the total volume of the tillering layer convex hull to the total volume of the entire convex hull is used as the tillering layer volume ratio feature.

9. The extraction method according to claim 1, characterized in that The method for calculating the density characteristics at the tillering position in step S8 is specifically as follows: Step S801: generating an orthographic projection scatter plot based on the layered tiller layer point cloud data; Step S802: Scan row by row along the bottom of the scatter plot and locate the first row containing the preset color value as the root reference line , set the offset and adjust the value according to the offset Adjust the baseline to the middle area of the tillering layer to obtain the position of the tillering layer baseline. The calculation formula is: in, is the height of the scatter plot, Usually set to 0.15; Step S803: At the tillering layer baseline Detect all continuous pixel segments and record the starting column of each segment and the termination column , and calculate the single-segment coverage length, total coverage length and total length of the tillering layer baseline; Step S804: Calculate the ratio of the length of the continuous plant area on the baseline to the total length of the baseline, and output the density feature of the tillering.

Images