Crop height extraction method and system based on density consistency denoising

Abstract

The invention discloses a crop height extraction method and system based on density consistency denoising, and the method comprises the steps: removing noise in point cloud data through employing a density consistency denoising method; then, ground points are extracted by applying an over-green index EXG and a random sampling consistency algorithm. And finally, performing linear interpolation based on the extracted ground points to fit missing ground points so as to estimate the winter wheat height. According to the method, the influence of point cloud data noise and ground point missing on height estimation can be reduced. According to the complete height estimation method, the root mean square error RMSE and the mean absolute error MAE are 7.81 cm and 6.18 cm respectively, the correlation coefficient R2 of the estimated plant height and the actual result is 0.904, and the significant level is achieved. The result shows that the method can effectively improve the winter wheat height estimation precision. And after the winter wheat enters the stem extension period, the noise and ground point loss condition is more serious, so that the improvement effect is more obvious.

Description

Technical Field

[0001] The invention relates to the technical field of crop recognition based on image processing, and in particular to a crop height extraction method and system based on density consistency denoising. Background Art

[0002] Crop height is an important indicator for monitoring crop growth, predicting yield, and estimating biomass. Accurate estimation of crop height is crucial to optimizing agricultural production decisions and can provide a scientific basis for management measures such as fertilization, irrigation, and harvesting. Wheat is one of the most widely distributed and largest grain crops in the world. Therefore, it is of great significance to obtain wheat height over a large area.

[0003] Traditionally, crop height is mainly obtained through field measurement, but this method is not only time-consuming and labor-intensive, but also the samples obtained are sparse and only applicable to small areas. Therefore, this method is not suitable for obtaining crop height over a large area. The rapid development of UAV technology has provided an efficient and economical way to obtain crop height information on a large scale. By using UAVs equipped with LiDAR for flight observation, crop height data of large areas of farmland can be obtained with high timeliness and high resolution. In recent years, researchers have begun to focus on methods for generating high-precision crop point cloud data models based on optical images of UAV platforms and Structure from Motion (SfM) algorithms. For example, Turner et al. introduced an automated process based on the SfM algorithm to generate geo-corrected digital terrain models from UAV images. Height is an important factor in monitoring the growth of winter wheat and guiding agricultural production management. Compared with traditional field measurements and high-cost LiDAR systems, point cloud data generated by the Structure from Motion (SfM) algorithm based on UAV images can quickly estimate the height of winter wheat in the target area at a low cost. However, as winter wheat grows, the swing amplitude of its leaves increases, resulting in SfM matching errors, which makes the generated point cloud data contain a lot of noise. In addition, starting from the stem extension period, the leaves of winter wheat gradually grow luxuriantly, interlacing and blocking the ground, resulting in missing ground points below the canopy point cloud data. Noise and missing ground points will seriously affect the accuracy of height estimation.

[0004] Bendig et al. used drones to obtain multi-temporal, high-resolution crop surface models for monitoring crop growth differences. Compared with expensive Lidar systems, using drones equipped with ordinary digital cameras combined with SfM algorithms to generate point cloud data has more obvious advantages. It is usually achieved using standard digital cameras and consumer drones, which are inexpensive; and digital cameras are light and low in power consumption, so they can extend the drone's flight time and improve its maneuverability. The required hardware and operating steps are relatively simple, making the SfM method fast to deploy and flexible to apply. These advantages are particularly suitable for scenarios with limited budgets and the need for rapid response.

[0005] Due to image defects, complex acquisition environment, matching uncertainty, non-diffuse surface and UAV jitter, the generated point cloud data often contains a lot of noise. These noises will greatly reduce the accuracy of crop height estimation. Therefore, before crop height estimation, these outliers need to be removed to obtain effective information. Some studies have attempted to apply the outlier removal method of LiDAR point cloud data to UAV-based point cloud data. Since UAV-based point cloud data cannot penetrate the dense vegetation canopy, LiDAR filtering methods may not be suitable for removing noise in UAV-based point cloud data. Khanna et al. removed the top 1% of the point cloud data as outliers to estimate crop height. Shin et al. used a fixed height threshold (4m) to clean up outliers at the top of the forest canopy. Chang et al. deleted areas with calculated crop heights greater than 4m or less than 0.05m as outliers. Song et al. proposed a moving cuboid filter to remove noise points according to the spatial distribution of UAV point cloud data, but when its judgment threshold exceeds a certain level, the misjudgment of normal point cloud data will suddenly become serious. Sun et al. used statistical outlier removal (SOR) to remove noise from point cloud data before estimating height, but this denoising method can only remove isolated outliers, and has limited effect on removing abnormal point cloud data clusters. In addition, the missing ground points caused by the interlaced occlusion of leaves in the vegetation canopy will also cause significant errors in the estimation of crop height. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a crop height extraction method based on density consistency denoising, which utilizes density consistency denoising to improve the subsequent data processing effect.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The crop height extraction method based on density consistency denoising provided by the present invention comprises the following steps:

[0009] S1: Determine the study area and obtain ground point cloud data;

[0010] S2: Apply density consistency denoising method to remove noise from point cloud data;

[0011] S3: Apply the overgreen index EXG and random sampling consistency algorithm to extract ground points from point cloud data;

[0012] S4: Fitting missing ground points based on the extracted ground points by linear interpolation to estimate crop height.

[0013] Further, the acquisition of the ground point cloud data is carried out according to the following steps:

[0014] Set the time and location of plant growth data collection according to the geographical location of the study area;

[0015] Set the data collection format;

[0016] Collect and store point cloud data.

[0017] Furthermore, the density consistency denoising method in step S2 specifically includes the following steps:

[0018] S21: Neighborhood statistics: For each point in the point cloud data, set an ellipsoidal neighborhood with a change in the major and minor semi-axis of the point; record the number of other point cloud data in each neighborhood in turn to obtain a set of data points (x i ,y i ),i=1,2,...,20, where x i is the length of the major semi-axis of the ellipsoid, y i is the number of point cloud data within the ellipsoid;

[0019] S22: Polynomial fitting: Apply the least squares method to fit the obtained data to a second-order polynomial y=ax 2 +bx+c, its least squares solution is:

[0020]

[0021] in,

[0022] The second-order polynomial was chosen because it can better capture the trend of the number of domain points changing with radius;

[0023] S23: Outlier judgment: Determine whether a point is an outlier by analyzing the quadratic term coefficient of the second-order polynomial. The judgment is made based on the quadratic term coefficient a. When a is less than a preset threshold, the point is considered to be noise, otherwise it is considered to be a normal point.

[0024] Further, in step S3, ground points in the point cloud data are extracted according to the following steps;

[0025] S31: Calculate the overgreen index EXG of each point cloud data;

[0026] S32: extracting data points whose over-green index EXG is less than a threshold;

[0027] S33: applying the RANSAC algorithm to fit the plane based on the extracted point cloud data;

[0028] S34: Remove data points whose distance from the plane is greater than a preset threshold to obtain ground point cloud data.

[0029] Furthermore, in step S4, the missing ground points are fitted by linear interpolation based on the extracted ground points to estimate the crop height, and the specific steps are as follows:

[0030] S41: Divide the entire xOy plane into a number of rectangular grids;

[0031] S42: taking the lowest point among the extracted ground points as the true ground point of each grid;

[0032] S43: performing linear interpolation on the grid that does not contain the ground point, and using the interpolation result as the real ground point of the grid;

[0033] S44: Subtract the highest point of the canopy in each rectangular grid from the actual ground point to obtain the crop height in the grid.

[0034] Furthermore, the overgreen index EXG is calculated according to the following formula:

[0035] EXG=2G–BR

[0036] in, r, g, and b are the grayscale values ​​of the red, green, and blue bands in the RGB image, respectively.

[0037] Furthermore, the preset threshold adopts a fixed threshold.

[0038] Furthermore, the abnormal value judgment in step S23 is performed by judging the quadratic term coefficient a. When a is less than a preset threshold, the point is considered to be noise, otherwise it is considered to be a normal point.

[0039] Furthermore, the point cloud data in step S1 is generated by a structure from motion (SfM) algorithm.

[0040] The crop height extraction system based on density consistency denoising provided by the present invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the above method is implemented when the processor executes the program.

[0041] The beneficial effects of the present invention are:

[0042] The crop height extraction method based on density consistency denoising provided by the present invention first applies the density consistency denoising method to remove the noise in the point cloud data. Then, the overgreen index EXG and the random sampling consistency algorithm are applied to extract ground points. Finally, linear interpolation is performed based on the extracted ground points to fit the missing ground points to estimate the height of winter wheat. This method can reduce the influence of point cloud data noise and missing ground points on height estimation. The root mean square error RMSE and mean absolute error MAE of the complete height estimation method of this method are 7.81cm and 6.18cm respectively, and the correlation coefficient R2 between the estimated plant height and the actual result is 0.904, which reaches a significant level. The results show that this method can effectively improve the accuracy of winter wheat height estimation. After winter wheat enters the stem extension period, the noise and missing ground points are more serious, making this improvement effect more obvious.

[0043] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for explanation.

[0045] Figure 1 Flowchart of the crop height extraction method based on density consistency denoising.

[0046] Figure 2 is the location of the study area and sampling points.

[0047] Figure 3 Aerial view of winter wheat point cloud data for six dates.

[0048] Figure 4 Flowchart of the density consistency denoising method.

[0049] Figure 5 Schematic diagram of the denoising process for different types of point cloud data.

[0050] Figure 6 Schematic diagram of missing ground points.

[0051] Figure 7 These are the winter wheat height estimation results using four different methods.

[0052] Figure 8 These are actual photos of winter wheat at three different growth stages.

[0053] Fig. 9Estimation errors of winter wheat height under different parameter combinations on six dates.

[0054] Fig.10 Cluster the height differences and their corresponding optimal axis ratios for the six dates.

[0055] Fig.11 Schematic diagram of the denoising results using different methods. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0057] Example 1

[0058] like Figure 1 As shown, the crop height extraction method based on density consistency denoising provided in this embodiment includes the following steps:

[0059] S1: Determine the study area and obtain ground point cloud data;

[0060] S2: Apply density consistency denoising method to remove noise from point cloud data;

[0061] S3: Apply the overgreen index EXG and random sampling consistency algorithm to extract ground points from point cloud data;

[0062] S4: Fitting missing ground points based on the extracted ground points by linear interpolation to estimate the crop (winter wheat) height.

[0063] The study area and ground data collection of this example are as follows:

[0064] The study area is a wheat-growing region near Melbourne in southwestern Ontario, Canada. Figure 2 As shown, Figure 2 (a) is the research area in Ontario, Canada. The growth cycle of winter wheat is usually about 220-270 days. The sowing time of winter wheat is from September to November each year, and the harvesting time is from the end of May to the beginning of June of the following year. Due to different climatic conditions, the sowing and harvesting time of winter wheat will also vary. Winter wheat in the research area is sown from October and matures in July of the following year.

[0065] 32 sampling points were set up along the rows of winter wheat in the study area, and a total of 6 field measurements were conducted from May to June 2019. Three winter wheat plants were randomly selected within 2 meters of the sampling point for height measurement, and their average height was used as the actual height of the winter wheat at the sampling point. A total of 192 measured data were collected. The spatial distance between any two sampling points is not less than 30 meters. The location of each sampling point is as follows: Figure 2 As shown, Figure 2 is the location of the study area and sampling points, Figure 2 As shown in (b), the locations of sampling points and control points in the study area are shown. In addition, to ensure the accuracy of the relative positions between data sets, 12 sampling points were selected as ground control points, and black and white chessboards with a size of 1 foot × 1 foot were set for subsequent data alignment.

[0066] In this embodiment, data is collected by drones, and the data collection and preprocessing are as follows:

[0067] This study used a DJI Phantom 4 drone system equipped with a 5K high-resolution digital camera to obtain drone images of winter wheat on six dates: May 2, May 11, May 16, May 21, May 27, June 3, and June 11, 2019. The drone flew at an altitude of 30 meters, with an image overlap rate of 90% and a resolution of 9 mm.

[0068] The point cloud data is generated from a series of RGB images using the Structure from Motion (SfM) algorithm.

[0069] The generated point cloud data is similar to LiDAR data, containing geometric information and RGB information of each point, such as Figure 3 As shown, Figure 3 This is an overhead view of winter wheat point cloud data for six dates. Figure 3 The winter wheat point cloud data and measured information of these six days are shown in Table 1.

[0070] Table 1 Point cloud data and measured information of winter wheat on six dates

[0071]

[0072] Due to factors such as environmental changes and matching uncertainty, the point cloud data generated by the SfM method contains a lot of noise, including isolated noise points and noise point clusters composed of a large number of closely adjacent noise points. These noises will significantly reduce the accuracy of crop height estimation and need to be eliminated to improve the estimation accuracy. Conventional point cloud data denoising methods such as statistical filtering and radius filtering can effectively remove isolated noise in point cloud data, but the effect of removing noise point clusters is limited.

[0073] The density consistency denoising method provided in this embodiment is based on the density of point cloud data, and can effectively remove isolated noise and most noise point clusters. The basic principle is: for normal point cloud data, its density remains almost unchanged as the radius increases; for isolated outliers, its density decreases rapidly as the radius increases; for abnormal point clusters, its density gradually decreases when the radius increases to a certain extent. By analyzing the relationship between the change in the number of points in the neighborhood of point cloud data and the radius, normal point cloud data and noise point cloud data can be identified.

[0074] like Figure 4 As shown, Figure 4 This is a flow chart of a density consistency denoising method. The density consistency denoising method provided in this embodiment specifically includes the following steps:

[0075] S21: Neighborhood statistics: For each point in the point cloud data, set an ellipsoidal neighborhood with the point as the center, a step length of 5 cm, and a major semi-axis that gradually increases from 5 cm to 100 cm. The ratio of the major and minor semi-axis lengths of the ellipsoid is 10:1; record the number of other point cloud data in each neighborhood in turn, and get a set of data points (x i ,y i ),i=1,2,...,20, where x i is the length of the major semi-axis of the ellipsoid, y i is the number of point cloud data within the ellipsoid;

[0076] S22: Polynomial fitting: Apply the least squares method to fit the obtained data to a second-order polynomial y=ax 2 +bx+c. Its least squares solution is:

[0077]

[0078] in,

[0079] The second-order polynomial was chosen because it can better capture the trend of the number of domain points changing with radius;

[0080] S23: Outlier judgment: Determine whether a point is an outlier by analyzing the quadratic term coefficient of the second-order polynomial. Usually, the neighborhood density of noise points varies greatly, causing the quadratic term coefficient of the fitting curve to deviate from the normal value. Therefore, the quadratic term coefficient a can effectively determine whether the point is noise; when a is less than a preset threshold, the point is considered to be noise, otherwise it is considered to be a normal point, and the preset threshold is set to T.

[0081] This method can effectively remove isolated outliers and most abnormal point cloud data clusters in the point cloud data while retaining the geometric features, greatly improving the accuracy of subsequent winter wheat height estimation. Figure 5 As shown, Figure 5 Schematic diagram of the denoising process for different types of point cloud data. Figure 5 (a) The denoising process of normal point cloud data, (b) the denoising process of isolated abnormal point cloud data, and (c) the denoising process of abnormal point cloud data clusters.

[0082] This embodiment extracts ground points from point cloud data according to the following steps. The specific steps are as follows:

[0083] S31: Calculate the overgreen index EXG of each point cloud data;

[0084] S32: extracting points where the over-green index EXG is less than a threshold;

[0085] S33: applying the RANSAC algorithm to fit the plane based on the extracted point cloud data;

[0086] S34: removing data points whose distance from the plane is greater than a preset threshold, to obtain ground point cloud data;

[0087] In this embodiment, the preset threshold value can be set to 6 to 10 cm, and can be specifically selected to be 8 cm;

[0088] Vegetation index is a simple, effective and empirical measure of the status of surface vegetation, and is often used to segment vegetation and soil in RGB images. Commonly used vegetation indices include the overgreen index (EXG), overred index (EXR), and normalized green-red difference index (NGRDI). Among them, the overgreen index is considered to be the best vegetation index for segmenting green vegetation and soil in RGB images. Therefore, the overgreen index is used to separate winter wheat canopy point cloud data and ground point cloud data.

[0089] The calculation formula of the green index is as follows:

[0090] EXG=2G–BR

[0091] in, r, g, and b are the grayscale values ​​of the red, green, and blue bands in the RGB image, respectively.

[0092] After calculating the overgreen index EXG of each point based on the color information of the point cloud data, the winter wheat vegetation point cloud data and the ground point cloud data can be separated by selecting a suitable threshold. The Otsu threshold method is often used to automatically select the threshold, which achieves the best binary segmentation effect by selecting the threshold that minimizes the intra-class variance. However, this method may produce poor segmentation results for binary segmentation targets that do not have a bimodal histogram.

[0093] The EXG calculated in this embodiment is mostly a single-peak histogram, which is not suitable for the Otsu method. In addition, due to factors such as shadow effects, weeds and light, the overgreen index EXG calculated for some canopy point cloud data may be too low, resulting in misclassification as ground point cloud data. For this part of the point cloud data, only using the color information of the point cloud data cannot completely extract the ground points, and the canopy point cloud data needs to be further filtered out based on the spatial structure information of the point cloud data.

[0094] Therefore, this embodiment uses the excessive green index EXG combined with a fixed threshold to perform preliminary extraction of ground points.

[0095] In this embodiment, a fixed threshold is set by selecting a point that can extract ground points to the greatest extent without including too many vegetation points. In this embodiment, 0.14 is used.

[0096] Then the Random Sample Consensus (RANSAC) algorithm is used to further filter out the canopy point cloud data.

[0097] The RANSAC algorithm is an iterative algorithm widely used in computer vision, image processing, and machine learning to estimate the parameters of mathematical models, especially when there is a lot of noise or outliers in the data. The main goal of RANSAC is to find an optimal set of model parameters from a data set containing outliers so that the model can maximally conform to the inliers.

[0098] This embodiment adopts linear interpolation to fit the missing ground points, and uses the interpolation results as the real ground points to estimate the height of wheat. The specific steps are as follows:

[0099] S41: Divide the entire xOy plane into a number of rectangular grids of 0.5 m × 0.5 m;

[0100] S42: taking the lowest point among the extracted ground points as the true ground point of each grid;

[0101] S43: performing linear interpolation on the grid that does not contain the ground point, and using the interpolation result as the real ground point of the grid;

[0102] S44: Subtract the highest point of the canopy in each grid from the actual ground point to obtain the crop height in the grid.

[0103] The rectangular grid in this embodiment may be a 0.5m×0.5m grid; the crop is winter wheat;

[0104] Estimation of winter wheat height at sampling point: Taking the sampling point as the center, select the five grids with the highest estimated winter wheat height in the 2m×2m area around it, and calculate the average value of these grids as the estimated winter wheat height of the sampling point. Selecting the highest five grids to estimate the height of the sampling point can avoid selecting grids where winter wheat is not planted when calculating the height, reducing the underestimation of the winter wheat height.

[0105] Starting from the stem extension stage, the overlap of winter wheat leaves gradually increases, resulting in the lack of ground part in the images taken by the drone, which in turn leads to the lack of ground point cloud data below some canopy point cloud data in the point cloud data generated by the SfM algorithm, such as Figure 6 As shown, Figure 6 This is a schematic diagram of missing ground points. Figure 6 (a) Booting stage (BBCH = 49) (b) Heading stage (BBCH = 59). The missing ground points will cause the winter wheat height in this area to be seriously underestimated. Therefore, the area with missing ground points must be processed to reduce the impact of missing ground points on the winter wheat height estimation.

[0106] The effect evaluation of the method provided in this embodiment is as follows:

[0107] In order to analyze the impact of point cloud data denoising and fitting missing ground points on the accuracy of winter wheat height estimation, this study added the following two additional height estimation methods:

[0108] (I) The original point cloud data is directly used to estimate the height of winter wheat without processing the point cloud data;

[0109] (II) Estimation of winter wheat height after denoising using only density consistency.

[0110] In addition, to compare the effectiveness of the proposed height estimation method, this study compared it with another winter wheat height estimation method (Khanna) which was also based on UAV point cloud data.

[0111] Khanna's method first divides the point cloud dataset into many columns, which are 3D grid cells with the same area. Secondly, the threshold is determined using the overgreen index EXG combined with the Otus method to classify the points into ground and vegetation parts. Then, the least squares method is used for the ground points in each grid to perform a second-order two-dimensional polynomial regression to fit the ground. The authors removed the top 1% of the vegetation points as outliers, and the final winter wheat height was calculated based on the highest point of the remaining vegetation points in the grid minus the corresponding height value in the fitted surface.

[0112] Six sets of measured data from 32 stations were used to evaluate the accuracy of winter wheat height estimation. Linear regression analysis was performed between the estimated height obtained by each method and the measured height, and the corresponding determination coefficient (R2), root mean square error (RMSE) and mean absolute error (MAE) were calculated to evaluate the height estimation performance of each method.

[0113] The results are as follows:

[0114] Four different winter wheat height estimation methods were compared:

[0115] (I) The original point cloud data is directly used to estimate the height of winter wheat without processing the point cloud data;

[0116] (II) Estimation of winter wheat height after denoising using density consistency;

[0117] (III) The complete height estimation method proposed in this embodiment;

[0118] (IV) Khanna's height estimation method;

[0119] The height estimation results of the four methods are shown in Figure 7 As shown in Tables 2 to 4, Figure 7 The winter wheat height estimation results of four different methods are shown in Figure 2. Figure 7 (a) directly using the original point cloud data for height estimation; (b) applying density consistency denoising to perform height estimation; (c) the complete height estimation method proposed in this embodiment; (d) Khanna's height estimation method;

[0120] Table 2 Average height of height estimation results of four different methods (cm)

[0121]

[0122]

[0123] Table 3 Root mean square error RMSE (cm) of height estimation results of four different methods

[0124]

[0125] Table 4 Mean absolute error (MAE) of height estimation results of four different methods (cm)

[0126]

[0127]

[0128] For data on May 11:

[0129] The RMSE of method I is 10.8 cm, the MAE is 10.3 cm, and the average height is 10.7 cm, which is 49.5% different from the measured average of 21.2;

[0130] The RMSE of method II is 10.7 cm, the MAE is 10.2 cm, and the average height is 10.7 cm, which is 49.7% different from the measured average;

[0131] The RMSE of method III is 10.4 cm, the MAE is 9.8 cm, and the average height is 11.0 cm, which is 58.0% different from the measured average;

[0132] The RMSE of method IV is 13.2 cm, the MAE is 12.8 cm, and the average height is 7.9 cm, which is 62.7% different from the measured average.

[0133] For data on May 16:

[0134] The RMSE of method I is 11.3 cm, the MAE is 10.5 cm, and the average height is 18.3 cm, which is 38.0% different from the measured average of 29.5;

[0135] The RMSE of method II is 12.7 cm, the MAE is 12.3 cm, and the average height is 16.7 cm, which is 43.3% different from the measured average;

[0136] The RMSE of method III is 12.7 cm, the MAE is 12.4 cm, and the average height is 16.7 cm, which is 43.3% different from the measured average;

[0137] The RMSE of method IV is 14.9 cm, the MAE is 14.1 cm, and the average height is 14.4 cm, which is 51.2% different from the measured average.

[0138] As shown in Tables 2 to 4. At the tillering stage, the four different winter wheat height estimation methods all showed significant underestimation, such as Figure 7As shown in the figure. Theoretically, the wheat canopy in the tillering stage is relatively sparse, and the height estimation will not be affected by the missing ground points. And because of the existence of abnormal point cloud data, and because method I directly subtracts the maximum and minimum values ​​in the divided grid, the height estimation result obtained should be higher than the actual height. However, the height estimation result obtained by method I is significantly lower than the actual height. After analysis, this situation occurs because the leaf tips of winter wheat in the tillering stage are relatively slender, and there are not enough detectable feature points when using SfM to generate three-dimensional point cloud data, making it difficult for SfM to obtain enough data for accurate reconstruction during the image matching process.

[0139] Therefore, the tip of the winter wheat leaf is missing in the generated point cloud data, which ultimately leads to an underestimation of the height estimation.

[0140] For data on May 21:

[0141] The RMSE of method I is 17.2 cm, the MAE is 10.3 cm, and the average height is 36.5 cm, which is 24.1% different from the measured average of 29.4 cm;

[0142] The RMSE of method II is 5.4 cm, the MAE is 4.4 cm, and the average height is 28.7 cm, which is 2.4% different from the measured average;

[0143] Method III has an RMSE of 4.8 cm, a MAE of 3.7 cm, and an average height of 27.3 cm, which is 7.3% different from the measured average;

[0144] Method IV has an RMSE of 13.6 cm, a MAE of 7.0 cm, and an average height of 28.8 cm, which is 62.7% different from the measured average;

[0145] As shown in Table 2 to Table 4. It can be seen that the effects of methods II and III are better than those of methods I and IV.

[0146] like Figure 8 As shown, Figure 8 These are actual photos of winter wheat at three different growth stages. Figure 8It can be found that the height estimation on this day slightly underestimated some sampling points. This is because the wind was very strong when data was collected on May 21, and the leaves on the top of the crop appeared to be lodged. Therefore, the estimated height will be lower than the measured height. At the same time, the lodging of the leaves on the top of the crop is equivalent to expanding the area of ​​the leaves on the top of the crop, so that the top of the crop has enough detectable feature points when using SfM to generate three-dimensional point cloud data, thereby improving the quality of the point cloud data on the top of the crop and avoiding the serious underestimation of crop height that occurred in the previous two days. However, changes in wind speed and direction may cause the same wheat plant to have inconsistent postures in different images, resulting in matching errors when the SfM algorithm generates three-dimensional point cloud data, causing the generated point cloud data to contain a lot of noise. Figure 8 It can be seen that compared with the point cloud data of the previous two days, the point cloud data on May 21 contains a lot of noise. Density consistency denoising can effectively reduce the impact of these noises on height estimation, while Khanna's percentile denoising method is not effective. Figure 8 (a) May 11 (tillering stage); (b) May 21 (stem extension stage); (c) June 11 (heading stage);

[0147] For data on May 27:

[0148] The RMSE of method I is 9.8 cm, the MAE is 6.7 cm, and the average height is 47.2 cm, which is 10.8% different from the measured average of 42.6 cm;

[0149] The RMSE of method II is 4.8 cm, the MAE is 3.0 cm, and the average height is 41.7 cm, which is 2.1% different from the measured average;

[0150] Method III has an RMSE of 3.2 cm, a MAE of 2.4 cm, and an average height of 43.1 cm, which is 1.1% different from the measured average;

[0151] The RMSE of method IV is 10.9 cm, the MAE is 9.6 cm, and the average height is 32.9 cm, which is 22.8% different from the measured average.

[0152] As winter wheat grows, the area of ​​its top leaves gradually increases. The point cloud data generated by SfM can avoid the loss of leaf tip point cloud data due to the lack of sufficient detectable feature points, and the density of winter wheat during this period has not reached the level of blocking the ground. Therefore, the generated point cloud data is of good quality, and there is almost no missing ground point. Except for individual sampling points, methods I, II, and III can all achieve good height estimation results. Khanna's method IV has an overall underestimation phenomenon. This is because the use of the overgreening index EXG combined with Otsu's method cannot well separate the ground and vegetation point cloud data. When using the least squares method to perform a second-order two-dimensional polynomial regression to fit the ground, a large amount of vegetation point cloud data is mixed in the fitting data, which makes the fitted ground height higher, resulting in an overall underestimation of the winter wheat height. At some sampling points, the dense leaves blocked the ground, resulting in the absence of ground points in the generated point cloud data, which caused a serious underestimation of the winter wheat height. Figure 8 It can be seen that method III solves the problem well.

[0153] For data on June 3:

[0154] The RMSE of method I is 13.7 cm, the MAE is 11.7 cm, and the average height is 59.4 cm, which is 10.7% different from the measured average of 53.7 cm;

[0155] The RMSE of method II is 13.0 cm, the MAE is 8.8 cm, and the average height is 46.8 cm, which is 12.9% different from the measured average;

[0156] Method III has an RMSE of 4.2 cm, a MAE of 3.4 cm, and an average height of 53.7 cm, which is 0.0% different from the measured average;

[0157] The RMSE of method IV is 19.7 cm, the MAE is 14.8 cm, and the average height is 38.3 cm, which is 28.7% different from the measured average.

[0158] For data on June 11:

[0159] The RMSE of method I is 18.2 cm, the MAE is 15.3 cm, and the average height is 72.6 cm, which is 15.7% different from the measured average of 62.8 cm;

[0160] The RMSE of method II is 16.18 cm, the MAE is 12.3 cm, and the average height is 53.0 cm, which is 15.6% different from the measured average;

[0161] Method III has an RMSE of 6.8 cm, a MAE of 5.5 cm, and an average height of 62.1 cm, which is 1.1% different from the measured average;

[0162] The RMSE of method IV is 17.79 cm, the MAE is 14.3 cm, and the average height is 51.6 cm, which is 17.8% different from the measured average.

[0163] The height estimation results of these two days were basically consistent with those based on the experimental principles. Methods I and IV had serious overestimation and underestimation in their height estimation results due to denoising problems and failure to fit missing ground points. Method II failed to fit missing ground points, resulting in serious underestimation of the winter wheat height at some sampling points.

[0164] The winter wheat height estimation method proposed in this embodiment has been completely processed, which greatly reduces the impact of point cloud data noise and missing ground points. However, due to the quality of point cloud data and the fitting accuracy of missing ground points, the fitting height error of winter wheat in these two days has increased compared with May 27.

[0165] The complete height estimation method proposed in this embodiment can achieve high accuracy during the stem extension period and thereafter, and the overall root mean square error of the last four dates is 4.9 cm. It can effectively handle noisy point cloud data and missing ground points.

[0166] In order to quantify the influence of ellipsoid axis ratio and threshold on the results in the density consistency denoising method, this example selected all stations without missing ground points on six dates and used method II for testing. The axis ratio ranged from 5 to 30, and the threshold ranged from 40 to 240. The RMSE of winter wheat height corresponding to these parameters is as follows: Fig. 9 As shown, Fig. 9 Estimation errors of winter wheat height under different parameter combinations on six dates.

[0167] According to the relationship between the RMSE of the estimated height on different dates and the parameter changes, 160 was selected as a fixed threshold, and different axis ratios were used according to the different growth stages of winter wheat. The height values ​​of the point cloud data of each date were clustered by k-means, clustered into two categories, and then the average values ​​of different categories were subtracted to obtain a height difference. This height difference can roughly reflect the growth stage of winter wheat. The height difference and the optimal axis ratio distribution of each date are shown in Figure 1. Fig.10 As shown, Fig.10 The height differences of clusters of six dates and their corresponding optimal axis ratios have a high correlation. Therefore, different parameters can be selected for denoising based on the height differences of clusters of different dates.

[0168] Among them, the comparison of point cloud data filtering effects is as follows:

[0169] In crop height estimation, UAV platforms are highly flexible and convenient, and can cover large areas of farmland in a short time and obtain high-resolution image data. The SfM algorithm can reconstruct high-density three-dimensional point cloud data from multi-view images, thereby accurately reflecting the geometric morphology of crops, and has a high degree of automation. Compared with traditional ground measurement or lidar scanning, UAVs are cost-effective, easy to operate and quickly deployed, which makes them widely popular in agricultural monitoring. However, in point cloud data processing, noise problems are inevitable. Common point cloud data denoising methods include statistical filtering, radius filtering, clustering-based methods, etc. These filtering methods can either remove isolated outliers in point cloud data, but have poor removal effects on some larger outlier clusters, or can cluster some outlier point cloud data clusters, but have poor removal effects on outlier point cloud data clusters connected to canopy point cloud data and ground point cloud data.

[0170] The denoising method based on density consistency proposed in this embodiment can achieve good results in removing isolated abnormal point cloud data and abnormal point cloud data clusters. Fig.11 As shown, Fig.11 This is a schematic diagram of the denoising results using different methods. Fig.11 (a) Point cloud data before denoising; (b) denoising result of statistical filtering (n=6, σ=1); (c) density consistency denoising result. Statistical filtering can correctly identify isolated outliers below the ground point cloud data, but it is not effective in removing abnormal point clusters above the canopy, and the misjudgment of canopy point cloud data is more serious. In contrast, density consistency denoising can remove isolated outliers below the ground and abnormal point clusters above the canopy point cloud data. Although it will cause a certain degree of misjudgment of the canopy tip point cloud data, the canopy integrity is generally good.

[0171] In addition, due to the missing ground points caused by the interlaced occlusion of the vegetation canopy leaves, obvious errors will also be caused in the estimation of crop height. The method of fitting missing ground points proposed in this embodiment can solve this problem well. First, it is necessary to extract ground point cloud data. The common method of separating vegetation and soil is to calculate the overgreen index EXG of a certain point and judge the type of the point by a threshold. The threshold is usually determined automatically using the Otsu method. However, since the Otsu method is only applicable to bimodal graphs, it has poor classification effect on unimodal graphs. The ground point cloud data extracted by the threshold calculated by Otsu will contain a large amount of canopy point cloud data. The histogram of the overgreen index EXG calculated in this embodiment is mostly a unimodal graph, and Otsu is not applicable to determine the threshold.

[0172] Therefore, this embodiment uses a fixed threshold combined with the overgreen index EXG to preliminarily extract ground points from the point cloud data, and the extracted ground points will contain a small number of vegetation points. These vegetation points can be removed using the spatial information (xyz coordinates) of the point cloud data, and this process is implemented using RANSAC fitting planes. Because of the missing ground points, a 2m×2m area may contain only a small number of ground points, which will cause the plane fitted by RANSAC to be located near the canopy or have a large inclination angle. Therefore, the missing ground points are fitted on a 10m×10m sub-area with sufficient ground points to ensure that the fitting plane is in the correct position. In this way, ground points can be correctly extracted from point cloud data that is severely obstructed by the canopy, ensuring that the canopy point cloud data is not used as ground points when estimating the height, resulting in a serious underestimation of the height of winter wheat.

[0173] The method proposed in this embodiment can extract some ground points from a wider range for interpolation to fit the height of the missing ground points below the canopy. The accuracy of crop height estimation is greatly improved. In addition to winter wheat, after adjusting the parameters according to the canopy structure characteristics of different crops, the height estimation method proposed in this embodiment can also be applied to other crops, such as rice and corn. The UAV combines the SfM algorithm to generate point cloud data, making it efficient, economical and simple to obtain crop height information on a large scale, and can be widely used to help farmers monitor their crops and make real-time decisions on farm management.

[0174] The winter wheat height estimation method proposed in this embodiment can reduce the impact of point cloud data noise and missing ground points on height estimation. When the drone acquires winter wheat images, due to the presence of wind, the posture of the same wheat plant in different images may be inconsistent, resulting in matching errors in the SfM algorithm, causing the generated point cloud data to contain a large amount of noise. And this noise increases with the growth of winter wheat. In addition, after the stem extension period, the winter wheat leaves intersect with each other and block the ground, resulting in missing ground points in the generated point cloud data. These factors will significantly reduce the accuracy of winter wheat height estimation.

[0175] According to the density difference between normal point cloud data and noise point cloud data, the density consistency denoising method is applied to remove isolated abnormal points and abnormal point clusters that deviate from the canopy point cloud data and the ground point cloud data. Then, the ground point cloud data is extracted by the overgreen index EXG and the random sampling consistency algorithm. Linear interpolation is performed based on the extracted ground point cloud data to fit the missing ground points to estimate the height of winter wheat.

[0176] The root mean square error RMSE and mean absolute error MAE of the complete height estimation method proposed in this embodiment are 7.81cm and 6.18cm respectively, and the correlation coefficient R2 between the estimated plant height and the actual result is 0.904, reaching a significant level. The results show that this method can effectively improve the accuracy of crop height estimation. After wheat enters the stem extension period, the noise and ground point missing situation are aggravated, making this improvement effect more obvious.

[0177] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A crop height extraction method based on density consistency denoising, characterized by: The following steps are involved: S1: Determine the study area and obtain ground point cloud data; S2: Apply density consistency denoising method to remove noise from point cloud data; S3: Apply the overgreen index EXG and random sampling consistency algorithm to extract ground points from point cloud data; S4: Fitting missing ground points based on the extracted ground points by linear interpolation to estimate crop height.

2. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: The acquisition of the ground point cloud data is carried out according to the following steps: Set the time and location of plant growth data collection according to the geographical location of the study area; Set the data collection format; Collect and store point cloud data.

3. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: The density consistency denoising method in step S2 specifically includes the following steps: S21: Neighborhood statistics: For each point in the point cloud data, set an ellipsoidal neighborhood with a change in the major and minor semi-axis of the point; record the number of other point cloud data in each neighborhood in turn to obtain a set of data points (x i ,y i ),i=1,2,...,20, where x i is the length of the major semi-axis of the ellipsoid, y i is the number of point cloud data within the ellipsoid; S22: Polynomial fitting: Apply the least squares method to fit the obtained data to a second-order polynomial y=ax 2 +bx+c, its least squares solution is: in, The second-order polynomial was chosen because it can better capture the trend of the number of domain points changing with radius; S23: Outlier judgment: judge whether a point is an outlier by analyzing the quadratic term coefficient of the second-order polynomial, and judge whether the point is noise according to the size and positive and negative sign of the quadratic term coefficient a.

4. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: In step S3, ground points in the point cloud data are extracted according to the following steps; S31: Calculate the overgreen index EXG of each point cloud data; S32: extracting data points whose over-green index EXG is less than a threshold; S33: applying the RANSAC algorithm to fit the plane based on the extracted point cloud data; S34: Remove data points whose distance from the plane is greater than a preset threshold to obtain ground point cloud data.

5. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: In step S4, the missing ground points are fitted by linear interpolation based on the extracted ground points to estimate the crop height. The specific steps are as follows: S41: Divide the entire xOy plane into a number of rectangular grids; S42: taking the lowest point among the extracted ground points as the true ground point of each grid; S43: performing linear interpolation on the grid that does not contain the ground point, and using the interpolation result as the real ground point of the grid; S44: Subtract the highest point of the canopy in each rectangular grid from the actual ground point to obtain the crop height in the grid.

6. The crop height extraction method based on density consistency denoising as claimed in claim 4, characterized in that: The overgreen index EXG is calculated according to the following formula: EXG=2G–BR in, r, g, and b are the grayscale values ​​of the red, green, and blue bands in the RGB image, respectively.

7. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: The preset threshold value adopts a fixed threshold value.

8. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: The abnormal value judgment in step S23 is performed by using the quadratic term coefficient a. When a is less than a preset threshold, the point is considered to be noise, otherwise it is considered to be a normal point.

9. The crop height extraction method based on density consistency denoising as claimed in claim 1, characterized in that: The point cloud data in step S1 is generated by a structure from motion (SfM) algorithm.

10. A crop height extraction system based on density consistency denoising, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images