Fresh corn maturity prediction method based on UAV remote sensing and deep learning

Abstract

The present invention discloses a method for predicting the maturity of fresh corn based on drone remote sensing and deep learning. The method uses deep learning technology to identify corn ears, and combines vegetation index extracted from drone remote sensing data and ground artificial sample measured values ​​to construct a sugar content and water content estimation model of fresh corn ears, which is used to predict the maturity of fresh corn plants in the whole field. The random forest model established by the present invention has strong robustness and can be adapted to a variety of field conditions. The present invention has low requirements for sensors, which greatly reduces the cost of users purchasing sensors.

Description

Technical Field

[0001] The present invention belongs to the field of agricultural automation, and specifically relates to a method for predicting the maturity of fresh corn, and in particular to a method for predicting the maturity of fresh corn based on unmanned aerial vehicle remote sensing and deep learning. Background Art

[0002] The maturity of fresh corn affects the taste and nutritional value of fresh corn, and is related to the market value of fresh corn. The water content and sugar content of fresh corn are important reference standards for its maturity. Therefore, constructing a fresh corn maturity prediction model can provide decision-making for the picking period of fresh corn.

[0003] The application of UAV remote sensing technology can achieve low-cost, high-efficiency and non-destructive prediction of the sugar and water content of fresh corn. At present, there are few studies on the judgment of the harvest period of fresh corn, especially in the extraction of the relevant characteristics of the female and male ears of fresh corn. Almost all of them are based on the male ears, and the yield is mostly estimated based on the male ears. In addition, the male ears of fresh corn are on the upper part of the corn, but their changes are not as sensitive as the female ears, and they cannot accurately reflect the maturity of fresh corn. In previous studies, no specific solutions were proposed for the harvest period of fresh corn.

[0004] This method uses a multi-rotor UAV platform equipped with a high-definition SLR camera to capture high-resolution images of fresh corn, and then uses deep learning and machine learning to extract and model them respectively, so as to achieve the purpose of predicting the harvest period of fresh corn. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In order to harvest fresh corn at the right time and ensure the highest economic value, the present invention provides a fresh corn maturity prediction method based on UAV remote sensing and deep learning, which realizes the accurate prediction of the fresh corn harvest period.

[0007] (II) Technical solution

[0008] In order to solve the above problems, the present invention provides the following technical solutions, and provides a method for predicting the maturity of fresh corn based on drone remote sensing and deep learning, as follows.

[0009] A method for predicting the maturity of fresh corn based on drone remote sensing and deep learning, characterized in that it comprises the following steps:

[0010] Step S1, using an unmanned aerial vehicle platform equipped with a high-definition digital camera to collect high-throughput data of fresh corn crop plants in the middle and late stages of heading, so as to obtain continuously changing dynamic phenotypic data;

[0011] Step S2, manually collecting a number of samples in the field to obtain measured values ​​of water content and sugar content of fresh corn;

[0012] Step S3, using a deep learning neural network to accurately identify the female ear region of fresh corn based on the image obtained in step S1;

[0013] Step S4, extracting the DN value of the female ear area identified in step S3, performing radiation calibration processing, and converting the DN value into the reflectivity of the outer surface of the atmosphere;

[0014] Step S5, calculating and obtaining the relevant color vegetation index based on the reflectance result;

[0015] Step S6, for the sample objects collected in step S2, correlation analysis is performed on the relevant color vegetation index and the artificially measured sugar content and water content, and the color vegetation index with good correlation performance is selected;

[0016] Step S7, using the random forest method to construct a prediction model for the sugar content and water content of the fresh corn ear using the selected color vegetation index;

[0017] Step S8, based on the sugar content and water content prediction model of fresh corn female ears, the relevant color vegetation index obtained in S5 is used as input to obtain the predicted values ​​of sugar content and water content of the whole field plants, and the maturity of fresh corn is evaluated by the predicted values.

[0018] More specifically, the data collection in step S1 is continuous large-area collection, covering the middle and late stages of the color change of the corn ear.

[0019] More specifically, in step S3, a deep learning neural network U-Net is used to perform image segmentation on the female ear of fresh corn.

[0020] More specifically, when using the deep learning neural network U-Net to perform image segmentation on fresh corn female ears, a 256×256 image is input, a convolutional layer with a window shape of 3×3 is used, followed by a maximum pooling layer with a stride of 2 and a window shape of 2×2; the convolutional layer keeps the height and width of the input unchanged, while the pooling layer halves it.

[0021] More specifically, the color vegetation index calculated in step S5 includes but is not limited to one or more of the following vegetation indices: greenness vegetation index GVI, normalized yellowness vegetation index NDYI, normalized difference vegetation index NDI, green leaf area index GLA, super red index EXR, R component R*, G component G* and B component B*.

[0022] More specifically, in step S7, the random forest algorithm in machine learning is used, the number of trees constructed is 100, and the number of input features when splitting each node of the decision tree is 50.

[0023] More specifically, in step S8, the image data of any time and area in the middle and late stages of heading acquired by the drone is used to extract the relevant color vegetation index to obtain the predicted values ​​of the sugar content and water content of the target fresh corn female ear.

[0024] (III) Beneficial effects

[0025] Based on the inventor's many years of technical accumulation and research and development in this field, the present invention proposes a method for predicting the maturity of fresh corn based on UAV remote sensing and deep learning. This method uses remote sensing data combined with deep learning and machine learning to predict the optimal harvest period of fresh corn.

[0026] Compared with the existing technology, the invention has the following technical advantages: (1) The phenotypic data obtained by the drone is used to extract the color vegetation index of the female ear area of ​​fresh corn, and the prediction model of the sugar content and water content of fresh corn is constructed by combining the random forest method. The model is more accurate than the model constructed by using the data of the male ear area of ​​fresh corn, and can better predict the maturity of fresh corn. (2) The female ear area of ​​fresh corn is obtained on the high-definition image obtained by the drone. The invention combines the method of deep learning neural network to obtain accurate and ideal results. Compared with manual visual interpretation to outline the ROI area, this method saves more time and has higher recognition accuracy. (3) The resolution requirement for the image is not high. At the appropriate resolution, similar and ideal results can still be obtained. Therefore, for remote sensing data collection using drones, it can save users the cost of purchasing multispectral cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flow chart of the method of the present invention

[0028] Figure 2 Regression analysis of sugar content and water content of fresh corn female ear and male ear

[0029] Figure 3 Comparison of the predicted value and measured value of the sugar content prediction model constructed for fresh corn female ears

[0030] Figure 4 Comparison of the predicted value and measured value of the moisture content prediction model constructed for fresh corn female ears DETAILED DESCRIPTION

[0031] The present invention adopts a technical solution to solve the technical problem, and provides a method for predicting the maturity of fresh corn based on drone remote sensing and deep learning. The method flow chart of the method is as follows: Figure 1shown.

[0032] A method for predicting the maturity of fresh corn based on drone remote sensing and deep learning, characterized in that it comprises the following steps:

[0033] Step S1, using an unmanned aerial vehicle platform equipped with a high-definition digital camera to collect high-throughput data of fresh corn crop plants in the middle and late stages of heading, so as to obtain continuously changing dynamic phenotypic data;

[0034] Step S2, manually collecting a number of samples in the field to obtain measured values ​​of water content and sugar content of fresh corn;

[0035] Step S3, using a deep learning neural network to accurately identify the female ear region of fresh corn based on the image obtained in step S1;

[0036] Step S4, extracting the DN value of the female ear area identified in step S3, performing radiation calibration processing, and converting the DN value into the reflectivity of the outer surface of the atmosphere;

[0037] Step S5, calculating and obtaining the relevant color vegetation index based on the reflectance result;

[0038] Step S6, for the sample objects collected in step S2, correlation analysis is performed on the relevant color vegetation index and the artificially measured sugar content and water content, and the color vegetation index with good correlation performance is selected;

[0039] Step S7, using the random forest method to construct a prediction model for the sugar content and water content of the fresh corn ear using the selected color vegetation index;

[0040] Step S8, based on the sugar content and water content prediction model of fresh corn female ears, the relevant color vegetation index obtained in S5 is used as input to obtain the predicted values ​​of sugar content and water content of the whole field plants, and the maturity of fresh corn is evaluated by the predicted values.

[0041] More specifically, the data collection in step S1 is continuous large-area collection, covering the middle and late stages of the color change of the corn ear.

[0042] More specifically, in step S3, a deep learning neural network U-Net is used to perform image segmentation on the female ear of fresh corn.

[0043] More specifically, when using the deep learning neural network U-Net to perform image segmentation on fresh corn female ears, a 256×256 image is input, a convolutional layer with a window shape of 3×3 is used, followed by a maximum pooling layer with a stride of 2 and a window shape of 2×2; the convolutional layer keeps the height and width of the input unchanged, while the pooling layer halves it.

[0044] More specifically, the color vegetation index calculated in step S5 includes but is not limited to one or more of the following vegetation indices: greenness vegetation index GVI, normalized yellowness vegetation index NDYI, normalized difference vegetation index NDI, green leaf area index GLA, super red index EXR, R component R*, G component G* and B component B*.

[0045] More specifically, in step S7, the random forest algorithm in machine learning is used, the number of trees constructed is 100, and the number of input features when splitting each node of the decision tree is 50.

[0046] More specifically, in step S8, the image data of any time and area in the middle and late stages of heading acquired by the drone is used to extract the relevant color vegetation index to obtain the predicted values ​​of the sugar content and water content of the target fresh corn female ear.

[0047] In order to better explain the technical solution of the present invention, taking corn female ears as specific application objects, the present invention obtained the following results:

[0048] (1) The model construction effect of using fresh corn pistils is better than that of stamens

[0049] Based on the stamens and pistils extracted by the deep learning U-Net segmentation network method proposed in the present invention, the selected vegetation index was calculated and then regression analysis was performed with the sugar content and water content. It was found that the determination coefficient R of fresh corn stamens was 2 Generally lower, such as Figure 2 As shown, no fresh corn pistil performed well.

[0050] (2) Prediction effect of fresh corn maturity

[0051] Based on the method proposed in the present invention, the optimal harvest period of fresh corn is predicted by the sugar content and water content of the pistil of fresh corn, such as Figure 3 As shown in Figure 2, the correlation coefficient R of the sugar content prediction model is 0.902 and the standard root mean square error NRMSE is 0.121; Figure 4 As shown, the correlation coefficient R of the moisture content prediction model is 0.776, and the standard root mean square error NRMSE is 0.116. It can be found that the present invention can stably predict the optimal maturity period of fresh corn.

[0052] (3) Application in corn planting areas in Hannan District, Wuhan

[0053] Based on the operation flow of the present invention, the field application was carried out in the self-planted corn field area (30.319°N, 113.961°E) of Dengxi Village, Hannan District, Wuhan City. It was carried out from June 1, 2021 to June 21, 2021, and the drone image data of the middle and late stages of fresh corn heading were collected, and the sugar content and water content of the target corresponding to the drone were measured at the same time. The sugar content and water content prediction model of the fresh corn in the area was constructed, and the sugar content and water content prediction values ​​of the fresh corn in the target area were successfully obtained. The maturity of the fresh corn was accurately reflected by the height of the predicted value, and the optimal harvest period was obtained.

[0054] The specific examples described in the application are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific examples described in the present invention, or replace them in a similar manner, but they will not deviate from the spirit of the present invention or exceed the scope defined by the attached claims.

Claims

1. A method for predicting the maturity of fresh corn based on drone remote sensing and deep learning, characterized in that: The steps include: Step S1, using an unmanned aerial vehicle platform equipped with a high-definition digital camera to collect high-throughput data of fresh corn crop plants in the middle and late stages of heading, so as to obtain continuously changing dynamic phenotypic data; Step S2, manually collecting a number of samples in the field to obtain measured values ​​of water content and sugar content of fresh corn; Step S3, using a deep learning neural network to accurately identify the female ear region of fresh corn based on the image obtained in step S1; Step S4, extracting the DN value of the female ear area identified in step S3, performing radiation calibration processing, and converting the DN value into the reflectivity of the outer surface of the atmosphere; Step S5, calculating and obtaining the relevant color vegetation index based on the reflectance result; Step S6, for the sample objects collected in step S2, correlation analysis is performed on the relevant color vegetation index and the artificially measured sugar content and water content, and the color vegetation index with good correlation performance is selected; Step S7, using the random forest method to construct a prediction model for the sugar content and water content of the fresh corn female ear using the selected color vegetation index; Step S8, based on the sugar content and water content prediction model of fresh corn female ears, the relevant color vegetation index obtained in S5 is used as input to obtain the predicted values ​​of sugar content and water content of the whole field plants, and the maturity of fresh corn is evaluated by the predicted values.

2. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 1 is characterized in that: The data collection in step S1 is continuous and large-area collection, covering the middle and late stages of the color change of the corn ear.

3. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 1, characterized in that: In step S3, the deep learning neural network U-Net is used to perform image segmentation on the fresh corn ear.

4. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 3 is characterized in that: When using the deep learning neural network U-Net to perform image segmentation on fresh corn female ears, a 256×256 image is input, a convolutional layer with a window shape of 3×3 is used, followed by a maximum pooling layer with a stride of 2 and a window shape of 2×2; the convolutional layer keeps the height and width of the input unchanged, while the pooling layer halves them.

5. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 1, characterized in that: The color vegetation index calculated in step S5 includes but is not limited to one or more of the following vegetation indices: greenness vegetation index GVI, normalized yellowness vegetation index NDYI, normalized difference vegetation index NDI, green leaf area index GLA, super red index EXR, R component R*, G component G* and B component B*.

6. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 1, characterized in that: In step S7, the random forest algorithm in machine learning is used, the number of trees constructed is 100, and the number of input features when splitting each node of the decision tree is 50.

7. The method for predicting the maturity of fresh corn based on drone remote sensing and deep learning according to claim 1, characterized in that: In step S8, the image data of any time and area in the middle and late stages of heading acquired by the drone is used to extract the relevant color vegetation index to obtain the predicted values ​​of the sugar content and water content of the target fresh corn female ear.

Images